Combining files(every 15 min) as one file(hourly)

Shell Programming and Scripting
1

Hello,

My system is generating two files every 15 minutes and file names are given automatically as below. (98,99,89,90 are the sequence numbers)

File1_09242013131016_000000098
File1_09242013131516_000000099
File2_09242013124212_000000089
File2_09242013124712_000000090

I want to combine these files and generate only one file in every one hour as below.

File3_09242013120000_00000001
File3_09242013130000_00000002

How can i achieve this?

2

Try this... Test it first and then try on actual files...

ls | awk '{x=$0;sub(/...._[0-9]+$/,"",x); if(!f[x]){f[x]=x"0000_"sprintf("%08d", ++seq)} print "cat "$0">>"f[x]}' | bash

This will not delete the files, if you do want to delete it, append the rm command to the print statement.

--ahamed

3

Thanks Ahamed,

That script didnt combine the files. This created seperate files by hourly based.
I prefer to combine "15 minutes generated two different files" "hourly in one file"

4

You wanted File1_09242013131016_000000098 File1_09242013131516_000000099 to be combined to File3_09242013130000_00000002 and so on right?

--ahamed

5

There are two types of files named File1 and File2. File1 and File2 are being generated every 15 minutes interval.

Hourly, I want to combine them and generate only one file like named File3.

6

Just adding my 2 cents here.
Say you have the files a1.txt, b1.txt, c1.txt, d1.txt.
How about you try writing a script that performs the following operations:
1) Create a new file, say 1 and append the contents of each file into it:

cat a1 b1 c1 d1 > 1

2) Leave the 1 file intact and delete the other files:

rm *.txt

(or you can move them to a different directory)
Repeat the process for as long as you need, creating files with .txt extension every 15 minutes, then combining them into a file without extension, and then deleting the .txt files.
Hope it helps.

7

Try

for i in 1 2 ; do for j in File$i*; do TGT=${j%????_*}0000_0000000$i; TGT=${TGT/[12]/3}; done; cat File$i* > $TGT; done

If files come in regularly, and you run this regularly, you'll need to either remove the quarter hourly files after cat ting, or make the j for loop somewhat more specific.