Redirect all logs files contents into a single log file

Shell Programming and Scripting
1

Hi ,

I have a Data cleansing process which creates different log file for each step , when the process runs it creates following log files in below order:

p1_tranfrmr_log.txt
p1_tranfrmr_stats.txt
p2_globrtr_log.txt
p2_globrtr_stats.txt
p3_cusparse_log.txt
p3_cusparse_stats.txt
'
'
'
'
'
'
'
'
'
p41_drtranfrmr_log.txt
p41_drtranfrmr_stats.txt
p42_tranfrmr_log.txt
p42_tranfrmr_log.txt

I want to redirect all the contents of above 'log' and 'stats' of each process ( tranfrmr,globrtr, cusparse, drtranfrmr etc ) into a single file
in the above p* order ( p1,p2,p3.....p42)

i need a shell script which merges the contents of several file into a single file.

Thanks,
Sonu

2 
cat p*.txt >>accumulatedlog.txt
3

Hi,

I tried executing the above commands but I need a 1 line space after merging the each contents of each file,

Thanks
sonu

4

Then step through them adding a line between concatonations.

for i in p*.txt; do
   cat $i >>accumulated_logs.txt
   echo >>accumulated_logs.txt
done
5 
 
for i in p*.txt
do
    cat "$i" >> accumulatedlog.txt
    echo "" >> accumulatedlog.txt
done
6 
awk 'FNR==1 && NR!=1{print ""}1' p*.txt > accumulated_logs.txt
7

Hi,

I tried doing above steps but it is not merging in sorted order.

For eg :it is merging the contents in below random order of file

p41_drtranfrmr_log.txt
p41_drtranfrmr_stats.txt

p3_cusparse_log.txt
p3_cusparse_stats.txt

p1_tranfrmr_log.txt
p1_tranfrmr_stats.txt

p42_tranfrmr_log.txt
p42_tranfrmr_log.txt

Thanks

8 
 
ls p*.txt > files.txt
 
sort files.txt | while read file
do
    cat "$file" >> accumulatedlog.txt
    echo "" >> accumulatedlog.txt
done
9

Try...

ls -1 p*txt | awk '{f=$0;gsub(/[^0-9]/,""); print ($0==""?0:$0), f}' | sort -n | while read num file; do cat "$file"; echo ""; done > accumulatedlog.txt