Process split files with same name by group

Shell Programming and Scripting
1

File examples

f17_mar_01_02_03_04_fsw1.xml
f17_mar_01_02_03_04_fsw2.xml
f17_mar_01_02_03_04_fsw3.xml

f17_feb_13_20_49_06_fsw1.xml
f17_feb_13_20_49_06_fsw2.xml
f17_feb_13_20_49_06_fsw3.xml

I have many xml files that are grouped with same file name, but are numbered from 1-to-many (fsw1..fsw2..fsw3 etc..). These example files are very short snippets of the thousands of grouped xmls� I have.

Problem: I need to process these files by filename group in a loop (basically combining them into one file per group), but I cannot seem to restart the loop every time the filename changes and it processes all the files together into one file.

Question: I need to process each group separately (fsw1..fsw2..fsw3 ) and then the next filename (fsw1..fsw2..fsw3). Is there an easy conditional way to loop on filename pattern and restart loop every time the filename changes?

Thank You

2

Try:

for first in *_fsw1.xml
do
  group=${first%_*}
  echo "Processing group ${group}"
  for file in "${group}"_fsw*.xml
  do
    echo "Processing ${file}"
  done 
done

To combine into a file, try:

for first in *_fsw1.xml
do
  group=${first%_*}
  echo "Processing group ${group}"
  for file in "${group}"_fsw*.xml
  do
    cat "${file}"
  done > "${group}.out"
done

If there are not so many files in a group that they would exceed line length limitations, then you could also try this:

for first in *_fsw1.xml
do
  group=${first%_*}
  echo "Processing group ${group}"
  cat "${group}"_fsw*.xml > "${group}.out"
done
3

Thank you! So far the very first example you gave me seems to work much better than what I had. I just put my "combine" code within your code and it separates the groups. I will do some more thorough testing and let you know if I run into an issue.

Thanks again!