I am writing a code that can move and archve all the files in a directory except the latest file based on file pattern provided in a controlfile.
The filename is in the form of pattern. So basically we find the all the files of the pattern provided and archive all of them, leaving one latest file.
I am on Sun Solaris unix machine.
The control file Testcontrol.csv is a .csv(comma delimited) file that contains three fields-
directorypath,FilenamePattern,archivedir
Something like below
DirectoryPath,Filename Pattern,ArchiveDirectory
/temp,test*cmp*.txt,/archive/temp/cmp1
/data,cmp raw * carrier cleaned*.txt,/archive/temp/carrier
/data,cigna raw * client*.txt,/archive/temp/client
I have written the code below that reads the control file and perform the logic. It works fine for filenames or patterns that does not have any spaces in the control file.
But the code below does NOT work with pattern that has spaces.This is my issue. I would appreciate if you can help me out in modifying the code below to work for filename pattern with spaces.
CNTRL_FILE=Testcontrol.csv
while read rec
do
v_dirpath=`echo $rec | cut -d',' -f1`
v_filename=`echo $rec | cut -d',' -f2`
v_arch_dir=`echo $rec | cut -d',' -f3`
echo "Processing Dir is: ${v_dirpath}"
echo "Processing File is: ${v_filename}"
echo "Archive Directory is: ${v_arch_dir}"
cd ${v_dirpath}
ls -tp ${v_ProcessingFile} | tail +2 # this finds all the files that matches the file pattern except the latest file
mv `ls -tp ${v_ProcessingFile} | tail +2` ${v_arch_dir};
cd -
done < ${REF_DIR}/${CNTRL_FILE}
The mv command can't tell spaces in file names from spaces used as separators between files, so you have to escape the former by either prefixing each with a \ or by enclosing the entire file name in quotes. An alternative solution would be to mv every single file by itself, like
{
read
while IFS=, read v_dirpath v_filename v_arch_dir
do echo "Processing Dir is: ${v_dirpath}"
echo "Processing File is: ${v_filename}"
echo "Archive Directory is: ${v_arch_dir}"
ls -tp ${v_filename} |
while read FN
do echo mv $FN ${v_arch_dir}
done
done
} < ${REF_DIR}/${CNTRL_FILE}
As you can see, the unnecessary variable assignments can be avoided by reading the variables from the input file using the correct input field separator ( IFS=, ). The header line is skipped by the first read outside the loop. The echo in the second while loop is for control / debugging - remove it if happy with the output.
Notabene, the variable v_ProcessingFile is unset and thus empty in above sample code.
I tried this as suggested in the code below. This works fine for filenames with no spaces but does not work for filenames that spaces. I have pasted the code I tried and output below:
{
read
while IFS=, read v_dirpath v_filename v_arch_dir
do
echo "Processing Dir is: ${v_dirpath}"
echo "Processing File is: ${v_filename}"
echo "Archive Directory is: ${v_arch_dir}"
echo �The output file is:�
ls -tp ${v_dirpath}/${v_filename}
done
} < ./Test.csv
Output from above code:
Processing Dir is: /apps/Inb
Processing File is: Agt_Test_Ready*.txt
Archive Directory is: /apps/archive/Inb
The output file is:
/apps/Inb/Agt_Test_Ready_201508261311.txt
/apps/Inb/Agt_Test_Ready_201407161159.txt
Processing Dir is: /apps/Inb
Processing File is: Health State Carrier up*.xls
Archive Directory is: /apps/archive/Inb
The output is:
/apps/Inb/Health: No such file or directory
State: No such file or directory
Carrier: No such file or directory
up*.xls: No such file or directory
You are right - it's the combination of spaces and wildcards in one pattern that is the killer. Spaces only you could deal with by enclosing in double quotes, which in turn prevent the expansion of the wildcards. Catch 22.
Solution would be escaping every single space with a \ , and using the (deprecated, as generally dangerous) eval command:
eval ls -tp ${v_filename// /\\ }
Please make sure you know EXACTLY what is being eval ed, as it would rigorously evaluate and execute e.g. rm / ...
And, enclose "$FN" in double quotes in my above script as well...