I need to read contents of directory and create a list of data files that match a certain pattern and process by renaming it and calling a existing .ksh script then archiving off to file another directory. Any suggestions or samples u could point me to on using .ksh perl or other to process through the list of data files? What would you recommend to strip off a suffix from a data file?. tia
test message
Please supply input files, data and desired results. It is much easier to understand from data rather than describing the situation.
To handle filename issues, do a >man basename
not clear exactly what you are trying to do.
file names:
BTIME_ACTUAL_WORK_HRS_wk_45_to_46.csv
BTIME_ACTUAL_WORK_HRS_wk_46_to_47.csv
...
sample processing:
1) strip off/rename each data file
e.g. BTIME_ACTUAL_WORK_HRS_wk_45_to_46.csv to
BTIME_ACTUAL_WORK_HRS.csv
2) call existing .ksh script (it will reference BTIME_ACTUAL_WORK_HRS.csv) each time
3) move/copy of BTIME_ACTUAL_WORK_HRS_wk_45_to_46.csv moved to /archive directory.
4) process next file
Ensure each file is only processed once
Copy of originally named file is archived
Process only non-zero byte files or wc > 3
Assumes job will be called from scheduler when files are ready/waiting
tia
I would humbly suggest you fix the existing ksh script to accept a file name argument. This is begging for ending up in a situation where you are not sure which file you have processed.
for f in BTIME_ACTUAL_WORK_HRS_*.csv; do
cp -f "$f" BTIME_ACTUAL_WORK_HRS.cvs
horrendous_legacy.ksh --majestic --righteous # FIXME: make it accept name of csv file
mv "$f" archive/ # assume it's not really /archive/
done
Hate to break this to you, but error handling left as an important exercise.
I agree but the existing .ksh calls an Informatica process that references a specific flat file for a source. It doesn't allow a dynamic flat file source name so I have to replace the existing one it reads...