I'm trying to create a DataStage job and I'm basing it off an existing similar project.
The 'Input' value of the job is:
(where "/DATA/CSV_FILES" is where all the files were located)
My understanding is that for each CSV, it added the 'filename' as Column A. And then it joined all the CSV files together and created a single output.txt file.
I've copied everything that this job did - only that my CSVs are slightly different. Unfortunately when I run the job, it does not work and tells me that 'output.txt' is not in the folder. So I'm guessing I need to tweak this UNIX code a bit to suit my needs.
Problem is I don't understand it. Is anyone out there possible to give me some guidance on how to do this? (or better yet, translate it to English please? )
cd /DATA/CSV_FILES # change dir
for i in *.csv # loop for all files with .csv extension
do if [ -f "$i" ] # if regular file
then sed "s/^/$i,/" "$i" > temp # prefix every line with respective file name
sed 's/.*/&,/' temp >temp2 # suffix every line with comma
fi
cat temp2 # list the file to stdout
done > output.txt # redirect all above (= every .csv prefixed) to file
rm -f temp* # remove tmp files
Here is a similar variation. I've taken the liberty to edit some.
cd /DATA/CSV_FILES # Get in correct directory
rm -f output.txt # Clear output file
for i in *.csv; do # For each input .csv file
if [ -f "$i" ]; then # If the file is a "regular" file (test not really needed)
sed "s/^/$i,/" "$i" > temp # Add file name as first column for all lines and write to output.txt
sed 's/.*/&,/' temp >> output.txt # Do you really want another comma at the end of the line?
fi
done
rm -f temp
)