Need Help _ Unix Script

UNIX AIX
1

Hello,

I have one requirement. We are getting fixed length file with trailer record on it.
We need to cleanse,validate and process the file.

As part of validation we have to do duplicate file check based on file name and trailer record.

I want to add the file name as last column to file, so that I can use the filed for validation purpose.

Also I have notice that the file is coming with "^M" at the end, which my guess is windows new line character . I have to eliminate this from the file.

Please help me getting this done.

2

Please show the input you have and the output you want.

3

For starters, send the file binary mode and you will not get the windows ^M.
If you can't change that (ie client sends ascii only) then convert using either:

strings file_with_ctrl_chars > file_without_ctrl_chars

Or you could use sed

Or you could use dos2ux (not seen this in a while)

Or you could use:

tr -d "\015" < my_dos_file > new_aix_file

be careful if you go down the orad of using sed/awk scripts - you need to be sure that EVERY line has a ^M or you risk stripping off some real caharacters!

4

Perhaps a typing error: it is exactly the way round. Send the file in ASCII mode if you transfer it via FTP, not BINARY. Binary will leave the wrong line ends alone, while ASCII will translate them correctly.

Here is a sed-script, which will safely(!) remove "^M"s from the end of the lines:

sed 's/^M$//' /path/to/infile > /path/to/outfile

Enter "^M" in vi this way: in insert-mode press <CTRL>-<V> and then <ENTER>. A "^M" will appear, which is only one character (change to command mode and go over it with the cursor to see it). Do NOT enter it as 2 characters, this will change its meaning completely (and definitely not do what you want to achieve).

I hope this helps.

bakunin

5

Bakunin is correct. Ascii will leave the file to be decoded by local OS (where the local OS will correctly interpret the ascii code as EOL or CR or whatever).

I may have been sleeping when writing my previous post :slight_smile:

Also, a good time to point out the ascii character map (referenced in my earlier post, using the tr command to strip out "015") which could be used to do a whole load of file manipulations.

LINK