UNIX/Linux - awk - displaying unxpected ! character when viewed in mail; but not in actual file

Greetings Experts,
I am on AIX using ksh; I am processing the input files and generating a awk_output.txt file using AWK. By reading that awk_output.txt file, I am building a output.html file which is cat and then fed to /usr/sbin/sendmail .When the shell script is triggered through command line, the script processes the multiple files at the given location and then builds the awk_output.txt file using awk Then by reading the contents of the awk_output.txt through while loop, I am building the output.html file with respective html tags. Then /usr/sbin/sendmail is invoked to read the contents of output.html file using echo `cat output.html` and send mail. I know without sample script/file it is really difficult to point out the exact issue. But I can't share the script/file as it is on remote machine.

Issue: When the mail is received, I see that there is extra ! {!followed by a space} at random locations at 5th line and 11th line. ( It occurs on the random line numbers after every re-run which deletes the same file and re-creates it )
When I open the same output.html file directly in the Unix server ( which opens in Internet Explorer), I am seeing the data as expected i.e., without ! I am really frustrated with this. Please note that when the number of characters in the output.html file is low (around 200 characters; 12 rows with 3 columns; column1 length=10; column2_len=3; column3_len=3) this issue doesn't exist. But when the output.html file contains more number of characters (12 rows with 5 columns; column1 length = 20; column2_len=30; column3_len=3; column4_len=3; column5_len=5) the issue creeps up. Assumption of any embedded non-printable character in the script is overruled as we pass 3 different arguments to the script 3 times and the ! occurs at different locations.
Eg contents when opened the output.html file in Unix server which doesnot have issue i.e., without ! :

account_info_file1|account_number,business_date1|yes|yes
account_info_file2|account_number,business_date2|yes|no
....
account_info_file5|account_number,business_date5|yes|yes
...
account_info_file12|account_number,business_date12|no|no

Eg contents seen through outlook mail which has issue i.e., ! occurs at random locations

account_info_file1|account_number,business_date1|yes|yes
account_info_file2|account_number,business_date2|yes|no
....
account_info_file5|account_nu! mber,business_date5|yes|yes
...
account_info_file12|account_number,busines! s_date12|no|no

As you can see, when the actual output.html file is opened in Unix server, it opens in Internet Explorer and I don't see any ! symbols. But the same file is cat and sent through mail and then I see those ! symbols at random positions at about 3 times. I have verified the octaldump of the lines that had ! in between them in the output.html file and I don't see any issues with it. Can you please help to understand and resolve this. Thank you for your time.

Why are you using

echo `cat output.html`

rather than

cat output.html

The first leaves all kinds of characters open to interpretation by the shell ..

How does that work with sendmail? What do you do exactly?

1 Like

Excellent Scrutinizer. You saved me :slight_smile: There was already an existing script written by someone. So, re-used it by modifying the needed ones. Your suggestion worked perfectly. Again, thank you.