Output to csv contains quotes

Shell Programming and Scripting
1

I borrowed scripting information from thread

and modified it for our environment to uuencode it and mail a csv to me. The issue I have is the output contains quotes that hinder the csv use appropriately.

My script looks like this

 
cd /tmp
df -Pg | awk 'BEGIN{
print "\t\t\t SPACE OF IMPORTANT FILE SYSTEMS\t\n-------------------------------------------------------------------------------\n FILE-SYSTEM\t TOTAL-SPACE\t USED-SPACE\t FREE-SPACE\t %USED\n"}
function disp(v1,v2,v3,v4,v5){
printf "%-21s\t %10s\t %13s\t %10s\t %5s \n",v1,v2,v3,v4,v5
}
NR>2&&(($3>($2*.80)&&disp($6,$2,$3,$4,$5))||($3>($2*.40)&&$3<($2*.80)&&disp($6,$2,$3,$4,$5))||($3<($2*.40)&&disp($6,$2,$3,$4,$5)))
END{print "-------------------------------------------------------------------------------\n"}' > email.csv
uuencode email.csv email.csv | /bin/mailx -s "$HOST diskusage usage" email@email.com
rm -rf email.csv

The output looks like this when pulled out of the csv

" SPACE OF IMPORTANT FILE SYSTEMS "
------------------------------------------------------------------------
" FILE-SYSTEM TOTAL-SPACE USED-SPACE FREE-SPACE %USED"
"/usr 5.00 3.37 1.63 68% "
"/var 3.00 2.35 0.65 79% "
"/tmp 3.75 0.05 3.70 2% "
"/home 2.00 1.30 0.70 65% "
"/admin 0.06 0.00 0.06 1% "
"/proc - - - - "
"/opt 3.00 1.52 1.48 51% "

If I can remove the quotes from the output, the spreadsheet works perfectly (since this is now the practice I have to do to get it to populate the rows and columns correctly.)

What can I do in the script side to remove the quotes from the end product?

I appreciate any input.

Stryker

2

Filter your quote-laden file through

sed 's;*;;g'
3

I can't see where all those quotes are added (unless uuencode does so, which would surprise me). Please post the contents of email.csv before being uuencoded and mailed.

BTW - as this is a sheer ASCII file, no need to uuencode - that's intended to be used on binary files.

4

I suppose the best reason I am using uuencode is I need an actual csv file and not just text output....and most likely am missing something very simple in my mail or mailx command that will send me a file not just text. Why did I do this? Simply because the output, no matter what I tried would not paste back into the spreadsheet "correctly, column and row specific, until after I remove the quotes. I saw a very similar issue 7 years ago on a unix2dos type conversion that remained hidden until I did the same thing.

RudiC - the very odd thing here is the only place the quotes show up is when I pull the data from the csv file and put it into notepad++. Even Notepad, wordpad and vi/vim do not show the characters. It just looks like regular ol' ascii txt.

So the output from all of them looks identical. The forum editor doesnt do it justice on formatting.

However the text pulled from the csv looks like this, still no quotes are visable

Ultimately, this is where the quotes show up.

5

Jpradley, I used the sed with no change in the results.

6

Hold it, hold it - you're talking windows now. Why don't you make it a real csv- file, using comma or semicolon as the output field separator in your awk script, run unix2dos across it to insert the \r char needed by almost all windows editors, and there you go. (You might as well try to make \r\n the output record separator)

7

No. No, RudiC. I would never disrespect this forum with a Doze problem. The only reason I have to deal with it in the first place is the final results come via email and open in Excel. I will have to verify if unix2dos is on any of the servers, predominantly they are AIX. The rest are RedHat. But as you probably are aware, in todays corporate gardens, not everything is how we used to like it to be.

But I think that as intriguing as it sounds, If I can locate one server in 450 +- that has it, you may have a good suggestion. I can run the script from there.

Have you tried the steps, like I suggested, and saw the output in Notepad++? If not, please do. It would help immensely if I can get an outside view, so that this does not look like it is environmentally the issue.

Let me know if you are up to that challenge. If not I need to locate someone who can. It also may be a new bug, that until now, no one has cared that the format is funky.

Any ways, I truly appreciate your input and advice (yes I will re-attempt to use \r\n).
Stryker

8

Can't use notepad++ as I'm not having a windows computer. And, why use that in the first place? csv is designed to be used in spread sheets and data bases, so feed it into one of those... Again, use awk (which you use anyhow) to place the comma separators and the \r\n into the file at the right places, and I guess, it will fly.

9

RudiC thank you for all of your input.

Here is the completed set of codes. Note that I have included both AIX and Linux versions. I also have screen and email output sections combined for space saving. Let me know if you have any ideas on making this more usable. Currently, in our enterprise, it is extremely difficult to incorporate cron tasks, since we use another vendor for those type tasks, which disables cron functionality. SO I just copy/paste when I need to use any part of it.

So here it is.

#!/bin/ksh
# AIX Version 1.0 by Stryker Cain.
# With assistance from Sujith Achutan
# Jeyagopal Dharmalingham Daniel Thornton and Jamie Banas
## Comment out the top df portion to only get a csv report emailed to you. ##
## Comment out the bottom df portion to only get a script output ##
## Leave as is to get both screen output and emailed csv report ##
## Future parameters will be to add input of email address ##
cd /tmp
HOST1=$(hostname)
df -Pg | awk 'BEGIN{
print "\t SPACE OF IMPORTANT FILE SYSTEMS on '$HOST1'\t\n----------------------------------------------------------------------\nFILE-SYSTEM\t TOTAL-SPACE\t USED-SPACE\t FREE-SPACE\t %USED\n"}
function disp(v1,v2,v3,v4,v5){
printf "%-10s\t %-10s\t %-10s\t %-10s\t %-6s \n",v1,v2,v3,v4,v5
}
NR>2&&(($3>($2*.80)&&disp($6,$2,$3,$4,$5))||($3>($2*.40)&&$3<($2*.80)&&disp($6,$2,$3,$4,$5))||($3<($2*.40)&&disp($6,$2,$3,$4,$5)))
END{print "----------------------------------------------------------------------\n "}'
df -Pg | awk 'BEGIN{
print "FILE_SYSTEM,TOTAL_GB,USED_GB,FREE_GB,PERCENT_USED"}
function disp(v1,v2,v3,v4,v5){
printf "%-1s,%-1s,%-1s,%-1s,%-1s\n",v1,v2,v3,v4,v5
}
NR>2&&(($3>($2*.80)&&disp($6,$2,$3,$4,$5))||($3>($2*.40)&&$3<($2*.80)&&disp($6,$2,$3,$4,$5))||($3<($2*.40)&&disp($6,$2,$3,$4,$5)))
END{print "'$HOST1'-\n"}' > $HOST1-df_All.csv
uuencode $HOST1-df_All.csv $HOST1-df_All.csv | /bin/mailx -s "$HOST1 diskusage script by Stryker Cain" "email address"
rm -rf $HOST1-df_All.csv
 
#!/bin/ksh
# Linux Version 1.0 by Stryker Cain.
# With assistance from Sujith Achutan
# Jeyagopal Dharmalingham, Daniel Thornton and Jamie Banas
## uuencode must be installed for this script (bottom half) to work. ##
## If not you have to comment out the uuencode part and change ##
## the mailx to the cat mail command ##
## Comment out the top df portion to only get a csv report emailed to you. ##
## Comment out the bottom df portion to only get a script output ##
## Leave as is to get both screen output and emailed csv report ##
## Future parameters will be to add input of email address ##
cd /tmp
HOST1=$HOSTNAME
df -Ph | awk 'BEGIN{
print "\t SPACE OF IMPORTANT FILE SYSTEMS on '$HOST1'\t\n----------------------------------------------------------------------\n FILE-SYSTEM\t TOTAL-SPACE\t USED-SPACE\t FREE-SPACE\t %USED\n"}
function disp(v1,v2,v3,v4,v5){
printf "%-10s\t %-10s\t %-10s\t %-10s\t %-6s \n",v1,v2,v3,v4,v5
}
NR>2&&(($3>($2*.80)&&disp($6,$2,$3,$4,$5))||($3>($2*.40)&&$3<($2*.80)&&disp($6,$2,$3,$4,$5))||($3<($2*.40)&&disp($6,$2,$3,$4,$5)))
END{print "----------------------------------------------------------------------\n "}'
HOST1=$HOSTNAME
df -Ph | awk 'BEGIN{
print "FILE_SYSTEM,TOTAL_GB,USED_GB,FREE_GB,PERCENT_USED"}
function disp(v1,v2,v3,v4,v5){
printf "%-1s,%-1s,%-1s,%-1s,%-1s\n",v1,v2,v3,v4,v5
}
NR>2&&(($3>($2*.80)&&disp($6,$2,$3,$4,$5))||($3>($2*.40)&&$3<($2*.80)&&disp($6,$2,$3,$4,$5))||($3<($2*.40)&&disp($6,$2,$3,$4,$5)))
END{print "'$HOST1'-\n"}' > $HOST1-df_All.csv
uuencode $HOST1-df_All.csv $HOST1-df_All.csv | /bin/mailx -s "$HOST1 UNIX diskusage script by Stryker Cain" "email address"
# cat $HOST1-df_All.csv | /bin/mail -s "$HOST1 UNIX diskusage script by Stryker Cain" "email address"
rm -rf $HOST1-df_All.csv