Request for advise on how to remove control characters in a UNIX file extracted from top command

karthikram · September 4, 2013, 2:56am

Hi,

Please excuse for posting new thread on control characters,
I am facing some difficulties in removing the control character from a file extracted from top command,
i am able to see control characters using more command and in vi mode, through cat control characters are not visible
through cat -v i am able to see the control characters.

i tried with sed , tr -dc, od command (not sure how to handle this) nothing worked and thought of reading line by line and removing control characters but not sure how to remove the control characters, Could anyone please advise how to remove the control characters.

below is my file in vi mode having control characters

^[[7m  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                         ^[[m^O^[[K
^[[m^O12759 aaaa_a    16   0  100m 3188 2616 S  0.0  0.0   0:00.23 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O24435 aaaa_a    16   0  100m 3200 2624 S  0.0  0.0   0:00.19 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O25623 aaaa_a    15   0  100m 3192 2624 S  0.0  0.0   0:00.28 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O29634 aaaa_a    16   0  100m 3204 2632 S  0.0  0.0   0:00.35 xxx aaaa_axxxxxx                                                                               ^[[m^O

Thanks,
Regards,
karthikram

Skrynesaver · September 4, 2013, 3:52am

 perl -pne 's/^[\[\d*m//g' file

Note the ^[ is a literal escape entered by hitting [CTRL]-V and then escape

On a side note top -b -n3 will run top 3 times with no control characters (mnemonic b for "batch" mode)

karthikram · September 4, 2013, 4:09am

Hi Skrynesaver,

I am getting the below error , actually i dont know perl exactly. Could you please advise.

$perl -ne 's/^[\[\d*m//g' file_name.txt
Unmatched [ in regex; marked by <-- HERE in m/^[ <-- HERE \[\d*m/ at -e line 1.

krishmaths · September 4, 2013, 4:24am

Did you try the tr option to remove control characters?

tr -d [:cntrl:] <filename >output

MR.bean · September 4, 2013, 4:51am

bash-3.2$ cat input.txt
^[[7m  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                         ^[[m^O^[[K
^[[m^O12759 aaaa_a    16   0  100m 3188 2616 S  0.0  0.0   0:00.23 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O24435 aaaa_a    16   0  100m 3200 2624 S  0.0  0.0   0:00.19 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O25623 aaaa_a    15   0  100m 3192 2624 S  0.0  0.0   0:00.28 xxx aaaa_axxxxxx                                                                               ^[[m^O
^[[m^O29634 aaaa_a    16   0  100m 3204 2632 S  0.0  0.0   0:00.35 xxx aaaa_axxxxxx                                                                               ^[[m^O

bash-3.2$ perl -pne 's/\^\[\[\d*m\^?O?//g; s/\^\[\[K//g' input.txt
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                         
12759 aaaa_a    16   0  100m 3188 2616 S  0.0  0.0   0:00.23 xxx aaaa_axxxxxx                                                                               
24435 aaaa_a    16   0  100m 3200 2624 S  0.0  0.0   0:00.19 xxx aaaa_axxxxxx                                                                               
25623 aaaa_a    15   0  100m 3192 2624 S  0.0  0.0   0:00.28 xxx aaaa_axxxxxx                                                                               
29634 aaaa_a    16   0  100m 3204 2632 S  0.0  0.0   0:00.35 xxx aaaa_axxxxxx

Skrynesaver · September 4, 2013, 5:37am

karthikram:

Hi Skrynesaver,

I am getting the below error , actually i dont know perl exactly. Could you please advise.
$perl -ne 's/^[\[\d*m//g' file_name.txt
Unmatched [ in regex; marked by <-- HERE in m/^[ <-- HERE \[\d*m/ at -e line 1.

See note on literal escape character in my original reply

karthikram · September 5, 2013, 10:11am

Hi Skrynesaver/MR.Bean/krishmaths,

Thanks all of your input , Please excuse Skrynesaver i didnt see the note top -b d 3 command worked fine, through more , cat -v and vi i didnt see control characters,

Thanks i tried tr -d [:cntrl:] also it removes the caret symbol and character like [7m,[m[K[m,[m[m , i am able to see that i hope i can remove using sed command.

I tried

perl -pne 's/\^\[\[\d*m\^?O?//g; s/\^\[\[K//g' input.txt

but didnt work , actually i am using ksh.

i tried using

perl -pne '$_ =~ s/[^[:ascii:]]//g; print;' YourTextFile

-- command worked but still [* character was available ^ was removed.
and

perl -ne '$_ =~ s/\P{isPrint}//g; print;' YourTextFile

Thanks All for your valuable inputs.

Thanks,
Regards,
karthikram

---------- Post updated 09-05-13 at 02:15 AM ---------- Previous update was 09-04-13 at 10:09 PM ----------

Hi ,

I am not sure, whether i can request for another query,

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2961 aaaa_a    15   0  100m 3196 2624 S  0.0  0.0   0:01.10 ssh aaa_a@aaaaaa
12759 aaaa_a    16   0  100m 3188 2616 S  0.0  0.0   0:00.34 ssh aaa_a@aaaaaa

I need to take this output in html format i am unable to get $12 value which is the COMMAND value , if there is space it prints the start of the word and not printing the entire.

I am unable to find a way to

print the file in pipe separted so that i can send to html
i am unable to print the file in mail , where it is printing the whole file in single line.

I tried,
2. using while loop which array nothing is working to print line by line in the mail.

and the file is containing blank space after the 12th field COMMAND.

Please advise.

Thanks,
Regards,
karthikram

---------- Post updated at 02:56 AM ---------- Previous update was at 02:15 AM ----------

Hi,

To print the top command output file in mail , i tried the below , but writing the entire file in a single line.

IFS='\n'
i=1
while read -A line; do
echo "${line[${i}]}"
i=$((i + 1))
done < top_list.txt >> mail.txt

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2961 aaaa_a 15 0 100m 3196 2624 S 0.0 0.0 0:01.10 ssh aaa_a@aaaaaa 12759 aaaa_a 16 0 100m 3188 2616 S 0.0 0.0 0:00.34 ssh aaa_a@aaaaaa

To record the top command output file in html format, the 12th field COMMAND value which has space so while appending to html more number of rows are creating instead of 12 field expected.

echo -e "<table border="0" cellpadding="0" cellspacing="0">" >> output_2.html
cat top_list.txt | head -1 > top_list.txt_temp
awk 'BEGIN{RS="\n";FS="\n";}
{
print "<TR>";
for (i = 1; i <= NF; i++)
{print "<TH width=\"100%\">" $i "</TH>";}
print "</TR>";
print "\n";
}' top_list.txt_temp >> output_2.html
echo -e "<tbody id="data">" >> output_2.html
rm -f top_list.txt_temp
more +2 top_list.txt > top_list.txt_temp
awk 'BEGIN{RS="\n";FS="\n";}
{
print "<TR>";
for (i = 1; i <= NF; i++)
{print "<TD width=\"100%\">" $i "</TD>";}
print "</TR>";
print "\n";
}' top_list.txt_temp >> output_2.html
echo -e "</tbody></table>" >> output_2.html
echo -e "<br>" >> output_2.html
echo -e "</body></html>" >> output_2.html

and 1 more thing , file is containing blank space after 12th field.

and Is there any approach to get the top command output in pipe separated file.

Please advise how i need to workaround.

Thanks,
Regards,
karthikram

---------- Post updated at 10:11 AM ---------- Previous update was at 02:56 AM ----------

Hi,

could anyone please advise, i was trying to get top command file into html where 12th field containing space , i am not sure to frame exact awk to command to accomadate the value inside 12th field instead of more fields after it.

Thanks,
Regards,
karthikram

RudiC · September 5, 2013, 4:36pm

Try

awk '. . . ; {for (i=12; i<=NF; i++) printf "%s ", $i; printf "\n"}

karthikram · September 6, 2013, 1:32am

Hi,

Thanks , command worked as expected. But in output i am getting more than 12th column i tried to merge using colspan of html
but still i am able to see more than 12 fields in html.

rm -f  output_2.html
echo -e "<html><head>" >> output_2.html
echo -e "<meta http-equiv="content-type" content="text/html\; charset=Cp1252">" >> output_2.html
echo -e "<!-- base href="http://apexdev.us.oracle.com:7778/pls/apx11w/" -->" >> output_2.html
echo -e "<style type="text/css">" >> output_2.html
echo -e "table {" >> output_2.html
echo -e "border-width:0px 0px 0px 0px;" >> output_2.html
echo -e "}" >> output_2.html
echo -e "td {" >> output_2.html
echo -e "font-family:Calibri;" >> output_2.html
echo -e "font-size:12pt;" >> output_2.html
echo -e "}" >> output_2.html
echo -e "p {" >> output_2.html
echo -e "font-family:Calibri;" >> output_2.html
echo -e "font-size:12pt;" >> output_2.html
echo -e "}" >> output_2.html
echo -e "</style>" >> output_2.html
echo -e "</head>" >> output_2.html
echo -e "<body>" >> output_2.html
echo -e "<br>" >> output_2.html
echo -e "<table border="0" cellpadding="0" cellspacing="0">" >> output_2.html
echo -e "<tbody id="data">" >> output_2.html
sed -r -i 's/\s+/ /g' top_list.txt
awk 'BEGIN{RS="\n";}
{
print "<TR>";
for (i=1; i<=11; i++)
{
printf ( "<TD>" );
printf "�";
printf ( $i );
printf "�";
printf ( "</TD>" );
}
for (j=12; j<=NF; j++)
{
printf ( "<TD colspan="10">" );
printf ( $j,"%s ");
printf ( "</TD>" );
}
printf "\n";
print "</TR>";
print "\n";
}' top_list.txt >> output_2.html
echo -e "</tbody></table>" >> output_2.html
echo -e "<br>" >> output_2.html
echo -e "</body></html>" >> output_2.html

Is there anyother way to get.

Please advise.

Thanks,
Regards,
karthikram