Omit Blank Lines while comparing two files.

Shell Programming and Scripting
1

Hello All,

I am writting file comparison Utility and I have encountered such a senario where there are 2 files such as follows-

1#!/usr/local/bin/python
2 import gsd.scripts.admin.control.gsdPageEscalate
3.gsd.scripts.admin.control.gsdPageEscalate.main()

1 #!/usr/local/bin/python
2
3 import gsd.scripts.admin.control.gsdPageEscalate
4
5 gsd.scripts.admin.control.gsdPageEscalate.main()

here the lines of code are exactly the same except for the mismatch in extra blank lines in the 2nd file. I have tried both diff and cmp commands with various options and also tried the sum command , but both these files are never shown identical. Can anyone please tell me if its possible to consider them equal by ingoring the blank lines/white spaces while comparing them?

2

I can give you a suggestion...
strip off all the blank lines using grep or sed or awk
and then compare.

3

Try this

comm -13 a1 a2 | sed '/^$/ d'

4

Guys, I just noticed all the files in 2nd directory have control-M characters ^M in them which is causing all files in 2nd directory to be shown as different. Is there a way I can write a script that will loop through all the files in the 2nd directory and remove the control characters from the file if they exits and save the file and then do the comparison.

5

Try:

for i in *
do
   sed 's/^M//g' $i > _tmp."$i"
   mv _tmp."$i" $i
done
6

Try the below to remove ^M characters:

perl -i -ne ' s/^M//g; print; *

or,

perl -e 'while(<>) { chop; print; }'  filename
7

If you have real carriage return characters (Octal 015), the sed from rakeshawasthi will not work.

One way of removing spurious carriage return characters:

cat filename | tr -d "\r" > newfilename

BTW: The problem you see usually comes from transferring text files from MSDOS to Unix with binary mode FTP (not text mode ftp). The line terminator in MSDOS is cr/lf whereas in unix it is just lf. A text mode ftp will translate the line terminator but in some versions of ftp will also expand tabs and corrupt extended ascii characters (which you may not want)

To see the line terminators:

sed -n l filename

cr displays as a $
lf displays as \r
tab displays as \t
8

Thanks a lot guys!!! I tried Rakeshawasthi's script somehow that din't seem to work but when I tried methyl's code it worked!!! all the carriage return characters were gone!!! thanks a lot guysss!!!!!!!!!!!!!!!

9

It depends on how you typed ^M...if you have just copy-paste the code, it wont work...
You have to use Ctrl-V and then press m in command mode.

10

I am using Edit-plus for writting the shell script and then uploading it on to the unix box...any clue as to how to type it out in Windows environment... using a notmal editor such Textpad or Edit Plus?

11

How you upload is the important factor. You need a method which will convert from MSDOS format to unix. If you do not have suitable software you need to convert the file on the unix server.

You will get better response from technicians if you provide the basic facts. In this case it would help to know which two Operating Systems are involved and what software you use to transfer files.