Remove Blank Rows

Hello All, I am having a problem in automating my UNIX Script to remove the blank spaces in between records. When I incorporate it into my script, the output file is deleting the whole content

Basically this is what I am trying to do:
File contains data like this:

12314213423,Georgia,123yh1

12987613423,Anaheim,167yu4

13399887123,Osaka,592zl8

and it needs to be like this:

12314213423,Georgia,123yh1
12987613423,Anaheim,167yu4
13399887123,Osaka,592zl8

This is the code I incorporated in the script:

tr -d ' ' <tmscadtrans> tmscadtranstemp1 --> to remove the spaces in between
grep -v "^$" tmscadtemp1> tmscadtranstemp
grep -v "^$" tmscadtranstemp> tmscadtrans --> to remove the blank line at the EOF.

The commands above worked if ran in command prompt but when I run the script it does not work and I do not know what is wrong

sed 'g/^ *$/d' inpFile

or

grep -v "^ *$" inpFile

Hi,

Using perl:

$ perl -lne 'print unless /^\s*$/' infile

Regards,
Birei

awk '!/^$/' file

or in that case:

awk '/,/' file

This removes both empty lines and lines with only whitespace

awk NF file

I got an error when I ran my scripts using

sed 'g/^ *$/d'

sed: Function g/^ *$/d cannot be parsed.

But when I double checked my code, it was enclosed in single quotes.

Here is the code:

src=/var/opt/sapbi/TV/scripts/$1
echo $src
sed 'g/^ *$/d' $src 

I tried
grep -v "^ *$" $src but it did not work.
I tried the awk commands but nothing happened to the file.

---------- Post updated at 08:32 AM ---------- Previous update was at 08:30 AM ----------

Just wondering, is there an alternative to Gunzip -f <InpFile>? Because upon checking, when the file is gunzipped, it creates those white spaces in between records. When I try to extract using 7zip, it does not add the white spaces.

If non of these work, perhaps there is something invisible on those lines. Run them through od -c to check. Perhaps a CR return character, because the file originated from a Windows platform?

In your tr -d ' ' command, add the \r :

tr -d ' \r' you could even add \t if you also want to remove <tab> stuff :

tr -d ' \r\t'

Then rerun your script and let us know

Here is what I got:

*
0632760          \r  \n   2   0   3   1   0   6   7   1   ,   7   3   0
0633000   7   ,   P   R   T   _   F   R   _   F   O   S   -   P   R   I
0633020   _   S   A   _   D   A   M   ,   D   0   0   2   0   2   0   0
0633040   0   0   6   ,   O   2   4   C   ,   P   R   T   _   F   R   _
0633060   F   O   S   ,   ,   P   R   I   _   S   A   _   D   A   M   ,
0633100   ,   .   1   ,   U   S   D   ,   7   3   0   7   ,   P   R   T
0633120   _   F   R   _   F   O   S   -   P   R   I   _   S   A   _   D
0633140   A   M   ,   D   0   0   2   0   2   0   0   0   0   6   ,   O
0633160   2   4   C   ,   P   R   T   _   F   R   _   F   O   S   ,   ,
0633200   P   R   I   _   S   A   _   D   A   M   ,   ,   .   1   ,   U
0633220   S   D   ,   1   3   ,   1   0   0   0   0   .   9   ,   0   ,
0633240   2   0   1   1   0   1   2   4
0633260
*
0634720                                                  \r  \n   2   0
0634740   3   1   4   0   1   9   ,   7   9   3   6   ,   P   R   T   _
0634760   I   T   _   V   C   E   -   P   R   I   _   S   A   _   D   A
0635000   M   ,   D   0   0   2   0   2   0   0   0   0   5   ,   O   2
0635020   4   C   ,   P   R   T   _   I   T   _   V   C   E   ,   ,   P
0635040   R   I   _   S   A   _   D   A   M   ,   ,   .   1   ,   U   S
0635060   D   ,   7   9   3   6   ,   P   R   T   _   I   T   _   V   C
0635100   E   -   P   R   I   _   S   A   _   D   A   M   ,   D   0   0

Does it mean it plugs-in an '*'? Also, what does \r \n denote? The last record only has that:

0641160
*
0642620                                          \r  \n      \n
0642636

This:

\r  \n

means carriage return / line feed (CRLF) , which is a Windows format. You need to convert the file to Unix format (LF).

can u share some part of your input file with us?

How do I code that? Here is the segment of my code where I unzip the file?

#Copy and uncompress the file

    cp -f ${SRCEFILE} ${TARGETFILE}

    resultcode=$?
    if [[ $resultcode != 0 ]]
    then
      echo "Error: Copying ${SRCEFILE} to ${TARGETFILE}"
      exit $resultcode
    else
      SourceFileSize=`wc ${SRCEFILE} | cut -d " " -f 3`
      TargetFileSize=`wc ${TARGETFILE} | cut -d " " -f 3`
      if [[ $SourceFileSize != $TargetFileSize ]]; then
        echo " File Copy failed "
        echo "Check the permission and Restart the Job"
        exit 1
      fi
     echo "File $SRCEFILE copied Successfully to ${TARGETFILE}"
     chmod 777 ${TARGETFILE}
     echo "File Permission modified "    
     gunzip -a ${TARGETFILE}

Normally gunzip -a would take care of that, but perhaps it is not supported on your platform? After the gunzip you need to do something like this:

tr -d '\r' < infile > outfile

where infile is the filename that remains after the gunzip

So encoding that, it would look like this correct?

cp -f ${SRCEFILE} ${TARGETFILE}

    resultcode=$?
    if [[ $resultcode != 0 ]]
    then
      echo "Error: Copying ${SRCEFILE} to ${TARGETFILE}"
      exit $resultcode
    else
      SourceFileSize=`wc ${SRCEFILE} | cut -d " " -f 3`
      TargetFileSize=`wc ${TARGETFILE} | cut -d " " -f 3`
      if [[ $SourceFileSize != $TargetFileSize ]]; then
        echo " File Copy failed "
        echo "Check the permission and Restart the Job"
        exit 1
      fi
     echo "File $SRCEFILE copied Successfully to ${TARGETFILE}"
     chmod 777 ${TARGETFILE}
     echo "File Permission modified " 
     tr -d '\r' ${TARGETFILE}   
     gunzip -a ${TARGETFILE}   

I think it is the other way around. The unzipped file had the \r\n, no?

After unzipping the file will probably no longer have the same name (perhaps ${TARGETFILE%.*} will give the proper name).