Unable to unpack files with bunzip2 using while loop

maroom · January 20, 2014, 11:40pm

Hi,

I have a problem with unzipping some file.xml.bz2 files to file.xml using while loop.

all other processing on files is successfull except bunzip2.

here is my piece of code

while read i
do
bunzip2 $i
done<file.lst;

output

: No such file or directory.le file1.XML.ftpfail.bz2
: No such file or directory.le file2XML.ftpfail.bz2
: No such file or directory.le file3.XML.ftpfail.bz2
: No such file or directory.le file4.XML.ftpfail.bz2

file.lst contains the names of the file names.

" i " is detecting the file name correctly..

when i use bunzip2 filename as a single command it is working.. but in loop it is failing to detect the file and showing the error as above.

kindly suggest me with the what is the issue.

thank a lot in advance.

Akshay_Hegde · January 20, 2014, 11:54pm

Put double quote and try

bunzip2 "$i"

How is your file.lst ?

maroom · January 21, 2014, 12:09am

Tried as you said, but results negative.

same error occurred.

file.lst contains only the filenames , line by line.

the files are in -rw-r--r--
file.lst is in 777 format.

Akshay_Hegde · January 21, 2014, 12:18am

set IFS and read line

IFS=$(echo -en "\n\b")
while read i; do
          bunzip2 "$i"
done <"file.lst"

maroom · January 21, 2014, 12:27am

I got the same error.

: No such file or directory.le file1.XML.ftpfail.bz2
: No such file or directory.le file2.XML.ftpfail.bz2
: No such file or directory.le file3.XML.ftpfail.bz2
: No such file or directory.le file4.XML.ftpfail.bz2

file.lst contains

file1.XML.ftpfail.bz2
file2.XML.ftpfail.bz2
file3.XML.ftpfail.bz2
file4.XML.ftpfail.bz2

--------

1 month back we used the same simple code to do work. previously it worked now it is not working.

Akshay_Hegde · January 21, 2014, 12:42am

There might be carriage-returns can you post o/p of od -c file.lst

maroom · January 21, 2014, 1:06am

output of

od -c file.lst

0000000   Z   H   K   2   6   S   Q   L   5   2   _   T   W   1   0   5
0000020   0   _   _   _   U   P   D   A   T   E   .   X   M   L   .   f
0000040   t   p   f   a   i   l   .   b   z   2  \r  \n   Z   H   K   2
0000060   6   S   Q   L   5   2   _   T   W   1   0   5   1   _   _   _
0000100   U   P   D   A   T   E   .   X   M   L   .   f   t   p   f   a
0000120   i   l   .   b   z   2  \r  \n   Z   H   K   2   6   S   Q   L
0000140   5   2   _   T   W   1   0   5   2   _   _   _   U   P   D   A
0000160   T   E   .   X   M   L   .   f   t   p   f   a   i   l   .   b
0000200   z   2  \r  \n   Z   H   K   2   6   S   Q   L   5   2   _   T
0000220   W   1   0   5   3   _   _   _   U   P   D   A   T   E   .   X
0000240   M   L   .   f   t   p   f   a   i   l   .   b   z   2  \r  \n
0000260

Akshay_Hegde · January 21, 2014, 1:09am

As I suspected use following one

$ while IFS= read -r i; do bunzip2 "${i%$'\r'}" ; done<"file.lst"

maroom · January 21, 2014, 1:33am

Hi ,

we are getting the same error. I think you are able to see my files , how they are.

//: No such file or directory.le ZH???????L52_TW1051___UPDATE.XML.ftpfail.bz2

what might me the reason. Is therr any other way to unzip files using loop

I tried

 while read i; do
 echo $i
 echo bunzip2 "$i" >>123.sh
 chmod 777 123.sh
 123.sh
 rm 123.sh
 done < file.lst;

it is also showing the same error...

Akshay_Hegde · January 21, 2014, 1:36am

Make it simple no need of 777

You can try if your interest is just to unzip files rather than unzipping specific files in a file.lst

for i in *.bz2; do
  bunzip2 $i
done

did you try post #8 ?

maroom · January 21, 2014, 1:59am

Yes I tried #8 I got the same error.

for i in *.bz2; do
  bunzip2 $i
done

this worked for me. but there will be lot of files which are failed, we need only few specific ones which needs to reprocessed.

I can do them also, by moving separately into a folder and try the above code. but it is a time consuming one.

we need a code that can be processed on one run.

how to run shell scripts as I mentioned in the previous post (123.sh). when I run simply as 123.sh and only one command in it, it is working fine.

in loop it is not working.

Thanks a lot for being helping. I was figuring out since yesterday.

help me in finding any other ways .

I used the below one but no use

for i in `cat file.lst`; do
bunzip2 $i
done

---------- Post updated at 01:59 AM ---------- Previous update was at 01:52 AM ----------

we have to change the file names from "*.xml.ftpfail" to ".xml"

and then ftp the files to another server.

Akshay_Hegde · January 21, 2014, 2:01am

#8 should work, don't know whats going wrong I have tested like this

[akshay@aix Stuff]$ cat test
file1.XML.ftpfail.bz2
file2.XML.ftpfail.bz2
file3.XML.ftpfail.bz2
file4.XML.ftpfail.bz2

[akshay@aix Stuff]$ od -c test
0000000   f   i   l   e   1   .   X   M   L   .   f   t   p   f   a   i
0000020   l   .   b   z   2  \r  \n   f   i   l   e   2   .   X   M   L
0000040   .   f   t   p   f   a   i   l   .   b   z   2  \r  \n   f   i
0000060   l   e   3   .   X   M   L   .   f   t   p   f   a   i   l   .
0000100   b   z   2  \r  \n   f   i   l   e   4   .   X   M   L   .   f
0000120   t   p   f   a   i   l   .   b   z   2  \r  \n
0000134

[akshay@aix Stuff]$ while IFS= read -r i; do tar -zcvf "${i%$'\r'}" test ; done<"test"
test
test
test
test

[akshay@aix Stuff]$ while IFS= read -r i; do ls "$i" ; done<"test"
: No such file or directory.ftpfail.bz2
: No such file or directory.ftpfail.bz2
: No such file or directory.ftpfail.bz2
: No such file or directory.ftpfail.bz2

[akshay@aix Stuff]$ while IFS= read -r i; do ls "${i%$'\r'}" ; done<"test"
file1.XML.ftpfail.bz2
file2.XML.ftpfail.bz2
file3.XML.ftpfail.bz2
file4.XML.ftpfail.bz2

It's problem with file.lst remove carriage return and use it

You can delete carriage return like this

$ tr -d '\r' <file.lst >newfile.lst

--edit--

You can run like this

$ chmod +x scriptname
$ ./scriptname

$ sh scriptname

$ bash scriptname

$ ksh scriptname

depending on what shell you are using

Kindly use codetags

maroom · January 21, 2014, 2:22am

hi I got the issue

$  od -c file.lst
0000000   Z   H   K   2   6   S   Q   L   5   2   _   J   P   0   0   0
0000020   1   _   _   _   U   P   D   A   T   E   .   X   M   L   .   f
0000040   t   p   f   a   i   l   .   b   z   2  \r  \n   Z   H   K   2
0000060   6   S   Q   L   5   2   _   J   P   0   0   2   1   _   _   _
0000100   M   O   T   O   R   O   L   A   .   X   M   L   .   f   t   p
0000120   f   a   i   l   .   b   z   2  \r  \n   Z   H   K   2   6   S
0000140   Q   L   5   2   _   T   W   0   3   5   5   _   _   _   U   P
0000160   D   A   T   E   .   X   M   L   .   f   t   p   f   a   i   l
0000200   .   b   z   2  \r  \n   Z   H   K   2   6   S   Q   L   5   2
0000220   _   J   P   0   0   0   6   _   _   _   U   P   D   A   T   E
0000240   .   X   M   L   .   f   t   p   f   a   i   l   .   b   z   2
0000260

$ for i in `cat file.lst`; do
> bunzip2 $i
> done

I ran the above code... only one file has been unzipped that is the last file

Z   H   K   2   6   S   Q   L   5   2
0000220   _   J   P   0   0   0   6   _   _   _   U   P   D   A   T   E
0000240   .   X   M   L   .   f   t   p   f   a   i   l   .   b   z   2

for this file there is \r \n .. thats y it was processed.

can you help me in removing those hidden characters.

---------- Post updated at 02:09 AM ---------- Previous update was at 02:07 AM ----------

hi thanks a lot. let me try and let you know the results.

thank you very much for helping me..
:)

---------- Post updated at 02:22 AM ---------- Previous update was at 02:09 AM ----------

Many a thanks . Issue resolved.

Can you give some precautions , what things can be done to avoid carriage return characters appended while copying to notepad from unix editor ?

Akshay_Hegde · January 21, 2014, 2:30am

Before running program do any of the following

$ tr -d '\r' < infile > outfile

$ sed -i s/\r// infile

$ dos2unix infile

maroom · January 21, 2014, 2:35am

 Sure I will follow your instructions and lastly lots of thanks