script for Gzip thousands of file

Hi experts,

I have thousands of file (data file and Gziped file) in same directory like below--

bash-2.05$ pwd
/home/mmc

bash-2.05$ file PP023149200709270546
TT023149200709270546: gzip compressed data - deflate method

bash-2.05$ file PP027443200711242320
TT027443200711242320: data

i have script which i used before to UNZIP the Gzip files and send it to another directory Among those files Gzip & data files.

#!/usr/bin/sh
cd /home/mmc/
for i in `ls PP02*`
do
l=`file $i|grep gzip|wc -l`

if [ $l -ne 0 ]; then
gunzip -c $i > /home/mmc/zip/$i
fi
done

Now i need to reverse way. i.e. GZIP the files among those.

Please help me to write the script.

Isn't that quite straight forward ( or I didn't read that completely ? )

ls file* | while read file
do
gzip "$file"
done

no it is not straight forward, becuase in the same directory there are thousands of GZIP and thousands of Data file. While Gziping i need to only GZIP the data files only.

Try this

ls prefix* | while read f
do
   if ! gzip -t ${f} 2>/dev/null
   then
      gzip ${f} 
      # you might want to rename ${f}.gz to ${f} here
   fi
done

Hi

The previous time you posted about the similar script I had a hard time understanding your request, so please include all the necessary details in your request. For example what are you going to do with the created files, move them, rename them,...? More details help us to help you.

The below script tested on a Linux machine.

for i in `ls | xargs file | awk -F: '! /gzip/{print $1}'`
 do
    gzip $i
 done

#Now the files have the extension gz. If you need to move the files in a different directory afterwards:

for i in `ls | grep gz`
 do
   mv $i /home/other_dir/
 done

Glad to hear you got the script working the first time.

what is the the meaning of the abve code? mainly 2> /dev/null

Many Thanks rubionis. Below things i wanna do-

-- I will Gzip the data file in the original name of the file.
-- I do not want .gz extensions with a Gziped file.
-- After Gzip i will send those files in different directory.

Please also tell me the explanation of ur below code-

for i in `ls | xargs file | awk -F: '! /gzip/{print $1}'`

The below code can be run as a whole script. Run it from an appropriate directory, for example /home/username.

# In the beginning it's good to create a temp directory temp_dir that will contain only the gzipped files.

mkdir /home/temp_dir

cd /home/mmc
for i in `ls | xargs file | awk -F: '! /gzip/{print $1}'`
do
gzip $i
mv $i".gz" /home/temp_dir/
done

# Now cd to temp_dir to remove extension gz from files just created 

cd /home/temp_dir
for i in `ls`
do
mv $i "$(basename $i .gz)"
done

# Now if you need the just created files to be moved to your specific directory say /home/ttc, while you are still at /home/temp_dir do a mv

mv * /home/ttc

ls | xargs file | awk -F: '! /gzip/{print $1}'

ls | xargs file --> list files and execute the file command ( to find out the file type)
awk -F: '! /gzip/{print $1}' --> from the results give me only the files that are not zipped, ( ! /gzip/).

gzip will report that a file is not in gzip format. 2>/dev/null discards the standard error output.

$ gzip -t blah

gzip: blah: not in gzip format

$ gzip -t blah 2>/dev/null
$

Hi Rubionis,

Many thanks. Script runs well.

thanks Frank for the information