Split the file with renaming.

Shell Programming and Scripting
1

Hi Team,.

I am writing a script find the files which is not .gz format, split it into files which are 50000 line entries per file(using split -l 50000) then compress the smaller files using gzip and delete the original/big file.

After splitting it is creating the files in its own naming format where as I want to keep the same name with .seg1, .seg2, .seg3 so on.

I am not getting a hint to get the same naming with .seg*

could you help.

-rw-rw-r-- 1 scmpadm scmpuser 2390849170 Jan 15 10:58 activity.log.20130114
-rw-rw---- 1 scmpadm scmpuser 43977463 Jan 18 06:00 bgaa
-rw-rw---- 1 scmpadm scmpuser 44904640 Jan 18 06:04 bgab
-rw-rw---- 1 scmpadm scmpuser 48075578 Jan 18 06:07 bgac
-rw-rw---- 1 scmpadm scmpuser 50093878 Jan 18 06:09 bgad
-rw-rw---- 1 scmpadm scmpuser 51356468 Jan 18 06:14 bgae
-rw-rw---- 1 scmpadm scmpuser 49490389 Jan 18 06:20 bgaf
-rw-rw---- 1 scmpadm scmpuser 50395455 Jan 18 06:25 bgag
-rw-rw---- 1 scmpadm scmpuser 49959803 Jan 18 06:32 bgah
-rw-rw---- 1 scmpadm scmpuser 48350470 Jan 18 06:37 bgai
-rw-rw---- 1 scmpadm scmpuser 49689464 Jan 18 06:43 bgaj
-rw-rw---- 1 scmpadm scmpuser 38375424 Jan 18 06:45 bgak

activity.log.20130114
is the big file I wanted to get it as
activity.log.20130114.seg1.gz
activity.log.20130114.seg2.gz
activity.log.20130114.seg3.gz
activity.log.20130114.seg4.gz ....

2 
FIRST=1
FILE=1
SPL=50000
while [ $SPL -le `wc -l activity.log.20130114`]
do
sed -n '$FIRST,$SPLp' activity.log.20130114 > activity.log.20130114.seg{$FILE}
FIRST=`expr $FIRST + 50000`
SPL=`expr $SPL + 50000`
FILE=`expr $FILE + 1`
done
3

Hi Pikk45. Thank you for the solution.

this code does it for one file. but this folder gets files everyday.

I tried to run this code I got the following error.

./test.sh[4]: test: 0403-021 A ] character is missing.

the script should pick the file from the folder and split and zip.

let me know what should I change.

4

Try this:

split -d -l 50000 activity.log.20130114 activity.log.20130114.seg
gzip activity.log.20130114.seg*
5

I think I did not asked my query properly, and the list of the files caused an confusion.

let me reiterate.

The files list provided above are created in the dir /data/OAS/rcap/

the script should be able to pick the file one by one and split. by having the suffix originalfilename.seg** where ** could be numbers or alphabetics.

later the split files should be gzipped by removing the original file.

---------- Post updated at 04:06 PM ---------- Previous update was at 04:04 PM ----------

Hi Pikk45,.

I think I did not asked my query properly, and the list of the files caused an issue.

let me reiterate.

The files list provided above are created in the dir /data/OAS/rcap/

the script should be able to pick the file one by one and split. by having the suffix originalfilename.seg** where ** could be numbers or alphabetics.

later the split files should be gzipped by removing the original file.

Could you let me know the changes accordingly.