Generating files.

Shell Programming and Scripting
1

I/P file name:- 20092008.txt
Check number of entries in i/p file by following command
ChkEnt -infl 20092008.txt -opfl 20092008_test.txt >count.txt

Dear Friends,
Please help me in automating following thing.

If output generated (count.txt) is having value more than 1000 i.e. say it has shown 3521 entries
Then, system should split file as follows

Splt -infl 20092008.txt -opfl 20092008_1.txt -b 0 -e 1000
Splt -infl 20092008.txt -opfl 20092008_2.txt -b 1001 -e 2000
Splt -infl 20092008.txt -opfl 20092008_3.txt -b 2001 -e 3000
Splt -infl 20092008.txt -opfl 20092008_4.txt -b 3001 -e 3521

	Here the script should be smart enough to calculate number of files to be generated automatically by looking at total no of entries in original file i.e. 3521 in earlier example.

Thank you in advance.
I am waiting for your reply
Bye and take care

2

split takes a linecount parameter.....iirs the default is 1000, i.e. it will split your file into 1000 line chunks.

is this not what you are after ?

3

No, my system works differently, it does not split file by line count. It works on a logic n that logic is incorporated the command that i hv mentioned i.e. Splt

4

aaah - you mean something like:


#  x=$(wc -l count.txt) ; y=0 

#  while [ $x -gt 1000 ]; do echo Splt -infl 20092008.txt -opfl 20092008_${y} -b $(expr $y \* 1000 + 1) -e $(expr $y \* 1000 + 1000); y=$(expr $y + 1); x=$(expr $x - 1000); done ; echo Splt -infl 20092008.txt -opfl 20092008_2 -b $(expr $y \* 1000 + 1) -e  $(expr $y \* 1000 + $x)
Splt -infl 20092008.txt -opfl 20092008_0 -b 1 -e 1000
Splt -infl 20092008.txt -opfl 20092008_1 -b 1001 -e 2000
Splt -infl 20092008.txt -opfl 20092008_2 -b 2001 -e 3000
Splt -infl 20092008.txt -opfl 20092008_2 -b 3001 -e 3521
5

Dear Tytalus, thanx for your quick reply.
I m new to unix n all, so can u plz tell me following things

  1. Can we use "cat" option i.e. cat count.txt to assign that count value to variable x?
    e.g
    x=`cat count.txt`

  2. Honestly i coulnt undrstnd this statement.
    while [ $x -gt 1000 ]; do echo Splt -infl 20092008.txt -opfl 20092008_${y} -b $(expr $y \* 1000 + 1) -e $(expr $y \* 1000 + 1000); y=$(expr $y + 1); x=$(expr $x - 1000); done ; echo Splt -infl 20092008.txt -opfl 20092008_2 -b $(expr $y \* 1000 + 1) -e $(expr $y \* 1000 + $x)
    Can u jst tell me, is the script smart enough to handle even if a particular file has n number of records? or it can handle only 3521 entries?

6

no worries.

cat will simply spew the file out.

wc will give you a word count (and the -l will be the number of lines alone).

Yes - the script should work for any length file - it simply sets a variable x as the number of lines, then repeatedly subtracts 1000 and spts out each individual command, until x' is less than 100 - then the loop finishes and it kicks out the final splt command

7

Ohh wow, why this logic didnt strike me, was thinking about the logic since 2-3 days...
Any ways, will try this on Monday evening. Will let you know if I need more help.
Thanx a lot for your help.
Bye & God bless you,
All the best

8

One more thing, if you dont mind, can u further simplify this 'while' statement?
I mean can u make it such a way that even a lay-man like me can easily understand it? No matter if it will become lil bigger... It will help me in maintaining it in better way, n also i can make changes in it if necessary in near future.. plz

9

ok - simplified - and modified so you can specify any size for you split:


spltsize=1000
x=$(wc -l count.txt | awk '{print $1}')
y=1

while [ $x -gt $spltsize ]
do
  start=$(expr $spltsize \* \( $y - 1 \) + 1) 
  end=$(expr $spltsize \*  $y )

  echo Splt -infl 20092008.txt -opfl 20092008_$y -b $start -e $end
  y=$(expr $y + 1)
  x=$(expr $x - $spltsize)
done

start=$(expr $spltsize \* \( $y - 1 \) + 1)
end=$(expr $start + $x - 1)
echo Splt -infl 20092008.txt -opfl 20092008_$y -b $start -e $end

HTH