Split A Large File

Shell Programming and Scripting
1

Hi,

I have a large file(csv format) that I need to split into 2 files. The file looks something like

Original_file.txt
first name, family name, address
a, b, c,
d, e, f,
and so on for over 100,00 lines

I need to create two files from this one file. The condition is i need to ensure both files have the header row "first name, family name, address" AND the second file only starts from the 60,000th row of original_file.txt

Can some please show me the code to do this in a shell script?

Really Appreciated

Thanks

2

Awk:

1==NR || 60000==NR {
  whichfile++
  output( "first name, family name, address" )
}
{ output( $0 ) }

function output( s )
{ print s  >( "output" whichfile ".csv" )
}
3

Hi,

you can make the first file by using:

head -60000 filename > filename_1to60000

you can make the second one as follow:

echo 'first name, family name, address' > filename_60001to100000
tail +60001 filename >> filename_60001to100000

Kind regards

4

I will try it out...thanks ill let you know how it goes

5

split -l 60000 in_file

In case of more complex fragmentation of a file, use the context based split command csplit.

Jerardjay