Splitting the Huge file into several files...

Shell Programming and Scripting
1

Hi

I have to write a script to split the huge file into several pieces. The file columns is | pipe delimited. The data sample is as:

6625060|1420215|07308806|N|20100120|5572477081|+0002.79|+0000.00|0004|0001|......
6625060|1445972|10476049|N|20100120|5572477081|+0003.29|+0000.00|0004|0002|......
6625060|1001309|10491766|N|20100120|5572459582|+0002.59|+0000.00|0006|0001|......
6625060|1002119|10303228|N|20100120|5572477081|+0002.49|+0000.00|0004|0001|......

My requirement is to split this file based on the date in 5th column. The condition is based on this date and to be splitted by month. So, the number of files would be 12 (a file for each month).

Please help me to write a Unix script to split several files.

Thanks
LakTeja

2

you can use awk:

awk -F'|' { file=sprintf("%s_%s", FILENAME, substr($5,5,2)); print $0 >> file}' inputfile

will produce files names inputfile_01 inputfile_02 ... inputfile_12 if there is data for all months in inputfile.

3 
awk -F"|" ' { print > substr($5,5,2) } ' file
4

Hi, lakteja:

Welcome to the forums. The following AWK one-liner will do what you ask. One file per month based on the date in the fifth pipe-delimited column. The filename will be in the form YYYYMM:

awk -F\| '{print > substr($5,1,6)}' file

Should you prefer the files to be named with just the month, MM:

awk -F\| '{print > substr($5,5,2)}' file

Regards,
Alister

---------- Post updated at 11:13 AM ---------- Previous update was at 10:55 AM ----------

A shell version which generates YYYYMM filenames:

#!/bin/sh

file=$1
IFS=\|
while read -r line; do
    set -- $line
    echo "$line" >> ${5%??}
done < "$file"