How to split a fixed width text file into several ones based on a column value?

Shell Programming and Scripting
1

Hi,

I have a fixed width text file without any header row. One of the columns contains a date in YYYYMMDD format.

If the original file contains 3 dates, I want my shell script to split the file into 3 small files with data for each date.

I am a newbie and need help doing this.

2

please post sample input and expected output.

Regards
Ravi

1 Like
3

Ok... here is a sample.

The first field is Date in YYYYMMDD.

As you can see, there are 3 dates viz. 20100313, 20100306 and 20100227.

The length of the Date field is 8 bytes.

I want my shell script to read this file and create 3 files, each with data for each of the 3 dates.

Thanks in advance. :slight_smile:

20100313  25     2899780446536301 0   1    9.99
20100313  22     9819780446541930 0   1    6.99
20100313  15     1539780446194068 0   1    7.99
20100313  25      459780316044936 0   1   11.24
20100313  20     1279780446570978 0   1   11.24
20100306  26     9959780446570961 0   1    5.99
20100306  24      659780316036221 0   1   16.79
20100227  16      189780446539746 0   2   14.98
20100227  25      279780446570978 0   2   20.98
20100227  19      149780446561334 0   2   22.48
20100227   8      499780446194068 0   2   11.98
20100227  22     8379780446539746 0   3   22.47
4 
awk '{ print > $1".txt" }'  input_file.txt
1 Like
5

Shorter ;):

awk '{print > $1 ".txt"}' file

@panyam was faster :smiley:

1 Like
6 
cut -c1-8 <s1.txt >s2.txt

pulling the first 8 columns of file s1.txt into s2.txt

7

I don't think this will serve the purpose !!!

Since OP want the output to be stored on different files based on column1 which is date.

8

My bad. I misunderstood what was desired for output. I read his request as simply to pull off the first 8 characters.

9

Cool! :b:

awk worked. :cool:

Couple of questions more.

What do I do if I do not want to hardcode the input_file_name in my shell script and pass it as a parameter from the prompt instead (while running the shell script)?

And how do I name the split output files as POS_Volume_<file_date_here>.txt?

10 
 
awk '{ print > "POS_Volume_"$1".txt" }'  $input_file_name
1 Like
11

Unfortunately, this one is not working.

Just keeps running without returning the prompt.

The previous awk worked like a charm.

How do I fix this one?

12

Change:

$input_file_name

to the name of your input file (without the $)

1 Like
13

But what if I don't want to hardcode the input filename? I want to pass the input filename dynamically while running the script. :slight_smile:

14

If input_file_name is set correctly before running ,awk, script should run.
Please post the script you are trying to run, showing how filename variable is set along with the awk command and how script is ran.

15

Hi.

Then, taking the awk from anurag.singh:

[ $# -eq 0 ] && echo "Usage: $0 <filename>" && exit 1
input_file_name=$1
awk '{ print > "POS_Volume_"$1".txt" }'  "$input_file_name"