File splitting and grouping using unix script

Shell Programming and Scripting
1

Hello All,

I have a small problem with file group/splitting and I am trying to get the best way to perform this in unix. I am trying with awk but need some suggestion what would be the best and fastest way to-do it.

Here is the problem. I have a fixed length file with filled with product data, I need to split each product in different file grouped.

Example main content file: data.txt

> cat data.txt
123456 Hello World 1
234455 Hello World 2
123456 Hello World 3
234455 Hello World 4
999922 Hello World 5
123456 Hello World 6
999922 Hello World 7

Layout
first 6 characters production identifier, rest of the data belong to the same product.

When I split, I need files name with product name with all product data grouped in one file.

Result after split. I will have 3 files namely 123456, 234455 and 999922.

123456.txt
123456 Hello World 1
123456 Hello World 3
123456 Hello World 6

234455.txt
234455 Hello World 2
234455 Hello World 4

999922.txt
999922 Hello World 5
999922 Hello World 7

If you can provide any input that will be awesome. :cool:

Thanks a lot.
_nandhan_

2

A sh solution:

while IFS= read -r l; do
    echo "$l" >> ${l%% *}.txt
done < data.txt

Regards and welcome to the forum,
Alister

3

Using awk...

awk '{print $0 > $1 ".txt"}' data.txt
4

If you hit a file descriptor limit, you can try:

awk '{f=$1".txt"; print >> f; close(f)}' data.txt
5

Thanks for quick reply... It works. I have a huge file (500mg) going to try now :slight_smile: