sort a file which has 3.7 million records

Shell Programming and Scripting
1

hi,
I'm trying to sort a file which has 3.7 million records an gettign the following error...any help is appreciated...

sort: Write error while merging.

Thanks

2

Hello,

Per our forum rules, all threads must have a descriptive subject text. For example, do not post questions with subjects like "Help Me!", "Urgent!!" or "Doubt". Post subjects like "Execution Problems with Cron" or "Help with Backup Shell Script".

The reason for this is that nearly 95% of all visitors to this site come here because they are referred by a search engine. In order for future searches on your post (with answers) to work well, the subject field must be something useful and related to the problem!

In addition, current forum users who are kind enough to answer questions should be able to understand the essence of your query at first glance.

So, as a benefit and courtesy to current and future knowledge seekers, please be careful with your subject text. You might receive a forum infraction if you don't pay attention to this.

Thank you.

The UNIX and Linux Forums

3

Howdy!

The most probable reason might be that your disk got full or that your disk quota was exceeded. If you have another mounted partition/disk with more space, try

sort -T /path/to/where/you/have/more/space

Hope it helped,

Matt

4

Please state which Operating System, the command you typed, the exact size of the file, and the amount of free disc space in each disc partition involved (source file, destination file, sort work files). In most versions of sort the default location of sort work files is /tmp .

5

How about breaking the file into multiple small files do the sort and merge them later.
As pointed out the disk is full, check with "du" in UNIX/Linux.
Try Perl/Python to do a intelligent sort, bubble or something (Please note: At this time I am not thinking of the big 'O' or what is efficient. Just giving few ideas.

6

Ex. split using awk. In this example has used two first letters/line to split file.

#!/bin/ksh
rm -f *.split 2>/dev/null
awk  -v keylen=2 '{
        key=substr($0,1,keylen)
        print $0 >> key".split"
        }' bigfile

> newbig
for f in ??.split
do
   sort "$f" >> newbig
done
7

You may want to check the disk space where your /tmp is ?

Thanks
Ashok