Append (cat) to beginning of file ?

Hi,

Fairly new to unix scripting, hoping to get some help.

using AIX v5

Basically I have 3 files 1). Header record 2). many detail record 3). Trailer record

My desired result is 1 file which contains Heaeder, Detail, Trailer

Currenty I am using a series of:
cat detail >> header
rm detail
cat trailer >> header
rm trailer
mv header detail

Now the problem with this approach is that my detail file quite often is in excess of 5gb, so the time is takes to do the cat seems like a waste of time to me. I would really like to be able to concatenate the header to the beginning of the detail and only "move" the one record.

Any ideas would be appreciated.

Thanks,

I assume some program is creating the "detail" file .. how much control over that program do you have? If you could go into it and tell it to stick your header information at the top of the file before it runs through the loop or whatever piece of code creates the 5gb detail file, that might be best...

We do have control over the program creating the files, but the header and trailer are created based upon the sum of the detail records so not possible to hold off writing all the details.

Chad.

I assume that before doing the following you already have the header, trailer and detail files

in that case

cat header detail trailer > newfile

I hope I havent much misunderstood the question

Won't that still copy the 5gb file ? I see how it is much more econimical as far as commands go though...

Thanks,

To edit any file in any way you will have to read it directly or indirectly
so any command/script/method will take atleast the time required to open the file and save it bak
so what you need to really worry is that you do minimum possible operations on the huge file

but you cannot avoid the time for

1. open
2. read

on the file to have it changed

penguin,

Thanks, that confirms my fear. However, I will try doing it in the one command and see how it works.

Thanks for your time and help.

There may be a way, but it's a little screwy.

You could have a dummy header that reserves enough space for the real header. For example, it might say:

line count = xxxxxxx

Then you append your detail records and compute the real header. Finally you rewrite the first n bytes where n is the number of bytes in the header. To rewrite the first bytecount bytes use:

dd if=realheader bs=1 count=$bytecount conv=notrunc of=outputfile

Hi, as Perderabo suggested, you may leave space at the beginning and end with strings like space_for_header and space_for_footer. You may at last use stream editor sed to replace those strings with your header and footer info. I am not sure about the time taken but you can sure use your time wisely if you use the sed command in a script and let it run in the background. Hope it helps. Cheers