insert a header in a huge data file without using an intermediate file

I have a file with data extracted, and need to insert a header with a constant string, say: H|PayerDataExtract

if i use sed, i have to redirect the output to a seperate file like

sed ' sed commands' ExtractDataFile.dat > ExtractDataFileWithHeader.dat

the same is true for awk

and in its simplist form i could say

echo 'H|PayerDataExtract' > ExtractDataFileWithHeader.dat
 cat ExtractDataFile.dat >> ExtractDataFileWithHeader.dat
 mv ExtractDataFileWithHeader.dat ExtractDataFile.dat

but in all of the above an extra file is created. If i were to do this in vi manually the extra file could be avoided.

Is there a way to avoid the extra file while still not having to manually use vi in a interactive manner? This is even more neccessary if the file uses over 50% filespace and an extra file will just double my usage , though temporarily

This is crazy, I agree.

Typically, in situations like this, I employ an "original_file_name.info" file instead.

In the .info file I'll put all the information necessary for another program/user
to verify the contents of the huge file i created.

There are other options . . . . like creating a dummy header record in
the file creation program . . . and then using fseek() to hop back to
the beginning and overwrite the header info....

but you may not have this much control over how this file is created.

There are 'edit in place' features for GNU sed and for perl (perl -pie ' perl script here').
However they do use tmp files behind the scenes

You can even use the sed 'insert' command along with -i (inline) option to place the the "header" at a specific line number (like line 1).

Something like this should work:

sed -i '1 i \Some Header Text Here' ExtractDataFile.dat

Why not create an ed script instead? And then use 'patch'? But that would still require a temp file in the background though you wouldn't notice.

hi ddreggors/jim

i tried this - and it fails with the below error

sed -i '1 i \HeaderText' ExtractDataFile.txt
sed: illegal option -- i

The system is a SunOS ussun1l 5.8 Generic_117350-60 sun4u sparc SUNW,Sun-Fire-15000

the perl option is out for me as the client may not want to install any piece of software without business cause, however as long as the tmp file is created in the /tmp space and not my file-dir i think i'm ok - any more ideas anyone?

try this:

sed 1'i\HeaderText' ExtractDataFile.txt

If this outputs to the screen as expected (this does not update the file) then the inner 'i' (insert command) is working fine.

Next try:

sed -i 1'i\HeaderText' ExtractDataFile.txt

If either gives an error then you may have to upgrade sed to get the use of the 'inline' flag (-i) or the insert command ('i\text') to be able to do this without writing to another file first and specifying a line number.

Try...

awk '{a[NR]=$0}END{a[0]=h;for(i=0;i<=NR;i++)print a>FILENAME}' h="HeaderText" ExtractDataFile.txt

Use nawk or /usr/xpg4/bin/awk on Solaris.

Unfortunately, any solution is going to use a temp file, either explicitly or behind the scenes.

You might as well go with:

(
echo WHATEVER HEADER INFO YOU NEED
cat original_file
) > temp_file

mv temp_file original_file

What I typically like to do though in situations like these is create an associated
file_name.info file which contains all the header and other useful information like
create-dt, created-by-user, number-of-records, number-of-bytes and check-sums.

This also works for using ftp when a polling program is trying to figure out when a
large file is finished being created so it can snatch it up right away. Instead of
polling for the size to stop growing, it just waits til the info file is created.

The solution which Ygor provided works!!

I would like to understand how it works though

thanx anyways

In fact one of the other methods i would think works is something like this
1) Create a file that has vi commands

CommandsOfVi.script

vi cmd to goto line 1
vi cmd to insert 'Header Text'
vi cmd to save and quit a file

2) Open a file using vi and non-interactive mode by using the above commands file as input

vi ExtractDataFile.txt < CommandsOfVi.script

I had done something a few years back, but don't remember the syntax etc. so just put it in a very simplified manner here - this approach, I believe, wont use a temp file