Formatting with Sed creating empty file

I’m having an input file with one million records and formatting and writing to another file to a different location but some times it is writing empty file(intermittent). Following is the scenario: Input.out

Adding Header:

echo “HSample” >> output.out Details : cat Input.out | sed -e ‘s/^/D/‘ >> output.out 

Trailer :

echo “TSample” >> output.out

In the output.out it is writing only Trailer record without header and details.

This is happening intermittently, if I rerun it is ok.

Please advise.

Thanks

It sounds like you’re experiencing a timing or file-access issue that might be causing output.out to be written prematurely or not fully at times.

Hints:

• Try to ensure Input.out is fully loaded.
• Use temporary file for consistency.
• Add some code for error handling.
• Consider reducing command substitutions.

Reviewing and tweaking these steps should increase consistency in the output and prevent intermittent empty files.

Hi Neo, Thanks for the information, however I doubt it is timing issue, as trailer record consists record count from input records and every time we had this issue, trailer record is the only record written to the output with correct record count. Thanks

That's a colon. I think you want a semicolon

;

See remark about : instead of ; (I assume it's just wrongly formatted post, and you don't need any of them, but if you really do need it - always use ; at the end of each complete command).
And you don't actually need cat for anything

echo "HSample" >> output.out;
sed -e 's/^/D/' Input.out >> output.out;
echo "TSample" >> output.out;

Make sure the last command (for writing trailer data) always uses >> to append and NOT a single > to overwrite. If not sure, in Bash you may use e.g. set -o noclobber that prevents > from overwriting an existing file, while still allowing >> to append to it.

Behind the scenes each >> does open(),seek(),close()
You can make a command group and redirect all its output (and even its input if desired):

{
echo "HSample"
sed -e 's/^/D/' Input.out
echo "TSample"
} >> output.out

zsh (not bash or other shells) also allows

>> output {
echo "HSample"
sed -e 's/^/D/' Input.out
echo "TSample"
}

that better reflects the execution order.