Hi Experts,
I've been asked if there is a fast way to duplicate a file(10GB) and zip it at the same time. The zipped file would be FTP'd.....management is asking this. Maybe there is a better method all together? any ideas? CP will not cut it.
Thanks in advance
Harley
You can avoid reading the file twice by piping it into tee:
tee copy < original | gzip > compressed.gz
This will write the original uncompressed data into copy, while at the same time writing it to stdout where a compressor like gzip can handle it.
If at all possible, avoid storing copy and compressed.gz on the same disk you're reading from, it will increase performance to not have to wait for writes to finish for reads to happen. In an ideal world, you could store copy on a different disk from compressed.gz too, to avoid splitting the disk's speed between different writes.
If you really mean .zip and not just some sort of compressor,
tee copy < original | zip compressed -
...will create a 'compressed.zip' containing a file named '-'.