I have 10,000 + files, each of which I need to zip using bzip2.
Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished?
There's a bunch of proposals on this topic in these forums. Did you consider the links at the bottom margin of this page?