I have about 12,000,000 mod files I'm trying to turn into a test of "unlimited cloud storage" by running them all through VLC and blowing them into mp3 files. I can get this to work serially but when trying to use openMPI or Parallel , something in the syntax is tripping it up some. Here is an example of the command I'm using. Open to any help or suggestions. 
for file in `find ./modarchive/modarchive_2007_official_snapshot_addendum1/ -name "*.mod" -type f`; do parallel -j4 --citation NEWFILE=`echo $file | sed 's/modarchive/mp3archive/'`;echo working on $file; vlc -I dummy -f $file --sout "#transcode{acodec=mp3,ab=128}:standard{mux=mp3,dst=$NEWFILE.mp3,acce~ss=file}" vlc://quit 2> /dev/null ; done
What indication do you have that "something in the syntax is tripping it up"? What are the symptoms that something is going wrong?
Of course, one might guess that part of your problem is with the command inside your for loop:
vlc -I dummy -f $file --sout "#transcode{acodec=mp3,ab=128}:standard{mux=mp3,dst=$NEWFILE.mp3,acce~ss=file}" vlc://quit 2> /dev/null
- You define, but do not use, the variable
NEWFILE in the command substitution that runs in a subshell environment and disappears before your for loop starts running. So, in that command $NEWFILE will expand to a constant (probably an empty string) value instead of a value based on the name of the file you're processing.
- Does the
vlc utility really want a sub-option of the form acce~ss=file , or is the tilde ( ~ ) in that string a typo?
- Instead of throwing away the diagnostic messages produced by the code you're running (
2>/dev/null ), it might help to actually read those diagnostics and see if they provide any insight into what might be going wrong.
I would also imagine that invoking sed 12 million times makes your script take a long time before it ever gets to the point that it invokes vlc the first time. Why not use a couple of variable expansions inside the loop to avoid invoking sed at all? Note that this operation needs to be in a position where NEWFILE will be paired with the file value it has modified; not in someplace where the vlc command you invoke will be using unpaired $file and $NEWFILE values.