Xargs to call python executable to process multiple bam files

Shell Programming and Scripting
1

I am running the below loop that to process the 3 bam files (which isn't always the case). A .py executable is then called using | xargs sh to further process. If I just run it with echo the output is fine and expected, however when
| xargs sh is added I get the error. I tried adding | xargs python instead, but get a different error where the standard options are not reconized. It seems like the executable is not be called
, but I do not know enough python to troubleshoot. Thank you :).

xxx_00.bam
yyy_01.bam
zzz_02.bam

echo "${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py $(for f in *.bam; do echo -n "--bam ${f} "; done;) --referenceFasta ${fasta} --callRegions ${bed} --exome --runDir ${dir} ${dir}/runWorkflow.py -m local -j 20" | xargs sh

echo only
set -x

echo "${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py $(for f in *.bam; do echo -n "--bam ${f} "; done;) --referenceFasta ${fasta} --callRegions ${bed} --exome --runDir ${dir} ${dir}/runWorkflow.py -m local -j 20" | xargs sh
+ xargs sh
++ for f in '*.bam'
++ echo -n '--bam xxx_00.bam '
++ for f in '*.bam'
++ echo -n '--bam yyy_01.bam '
++ for f in '*.bam'
++ echo -n '--bam zzz_02.bam '
+ echo 'path/to/configureStrelkaGermlineWorkflow.py --bam xxx_00.bam --bam yyy_01.bam --bam zzz_02.bam  --referenceFasta hg19.fasta --callRegions file.bed --exome --runDir path/to/data path/to/data/runWorkflow.py -m local -j 20'

error

/path/to/bin/configureStrelkaGermlineWorkflow.py: line 23: $'\nThis script configures the strelka germline small variant calling workflow\n': command not found
import: unable to open X server `' @ error/import.c/ImportImageCommand/369.
/path/to/bin/configureStrelkaGermlineWorkflow.py: line 27: syntax error near unexpected token `('
/path/to/bin/configureStrelkaGermlineWorkflow.py: line 27: `if sys.version_info >= (3,0):'
2

(3,0) A parenthesis is a reserved symbol in bash, not a string.
"(3,0)" Try making it a string...

1 Like
3

xargs is used to turn a stream on stdin into command line arguments / parameters. sh expects its commands on stdin. Try without xargs , like

echo "..." | sh

Or, use your .py as the xargs ' command but make sure it will be interpreted by python (by defining the correct "shebang").

1 Like
4

I changed the shebang line in the python executable to #!/usr/bin/env python to match the other .py on my system and get the below error.

    python --version on centos 7
    Python 2.7.5

    echo "${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py $(for f in *dups.bam; do echo -n "--bam ${f} "; done;) --referenceFasta ${fasta} --callRegions ${bed} --exome --runDir ${dir} ${dir}/runWorkflow.py -m local -j 20" | python
    
    
    File "<stdin>", line 1
    ${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py --bam xxx_00.bam --bam yyy_01.bam --bam zzz_02.bam  --referenceFasta ${fasta} --callRegions ${bed} --exome --runDir ${dir} ${dir}/runWorkflow.py -m local -j 20
                                                                                                                                                                                                ^
                                                                                                                                                                          SyntaxError: invalid syntax
         .... | xargs python  
                                                                                                                                                                          
            error: no such option: -m  this is a standard option
            
         .... | sh
                                                                                                                                                                          
            error: no such option: -m    this is a standard option
            
         .... | xargs sh
                                                                                                                                                                          
           ${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py: line 23: $'\nThis script configures the strelka germline small variant calling workflow\n': command not found
           import: unable to open X server `' @ error/import.c/ImportImageCommand/369.
           ${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py: line 27: syntax error near unexpected token `('
           ${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py: line 27: `if sys.version_info >= (3,0):'

I also tried

    echo "${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py $"(for f in *dups.bam; do echo -n "--bam ${f} "; done;)" --referenceFasta ${fasta} --callRegions ${bed} --exome --runDir ${dir} ${dir}/runWorkflow.py -m local -j 20" | xargs sh

    -bash: syntax error near unexpected token `('

Thank you :).

5 
ls *dups.bam \
| xargs -I % echo --bam % \
| xargs python "${path_to_strelka}"/bin/configureStrelkaGermlineWorkflow.py \
--referenceFasta "${fasta}" --callRegions "${bed}" --exome \
--runDir "${dir}" "${dir}"/runWorkflow.py -m local -j 20