Errors trying to use all files of a type

I am trying to create a code that will use all the bam files stored on a separate drive (/media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215), run them in a program that I have changed the directory to, and the output gets re-directed to (/home/cmccabe/Desktop/NGS/pool_I_090215). I have tried the script two ways getting two different errors and cant seem to fix it. the bold in the second error is the output so why is it looking for those files Thank you :).

cmccabe@HPZ640:~$ cd "/home/cmccabe/Desktop/NGS"
cmccabe@HPZ640:~/Desktop/NGS$ for f in /media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/*.bam do prefix=${f%%.bam} samtools view -H $f | sed '/^@PG/d' | samtools reheader - $f > /home/cmccabe/Desktop/NGS/pool_I_090215/${prefix}_newheader.bam done

bash: syntax error near unexpected token `|'


cmccabe@HPZ640:~/Desktop/NGS$ for f in /media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/*.bam
> do
> prefix=${f%%.bam}
> samtools view -H $f | sed '/^@PG/d' | samtools reheader - $f > /home/cmccabe/Desktop/NGS/pool_I_090215/${prefix}_newheader.bam
> done

bash: /home/cmccabe/Desktop/NGS/pool_I_090215//media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_008_150902_newheader.bam: No such file or directory
bash: /home/cmccabe/Desktop/NGS/pool_I_090215//media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_015_rawlib_newheader.bam: No such file or directory
bash: /home/cmccabe/Desktop/NGS/pool_I_090215//media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_016_150902_newheader.bam: No such file or directory 

You are just removing the '.bam', you are not removing the long pathname.

Try FILE=$(basename "$FILE") to remove the path.

The below code runs but no output is created. Thank you :).

cmccabe@HPZ640:~/Desktop/NGS$ FILE=$(basename "$FILE")
cmccabe@HPZ640:~/Desktop/NGS$ for f
> do
> prefix=${f%%.bam}
> samtools view -H $f | sed '/^@PG/d' | samtools reheader - $f > /home/cmccabe/Desktop/NGS/pool_I_090215/${prefix}_newheader.bam
> done 

tried this as well:

 cmccabe@HPZ640:~/Desktop/NGS$ for f in /media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/*.bam
> FILE=$(basename "$FILE")
bash: syntax error near unexpected token `FILE=$(basename "$FILE")'
cmccabe@HPZ640:~/Desktop/NGS$ do
bash: syntax error near unexpected token `do'
cmccabe@HPZ640:~/Desktop/NGS$ prefix=${f%%.bam}
cmccabe@HPZ640:~/Desktop/NGS$ samtools view -H $f | sed '/^@PG/d' | samtools reheader - $f > /home/cmccabe/Desktop/NGS/pool_I_090215/${prefix}_newheader.bam
bash: /home/cmccabe/Desktop/NGS/pool_I_090215//media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_016_150902_newheader.bam: No such file or directory

Tried what? What exactly did you do? The code is scrambled by all the errors.

Why are you using FILE when your variable is named f?

I guess im not sure how to use basename . Thank you :).

The same technique as prefix=${f%%.bam} can be used to obtain the name of the file without the path

base_name=${f##*/}

And chained

basename=${f##*/}
prefix=${basename%%.bam}

Thank you very much.

Are the two variables $pref and $prefix different? Thank you

There was nothing in this thread that set or referenced the shell variable pref until you asked this question. Without knowing how you set pref , there is no way that we can guess whether or not the shell variable expansion $pref will expand to the same string as the shell variable expansion $prefix .

Here is my bash loop that works perfectly.

  for f in /media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/*.bam ; do
      bname=`basename $f`
      pref=${bname%%.bam}
      samtools view -H $f | sed '/^@PG/d' | samtools reheader - $f > /home/cmccabe/Desktop/NGS/pool_I_090215/${pref}_newheader.bam
 done
  

I also tried the below with no luck:

  for f in /media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/*.bam ; do
      bname=`basename $f`
      prefix=${f%%.bam}
      samtools view -H $f | sed '/^@PG/d' | samtools reheader - $f > /home/cmccabe/Desktop/NGS/pool_I_090215/${prefix}_newheader.bam
 done
  bash: /home/cmccabe/Desktop/NGS/pool_I_090215/media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_008_150902_newheader.bam: No such file or directory
bash: /home/cmccabe/Desktop/NGS/pool_I_090215/media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_015_rawlib_newheader.bam: No such file or directory
bash: /home/cmccabe/Desktop/NGS/pool_I_090215/media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_016_150902_newheader.bam: No such file or directory 

I am just curious why the second command was looking for the output in the directory. Thank you :).

bname=${f##*/} will always be faster than bname=`basename $f`

You are not working with the same variable. In pref you use bname as the base; in prefix , you are using the f variable and not bname .

Perhaps this demo will clarify:

[cmccabe]$ f="/media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_008_150902.bam"
[cmccabe]$ echo $f
/media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_008_150902.bam
[cmccabe]$ bname=`basename $f`
[cmccabe]$ echo $bname
IonXpress_008_150902.bam
[fcmccabe]$ pref=${bname%%.bam}
[cmccabe]$ echo $pref
IonXpress_008_150902
[cmccabe]$ 
[cmccabe]$ prefix=${f%%.bam}
[cmccabe]$ echo $prefix
/media/cmccabe/C2F8EFBFF8EFAFB9/pool_I_090215/IonXpress_008_150902
[cmccabe]$ 

I highlighted the difference

Thank you for clarifying and for the great demo