Search for specific file type in subdirectory with multiple folders

Shell Programming and Scripting
1

I have a directory that is in the below order (the --- is not part of the directory tree, only there to help illustrate:

DATE      --- main level
Folder1   --- level under DATE
plugin_out  --- level under Folder1
variantCaller_out.40   --- level under plugin_out
001,002,003  --- level under variantCaller_out.40

The bash below reads each line in out.txt and uses that in a below. So for this example a would be /home/cmccabe/nfs/exportedReports/Folder1 . What I am having trouble with is the vcf.gz being searched for is two more levels down (last level in the above). /home/cmccabe/nfs/exportedReports/plugin_out/variantCaller_out.40/001 . The level right before that /home/cmccabe/nfs/exportedReports/plugin_out/variantCaller_out.40 is consistent unitil the . , then the numbers varies. So in this example variantCaller_out.40 the portion in bold will change. The last level has 3 folders 001, 002, 003 that the vcf.gz can be found in.

I guess what I am trying to do is search for each vcf.gz in

/home/cmccabe/nfs/exportedReports/Folder1/plugin_out/variantCaller*/{001,002,003}

Bash

a=/home/cmccabe/nfs/exportedReports;DEST="/home/cmccabe/Desktop/NGS/API/$DATE";
while read line; do find $a/$line -name "*vcf.gz" -exec cp {} $DEST \;
done < out.txt

contents of out.txt

Folder1

within Folder1

plugin_out

next level Folder1 / plugin_out

variantCaller_out.40

next level Folder1/plugin_out/variantCaller_out.40

001
002
003

Currently I am getting syntax errors. Thank you :).

2

If you are getting syntax errors, have you tried using set -x . This will let you see the exact commands being executed with variables being replaced with actual values.

1 Like
3

set -x showed that, while I am still not sure of the best way to accomplish this, the final vcf.gz that comes from the 001 folder is called variants.vcf.gz . The final vcf.gz that comes from the 002 folder is also called variants.vcf.gz . So unless the files are named differently somehow they are always going to be replaced. Thank you :).

4

Hello cmccabe,

I am sorry, I am confused about your requirement. But by seeing your find command I could say you need to copy few files by finding them from one location to another and copy to detination directory, so you could try to change your command with following(not tested though).

find $a/$line -name "*vcf.gz" -exec cp -t $DEST {} \+

It will copy all matched files to destination directory. Please try it and do let us know how it goes then. If your requirement is different then please be more clear and specific in your requirements.

Thanks,
R. Singh