I have a directory /home/cmccabe/nfs/exportedReports that contains multiple folders in it. The find writes the name of each folder to out.txt . A new directory is then created in a new location /home/cmccabe/Desktop/NGS/API , named with the date . What I am trying to do, unsuccessfully at the moment, is copy each specific file type .bam from each folder in out.txt to the new date folder in /home/cmccabe/Desktop/NGS/API/$DATE .
For example, after the find command out.txt :
Folder1
Folder2
within each of those folders there are 3 .bam files to move to the new directory /home/cmccabe/Desktop/NGS/API/$DATE . Thank you :).
#!/bin/bash
cd /home/cmccabe/nfs/exportedReports
find . -maxdepth 1 -type d | cut -c 3- | awk -F _ 'NF > 1' > out.txt
DATE=`date +%-m-%-d-%Y`
mkdir -p /home/cmccabe/Desktop/s5/$DATE
MY_DIR="/home/cmccabe/nfs/exportedReports/out.txt"
DEST="/home/cmccabe/Desktop/NGS/API/$DATE"
FILEEXT=".bam"
NEWEST=`ls -tr1d "${MY_DIR}/"*.${FILEEXT} 2>/dev/null | tail -1`
if [ -z "${NEWEST}" ] ; then
echo "No file to copy"
exit 1
elif [ -d "${NEWEST}" ] ; then
echo "The most recent entry is a directory"
exit 1
else
echo "Copying ${NEWEST}"
cp -p "${NEWEST}" "${DEST}"
fi
Above code is not doing what you explained in your post.
You need to read line by line in out.txt file then find / list the files with *.bam extension , copy etc.
It may be done in oneliner , if you can add more description about structure of Folder1,Folder2 and /home/cmccabe/nfs/exportedReports with clear example.
if not, try like this:
cat out.txt
DIR1
DIR2
$ ls -a DIR1/
. .. 123.bam abc.bam xyz.log
$ ls -a DIR2/
. .. cool.txt me.bam
Sample code written here to update your script.
a=/home/cmccabe/nfs/exportedReports;DEST="/home/cmccabe/Desktop/NGS/API/$DATE";
while read line; do find $a/$line -name "*.bam" -exec cp {} $DEST \;
done < out.txt
$ ls -a $DEST
. .. 123.bam abc.bam me.bam
The below is all most working, there are sub-directories in each folder and the find and cp doesn't need to search these, as the .bam files are all in the main folder. Thank you :).
I am getting an error:
cp: cannot stat �/home/cmccabe/nfs/exportedReports/Auto_user_S5-00580-5-Medexome_66_030/plugin_out/coverageAnalysis_out.39/IonXpress_007/scraper/link.summary.bam': No such file or directory
--- this is a subdirectory that can be excluded from the find , but I am not sure how.
If I understood your requirement correctly(though I am not sure) it is to copy a specific extension files from source folder to target folder, so could you please try following and let me know if this helps you(not tested though).
So in above code you could put .*bam files in place of test* and could put location in place of /tmp above(destination dire I have put echo (in bold and underline) for testing purposes, so if you are happy with above command's result you could remove that echo . You could set the value of maxdepth as per your requirements too.
Please do let us know how it goes then.
Is the goal to copy the *.bam files in *_* folders, i.e. *_*/*.bam files?
Then try this simple one
DATE=`date +%-m-%-d-%Y`
FROM="/home/cmccabe/nfs/exportedReports"
DEST="/home/cmccabe/Desktop/s5/$DATE"
if
cd "$FROM" &&
mkdir -p "$DEST"
then
cp *_*/*.bam "$DEST"
fi
In your post#3 I saw you asked how to exclude any directory or sub-directory in find command then following could help you in same. To exclude a directory and only print all the file names following may help you in same.
find . -maxdepth 2 -type d -name "again_test*" -prune -o -type f -name "test*" -print
If you are happy with above command then you could copy files to the destination directory by following command.
find . -maxdepth 2 -type d -name "again_test*" -prune -o -type f -name "test*" -exec cp -t /tmp {} \+
So in above codes change values of again_test* to that directory which you want to exclude, test* with those file extensions/names which you want to find here, /tmp with your destination path please. Change maxdepth value as per your need here. Kindly do test the same and please do let us know how it goes for this.