I have been trying to execute the below command by changing directory and then copying contents of one directory to another by doing some file name manipulations in between. However this isnt working since as soon as the statement completes it goes back to the original folder. Can someone help in telling me how to change a directory in a script so I can execute subsequent steps within it using and modifying a variable through the steps.
This works fine from command line, however gives bad substitution error from the script because of the issue explained above
for file in /ABC/DEF/*.done; do file="${file%.*}"; file="${file##*.}"; cp "${file:0:31}"*.* /ABC/EDG/; done
Basically check for all files with name like CAT_TURNAROUND_201811120_075221_3.done in the folder ABC/DEF/*.done , then get all the files starting with just this part CAT_TURNAROUND_201811120_075221 from this directory /ABC/DEF/ copied to /ABC/EDG/
Will really appreciate a solution. I have been trying to figure this out and checked various forums and tried alias, function etc. But it isn't working.
Not sure I understand the problem. Would you please carefully proofread your post and correct typos and logical errors (using the "edit" button at the lower right)? Like the ABC?DEF/*.done - is that a (wildcarded) directory or a sequence of files?
You don't seem to cd to anywhere, so why should it (what is it?) go "back to the original folder"? What exactly "isnt working"? Show the error messages.
And, give some data, like real filenames, as your sample "FilePrefix_20181120_075221_3.done" doesn't go well with the "${file:0:31}" shell expansion. Is your target directory meant to be an absolute path? Did you realize that your second expansion "${file##*.}" doesn't do anything as there's no . left in the file name?
On top of what my esteemed colleagues already said: you should not need to do any cd in any script! Use absolute pathes whenever possible (that effectively means: always), this will always guarantee that the script works the same from whereever you call it.
Instead of i.e.
cd /some/where
for file in * ; do
cp $file ../../some/where/else
done
Do it this way:
fSource="/some/where"
fTarget="/some/where/else"
for fFile in "$fSource"/* ; do
fFile="${fFile##*/}" # /some/where/filename -> filename
cp "${fSource}/${fFile}" "${fTarget}/${fFile}"
done
Thanks for the responses Madeingermany and Bakunin. I am using #!/bin/bash on the first line of the script. I am new to scripting, I will lookup and try the source script method for this. Hoping that works, will let you know if it does.
Bakunin, I had tried using absolute paths, as my statement has multiple steps in between, the value of 'file' is lost in each step.
for file in "$fSource"/*.done; do file="${file%.*}"; file="${file##*.}"; cp "${file:0:31}"*.* "$fPrepped"/; done
I do not think that the value "file" is "lost", i just think that "file" at each step doesn't contain what you think it does. This is your code:
for file in "$fSource"/*.done; do file="${file%.*}"; file="${file##*.}"; cp "${file:0:31}"*.* "$fPrepped"/; done
The first thing is: do NOT write one-liners. It is hard to understand what is going on when. So first, let us reformat:
for file in "$fSource"/*.done ; do
file="${file%.*}"
file="${file##*.}"
cp "${file:0:31}"*.* "$fPrepped"/
done
Next thing: make sure that each step really does what you think it does. Notice that i took the final manipulation out of the cp command for clarity:
for file in "$fSource"/*.done ; do
echo " after for: $file"
file="${file%.*}" ; echo "first manipulation: $file"
file="${file##*.}" ; echo "second manipulation: $file"
file="${file:0:31}" ; echo "third manipulation: $file"
echo "cp ${file}*.* $fPrepped/"
done
Now run that and observe: does "$file" in every step really contain what you expect it to contain? I suppose it doesn't. It is not "lost" IMHO. To quote from your post #1:
So let us assume the content of "$file" is "CAT_TURNAROUND_201811120_075221_3.done" or "/ABC/DEF/CAT_TURNAROUND_201811120_075221_3.done", depending on you using absolute pathes or not. Running the manipulations above would give:
As you can see the one manipulation in the middle does absolute nothing. It can't do anything, because you cut off any "." that is in a files name with your first manipulation! And the last manipulation depends entirely on the exact length of the filename. What would it do with, say, "HORSE_TURNAROUND_201811120_075221_3.done"?
Let us try again: you want to cut off the extension (".done") from the filename. Fine! Then just do so! You know from the for-loop that all the values of "$file" will have exactly ".done" as extension, so there is no need for ${file%.*} . Do it explicitly: ${file%.done} . The net manipulation doesn't do anything (and will never do anything, see above), so get rid of it. Now for the last one: it produces the correct result in your example but as i have shown you it might break with other values. Taking a step back: we want to cut off everything up to the last underscore ("_") from the filename. So let us do that: ${file%_*} .
Notice that we could have done so immediately without all the intermediary steps! This of course is only the case as long as the extension contains no underscore. So until you change your for-loop to something like for file in *.do_ne you can just skip all the steps and use just the last one i suggested.
Would the result be any different if we use absolute pathes? No, because we only cut off from the end where the filename is.