Run perl script on files in multiple directories

Shell Programming and Scripting
1

Hi,

I want to run a Perl script on multiple files, with same name ("Data.txt") but in different directories (eg : 2010_06_09_A/Data.txt, 2010_06_09_B/Data.txt).

I know how to run this perl script on files in the same directory like:

for $i in *.txt
do
perl myscript.pl $i > $i.new
done

But my filenames are same, and they are in different directories. How to go about this? This may be a very basic question, but I am just a beginner!!!

Thanks!

2

Simply pass full/relative paths with directory name to your perl script. (This is assuming your perl script does not expect the file to be in the current working directory).

## Assuming the following directory structure:
## /abc/xyz/2010_06_09_A
## /abc/xyz/2010_06_09_B
## ...
## /abc/xyz/2010_06_09_Z

cd /abc/xyz
for i in */Data.txt
do
  perl myscript.pl $i > $i.new
done

If your perl script needs the file to be current working directory:

cd /abc/xyz
for i in */Data.txt
do
  dir=${i%/*}
  file=${i##*/}
  cd "$dir"
  perl myscript.pl ${file} > $i.new
  cd -
done
1 Like
3

.... <Shell script snipped> ...
a_programmer's shell for loop is far better.

Perl globbing example follows -

You can iterate through all files named "Data.txt" in the Perl program itself, thereby avoiding the shell's for loop.

Example of your Perl script "myscript.pl" -

#!/usr/bin/perl -w
...
while ($file = glob("<your_directory>/*/Data.txt")) {
  # Now you have the full path in $file.
  # Process it the way you want.
  # You can even create and write to "$file.new" thereby 
  # avoiding the shell's redirection operator.
  ... your code here
}

HTH,
tyler_durden

1 Like
4

I tried running the script according to a_programmer's code, but it is now giving me an error saying:

./myScript.sh: $i: is not an identifier

The following is my code:

#!/bin/tcsh
cd /abc/xyz
for $i in */Data.txt
do
    perl myScript.pl $i > $i.new
done

Am I overlooking something obvious in this?
Thanks again!

5

Remove the "$" sign and try

1 Like
6

Yes, you are. $i is a Perl variable; not a shell variable.

tyler_durden

7

I am still not getting the desired output. Moreover when I print $i it is:

*/Data.txt

It is not taking the entire path for the file ?
What may be wrong?

---------- Post updated at 11:12 AM ---------- Previous update was at 11:11 AM ----------

When I run this :

cd abc/xyz

for i in */Data.txt
do
    #echo $i
    perl myScript.pl $i > $i.output
done

I get this error:

./yourScript.sh: line 10: */Data.txt.output: No such file or directory

Need your feedback??!!

8

Did you replace "cd /abc/xyz" with the common parent directory where all your data files are?

So, if you type the following on the command line, you should see a list of Data.txt files:

cd /abc/xyz
ls -lrt */Data.txt
9

I did add the entire path for my directories in my code.
What is this lrt option? I am not using this in my script!!!