Using Shell Script To Loop Program Through Multiple Text Files

Hello,

So I have approximately 300 files of raw data (.txt) files that I am using to perform statistical analysis. I have been able to construct a Fortran program that is able to perform my statistical analysis on a file by file basis.

However, I now want to be able to loop program through all of the text files through the use of a shell script, rather than going running the program ~300 times which is not realistic.

I'm thinking about performing some sort of a Do Loop in the terminal to loop through the program with for each file. However, I do not know how to change my fortran program such that I can replace the name of the file to open/write to.

Any help would be greatly appreciated.

Thanks

If you can have your fortran program read from standard input and write to standard output, then it will be easily automated. Just redirect into and out of the program.

Your program may become actually simpler; no file-opening code at all

for FILE in *.txt
do
        ./myprogram < "$FILE" > "$FILE.out"
done

Assuming your fortan code can handle arguments...

for FILE in *.txt; do fortran_progname ${FILE}; done

Thanks for the help, I'll mess around with it for a little bit.

So "$FILE" serves as the name of the text file. So in my Fortran program would I have to put $File where I am attempting to open the txt files? That way we are running the program through with each respective txt file?

No. The shell will evaluate the variable $FILE and perform Variable Expansion to convert the variable into the actual filename.

If your program can handle arguments, the code would then use Parameter Expansion to evaluate the 1st, 2nd, 3rd argument, etc...

You absolutely shouldn't use the variable $FILE in your fortran code, unless it is defined explicitly.

Also note that shell variables are case sensitive, and that $FILE is NOT the same as $File

How do you execute the fortan program for a single file ??

Well I currently just have a basic OPEN statement of the form

OPEN (10,file='389534ascii_vF.txt',status='old')

Then simply read in the data and perform calculations and then output it.

Compile it using
gfortran filename.f90 -o filename.o
Run
./filename.o

Is that what you were wondering?

Well, yes. It has been 6-8 yrs since I have read any fortran code, but I believe arguments or parameters are not valid.

I would follow up on Corona688's suggestion of modifying the program to accept standard input and output. Otherwise sorry, over my head.

Sorry I know this is probably a very basic question but I have not been able to grasp this idea really of what you mean by standard input/standard output.

My current open statement reads as
OPEN(unit=10,file='nameofsinglefile.txt',status='old')
Would I adjust the above line to
file='$FILE.txt'

Any help would be greatly appreciated.
Thanks.

I know nothing about Fortran, but by the power of Google:

Fortran Input/Output

Whenever I try to run this script, I get the error For: command not found and Do: command not found.

What shell are you running it in?

I'm running in Bash shell.

I have gotten past my original problem where it was just saying command not found. However, now I am getting the message that the file does not exist when I run through the shell script, however, if I run the fortran script with just one file it does run...so I know the files do exist. And they are all in the same directory.

My bash script is this

#!/bin/bash
# Running through Fortran Program used to find anomaly of Rain and Temperature

#FILES="/Users/jduncan/Desktop/rawdata/"

for File in *.txt
do
     ./runthrough.o "$File" 

done

And I simplified the fortran code to illustrate if it was running through the files correctly as

Program runthrough

IMPLICIT NONE

CHARACTER(31)::text,header,header2
CHARACTER(15)::station
CHARACTER(2)::state
CHARACTER(10)::county
REAL::lat,lon
INTEGER::i


OPEN(10,file='"$File"',status='old')

READ (10,'(a10,a15)') text, station
READ (10,'(a8,a2)') text, state
READ (10,'(a7,a10)') text, county
Do i=1,4
   READ (10,'(a)') text
ENDDO
READ (10,'(a27,f8.5)') text, lat
READ (10,'(a27,f8.5)') text, lon
!Do i=1,3
!   READ (10,'(a)') text
!ENDDO
READ (10,'(a)') text
READ (10,'(a)') header
READ (10,'(a)') header2

print*,header

END Program runthrough

for File in *.txt

Here you say nothing about WHERE you are and so depending where you are , you might effectively have no *.txt...

Do I need to declare a path even if I am in the same directory?

In that directory when I perform a ls *.txt I have approximately 200 files, and my .f90 code is within the same directory.

I would... because if the script is in your PATH, it cane be executed anywhere...
So for safety:

#!/bin/bash
# Running through Fortran Program used to find anomaly of Rain and Temperature

#FILES="/Users/jduncan/Desktop/rawdata/"

cd <to where the *.txt are>


for File in *.txt
do
     ./runthrough.o "$File" 

done

This ./runthrough.o is where?

Second thought: Since I dont know what ./runthrough.o does:
Try to see if loop is working:

#!/bin/bash
# Running through Fortran Program used to find anomaly of Rain and Temperature

#FILES="/Users/jduncan/Desktop/rawdata/"

for File in *.txt
do
     # ./runthrough.o "$File"  comment out for the test: If test works,the culprit is here somewhere...
   ls -l $FILE

done

OPEN(10,file='"$File"',status='old')

FORTRAN is not shell. "$File" doesn't work here.

You can get variables you've exported with GETENV, or things you've fed it on the commandline with GETARG, but we don't seem to be FORTRAN experts, ergo it would really be much simpler to just follow the suggestion I gave you days ago: Don't bother opening any files inside FORTRAN at all. Read from standard input, write to standard output. This is often how UNIX utilities are supposed to work anyway -- makes things easier to put in a pipe chain.

'standard input' and 'standard output' amount to pre-opened file numbers, stdin being either 5 in F77 or 100 in F95, and stdout being 6 in F77 or 101 in F95 (according to According to this). Reviewing the thread, I see this was even linked earlier.

When you don't redirect them, they inherit it from whatever it's being run from -- in a terminal, it would get your keyboard as stdin and your screen as stdout. But you can send them anywhere you want with the shell:

./myprogram < inputfile > outputfile

So if myprogram was an F77 program, writing to 6 would write to outputfile and reading from 5 would read from inputfile...

Thanks for your Enlightenment Corona688 !

Thanks, I was confused earlier with the 'standard input/output' because I always though you had to have a file name no matter what unit identifier you used. That's where the confusion was.

I'm still having one small problem but I'll play around with it and see what happens...But I am making progress!!!

I think FORTRAN predates the concept of "file", so...