So I have a python program that I run, which runs accordingly to options I have listed in a text file (ie user_prefs). Now there are many options listed in this user_prefs.txt, but the one of most interest to me is that of the file path of the time series.
I have over a hundred of these time series that I would like to run this python program through (which I would rather not do randomly).....My question is how could I possibly write a script a shell script to go through and just change the time series file name and then in a loop I would be running the other python program.
All the time series files have a common suffix, but just a numbered prefix, but the number are not of any pattern, rather random. (ie 843523ascii_vF.txt)
I though of trying to write a fortran program to go in and just replace that line but cant think of the way to write the prefix from the file name to inside the program.
Any help would be greatly appreciated, thanks!
---------- Post updated 07-31-12 at 11:13 AM ---------- Previous update was 07-30-12 at 03:00 PM ----------
I was thinking about possibly constructing a shell script using
sed -i "" 's/phrase/newphrase/'g' user_prefs.txt
in order to replace the file name, however, i would want the newphrase to be the filename. I know when using a fortran I could do something like
for File in *.txt (where File new phrase I would like to insert into the txt file)
do
./blah.o < "$File" >
done
so would I be able to similarly do something like this for my problem.
for File in *.txt
do
sed -i "" 's/phrase/"$File"/'g' user_prefs.txt
done
Thanks for the help
What you want is probably possible in awk but you'll need to be more specific.
Show some example input and output, please.
So I have a hundred or so time series files labeled as such:
10831ascii_vF.anom.txt
13165ascii_vF.anom.txt
83437ascii_vF.anom.txt
Now I have a main program to run, which when I run, runs based upon a user_prefs text file which tells the main program which time series to use, and specifies several other options for the main program. However, I am only interested in changing the name of the time series in the varying runs.
So more or less I want to run the main program through for each time series, but want to do so without having to run that program over a hundred times manually each time opening, editing, and closing the user_prefs file.
So all I really need is a command which I could have in a loop to change the name of the time series based upon all the time series within the directory.
So some type of script structured as this:
- For this time series
Change User Prefs, replacing old time series with new time series name (just deleting and replacing a line with new file name)
Run main program coaps.py
Re Loop
I have never worked with awk so any help would be perfect. Thanks.
Yes, that's what the files are named...
Now, what do the contents look like? And what in them do you want changed?
I want nothing changed in the time series files, the file ####ascii_vF.anom.txt simply contain a single list of 2000 observations.
What I want changed is in the user_prefs file, which mandates how the main program is run.
In there, I want the 266th to be altered, all the other lists, options, should remain the same.
The only file to be altered is my user preferences text file, all the time series text files should remain unaltered.
Just want Path=/User/fjjfjf/fjfjfj/timeseries.txt
to change from one ascii file name to another with each run.
So
Path=/User/fjjfjf/fjfjfj/10831ascii_vF.anom.txt
to
Path=/User/fjjfjf/fjfjfj/13165ascii_vF.anom.txt
to
Path=/User/fjjfjf/fjfjfj/83437ascii_vF.anom.txt
Instead of editing the same file over and over, I'd keep an original template which I work from every time. It's not a good idea to edit your originals; if anything goes wrong, you've lost your data.
for FILE in *anom.txt
do
awk '/Path=/ { print "Path=" F; next } 1' F="$FILE" config_template.txt > config.txt
do_something_with $FILE
done
ya I have the original copied and stored in a few other directories so no problem there, but I will give this a go and let you know
---------- Post updated at 05:10 PM ---------- Previous update was at 11:58 AM ----------
Got it to work using sed, just for the possible future need of others I will post how I got it to work
for File in *.txt
do
eval "sed -e 's/oldwordstring/"${File}"/' usertemplate.txt>user_prefs.txt"
#Must make output a different file name
done
The 'eval' is unnecessary there, and frankly eval is a giant security hole. If you somehow end up with `rm -Rf ~/` in your File variable, eval will run that!
What you really needed was double quotes.
sed -e "s/oldwordstring/${File}/" usertemplate.txt>user_prefs.txt
I take it my awk solution didn't work for you. In what way did it not work?
Your awk command should work and I plan on looking into it a little more today. I just found a way to perform what I wanted with the sed command which was a little quicker for me yesterday in terms of familiarity.
I appreciate all the help though.