processing an input file and then the output file

Shell Programming and Scripting
1

Hi.

I am new to scripting and could really do with some advice on the best way to put a script together. Here is the scenario I am working to;

  • i will get files via ftp to a tmp directory on the server
  • all files will have a unique file name but with the same extension (.USM)
  • for each file I need to run an application process that creates a new file by prefixing each file with 'mt' and appending '_file_01
  • therefore from 1234.USM a new file name mt1234_file_01 will be created.
  • a second process will then take mt1234_file_01 and make mt1234_file_02
  • once the application processes have been created I will then need to move all the files to a new directory so that the FTP side of the script can run the next day and only new files are processed again.

The bit I am stuck on is defining the three types of files so that I can run a foreach command processing that set of files.

I have something if there is just one set of files which is like this;

while (1 == 1)
   if ( $status == 0 ) then
	set MARC_FILES = (*.USM)
   else
      set MARC_FILES = " "
   endif
   foreach F ($MARC_FILES)
       if ( "$F" != "*.USM" ) then
# -- application script that takes a file and makes a new one
       csh -f $proc/p_file_01 TST01,$F,mt$F_file01,
         set MARCLOADERRORS=0
         echo " " > $DATA_DIR/marcload_email_txt.dat
endif
end

What I need some help with is the logic in doing a foreach of the 1st set of newly created files and then the 2nd set of newly created files.

$F is set in the script above.
$F1 should me something like mt$F_file_01 ($F needs to lose the .USM extension)
$F2 should then be mt$F_file_02 (again where $F needs to lose the .USM extension)

help....
any pointers would be great as I am not sure where the best place is to start.
thanks,
Mark

---------- Post updated at 05:00 PM ---------- Previous update was at 04:59 PM ----------

forgot to mention. this is using .csh

the application scripts are using .csh so i wanted to keep this additional script in the same programming language.

2

How about this:

set BN=`basename $F .USM`
set F1=mt${BN}_file_01
set F2=mt${BN}_file_02

B.T.W., I dont think if ( "$F" != "*.USM" ) then is doing what you want. Use switch/case to do wildcard tests:

switch ( $F )
  case "*.USM":
     # Code to process *.USM here ...
  breaksw;
endsw
3

thanks for the advice Chubler_XL. I made a test script and managed to get someething to work if there was only one file named .USM.

set F=("*.USM")
set BN=`basename $F .USM`
set F1=mt${BN}_file_01
set F2=mt${BN}_file_02

echo $BN
echo $F1
echo $F2

switch ( $F )
  case "*.USM":
   echo 'real script will change' $BN 'into '$F1
   echo 'real script will then change '$F1 ' into '$F2
  breaksw;
endsw

The above script fails if there are multiple files with .USM extensions. I get this messages 'basename: too many arguments', '*.USM: Ambiguous.'

Could you help any further please?

4

I was thinking more along the lines of this

#!/bin/csh
foreach F (*)
   switch ($F)
       case "*.OTH":
           echo "Some other processing you might want to do with .OTH file - $F"
       breaksw;
       case "*.USM":
           set BN=`basename $F .USM`
           set F1=mt${BN}_file_01
           set F2=mt${BN}_file_02
           echo 'real script will change' $BN 'into ' $F1
           echo 'real script will then change' $F1 'into ' $F2
        breaksw;
   endsw
end
5

thanks again for your help and i've now got this working.

i'm stuck on a new problem that i am trying to solve. my original plan was to download all the files and process them in a holding directory. then, after processing, i was going to move them to the correct directory along with past files.

i can't do this now as the scripts need to run in a specific directory.

therefore the problem i am trying to work out is how to calculate what existing .USM files there are in a folder and run the scripts for each new .USM file only. as you can tell i am well outside my comfort zone...

---------- Post updated at 04:20 PM ---------- Previous update was at 02:43 PM ----------

i think i have the answer...

i download the files to a holding directory. Then do a

ls *USM >> textfile.txt

then i can move all my files to the directory where they are to be processed.

and then i can use;

foreach F (`cat textfile.txt`)

this does a foreach on only the files listed in the text file. the script will not process old files in that directory as they are not on that list.

Thanks for all your help.

6

if 1234.USM is file comin everyday. then you need that like this "mt1234_file_01".
then use cat 1234.USM > $TgtPath/mt1234_file_01.
just save that file in one directory like mt1234_file_01.
after file creation completed then take the last file and store that last two numbers into one variable. after that increment +1 to that and save the next file.

if you use date then files should not overwrite.

regards
rajesh