Look for a file,if not found wait and look again

Shell Programming and Scripting
1

Hi All, i have a requirement
I created a file list as below :

more abc_file_list.txt

Rem_DD.csv
Rem_Non.csv
Rem_Ld.csv
CC_Ld_Non_IRA.csv ...

Above are the 4 files (For now ,we dont know how many in the file) which need to be present else need to get an email for missing files.

script needs to look for all files.. if any file doesnot present on the source directory wait for 60 sec and again look for the missing file if not found look for the next file.. i written the below script but i was unable look for second time

#!/usr/bin/ksh
set -vx
###### Variables #########
FILE_LIST=/home/username/abc_file_list.txt
DATA_SRC_DIR=/home/username
Log_file=/home/username/test.log
SUPPORT_EMAIL=abc@someemail.com
NumofFiles=`wc -l<"$FILE_LIST"`
#Check for the CISS files in sleep mode
dt=`date +"%Y-%m-%d %X"`
printf "started at $dt \n " > $Log_file
FILE_FOUND=N
export i=0
while [ $FILE_FOUND = N ]
do
  ip=$FILE_LIST
  for f in `cat "$ip"`
  do
    if [ -r $f ]; then
      printf " CISS File $f exists in "$DATA_SRC_DIR" \n" >>$Log_file
      FILE_FOUND=Y
      export i=`expr $i + 1`
    else
      export i=`expr $i + 1`
      printf " abc File $f Not received in "$DATA_SRC_DIR" \n" >>$Log_file
      #Check for No.of tries
      if [ $i -eq $NumofFiles ]; then
        cat $Log_file| mailx -s "abc File $f Not received and Reached MAX checks" $SUPPORT_EMAIL
        exit 1
      else
        sleep 60
      fi
    fi
  done < "$FILE_LIST"
done
2

This will show you immediately which of the files in abc_file_list.txt is missing in the PWD:

grep -hv "$(ls $(< abc_file_list.txt) 2>-)" abc_file_list.txt

Put a

 while ((i++ < $MAX)); do ... done

around it to check MAX times, and then send your alert mail. Should work in recent shells (bash, ksh, ...)

3

Not sure if this is what you are looking for but this is something I've used before in the past. This will loop through every 60 secounds looking for any file that is not found and sending out an alert of the files. Once all files are there the script will stop. Not sure if you need to take action if all files are found.

#!/usr/bin/ksh
set -vx

FILE_LIST=abc_file_list.txt
DATA_SRC_DIR=/home/username
SUPPORT_EMAIL=abc@someemail.com

NumofFiles=`wc -l ${FILE_LIST}|awk '{print $1}'`

filecheck ()
{
  x=0
  for file in `cat ${FILE_LIST}`
  do
      if [ -f ${file} ];then
          x=$(( ${x} + 1 ))
      else
          missing="${missing} ${file}\n"
      fi
  done
  missing=`echo -e "${missing}"|awk '{if ( $1 != "") print $0}'|sed "/^$/d"|sed "s/^ *//g"`

}


filecheck

if (( ${x} != ${NumofFiles} ));then
     until [ ${x} = ${NumofFiles} ]
     do
        if [ "${missing}" ];then
             MSG=`(echo "Missing Files:";echo;echo "${missing}")`
             echo "${MSG}"|mailx -s "abc File Not received in "$DATA_SRC_DIR" " ${SUPPORT_EMAIL}
             missing=""
        fi
        sleep 60
        filecheck
     done
fi
4

bash, ksh:

cd somedatasrcdir
alldone=0
cnt=0
MAX=1000
while ((  alldone==0  ))
do

        ok=1
        ((cnt+=1))
        msg=""
        while read file
                if [ ! -f "$file" ] ; then
                        ok=0
                        msg="$msg $file"
                fi
        do < filelist.txt

        ((  ok==1     )) && alldone=1 && continue
        ((  cnt>MAX )) && alldone=1 && continue

        sleep 60
done


if ((  ok==0  )) ; then
        echo "file(s): $msg not received" | mailx ...
fi