Looking out for the file-watcher

Shell Programming and Scripting
1

Hi friends,

I do not have much thoughts so need any help on the below issue:

I need to create shell script that will find the files & throw an error through job (autosys) when file not found.
Daily we use to receive 3 files from a system.

Obstacles:

1) All 3 files names are same.
2) Timestamp is same.
3) Even the received time may be same
Thats why got stuck.
Script which I have created:

first_time_stamp=`ls -l/apps/rdp/prd/prd1/interfaces/auto/input/archive/*tri003_global*20130131* | awk '{print $8}' | sed -n '1p'`
sec_time_stamp=`ls -l /apps/rdp/prd/prd1/interfaces/auto/input/archive/*tri003_global*20130131* | awk '{print $8}' | sed -n '2p'`
third_time_stamp=`ls -l /apps/rdp/prd/prd1/interfaces/auto/input/archive/*tri003_global*20130131* | awk '{print $8}' | sed -n '3p'`

if[ $first_time_stamp -eq $sec_time_stamp ]; then
echo "Two files received at the same time"
elif [ $first_time_stamp -gt $sec_time_stamp && $third_time_stamp -gt $sec_time_stamp ]; then
echo "Three files reveived & time stamp's are different"
else
echo files are missing"
fi

My inner feeling is that the above script is not good to solve my pupose.

Script should be give this:

[first file not arrived]; then
contact upstream for the first file
[second file not arrived] then
contact upstream for the second file
[third file not arrived] then
contact upstream for the third file
else
echo "All looks good & all 3 files arrived"

Sample file name:

-rw-rw-r-- 1 32598 wmadmin 157335 Jan 31 13:10 tri003_global_201301310810_erd.txt-20130131131318
-rw-rw-r-- 1 32598 wmadmin 157334 Jan 31 15:31 tri003_global_201301311030_erd.txt-20130131153255
-rw-rw-r-- 1 32598 wmadmin 157368 Jan 31 18:24 tri003_global_201301311324_erd.txt-20130131182819

Really appreciate if someone can help me out here.

Thanks very much

2

Why don't you go by the time-stamp in the file name itself instead of checking the received time-stamp to identify the sequence of files received?

tri003_global_201301310810_erd.txt-20130131131318
tri003_global_201301311030_erd.txt-20130131153255
tri003_global_201301311324_erd.txt-20130131182819

There is no way you can have more than one file with the same name, if a file with same name is pushed from upstream it will overwrite the existing file.

3

Thanks for the reply mate but overwrite is not the case here as you already know that some of the part of the timestamp is diff

ex: tri003_global_20130131<first_part>_erd.txt-20130131<Sec_part>

first & sec part i cant control its a system generated, If i would have been aware of the generation of first & sec part then I never asked this question :slight_smile:
hope you got my point but if you have some solution with generating the seq no then plz share it will be helpful

4

Use -t option with ls command to sort by modification time, newest first.

Use -r option to reverse order while sorting:

ls -ltr

Now you will know the sequence of file received.

5

This is in AutoSys? Why don't you just create a box job with 3 file watcher jobs who have dependencies on each other? Unsure if you have the few job types available in 11.0 or the many in 11.3, but it can handle simple things like new file, different file size, etc. Something like

SomeBox
SomeBox.00.EmptyDir - command, empties directory
Somebox.05.BoxStatus - No deps, just runs to see when things finish/status (optional)
SomeBox.10.FileWatch1 - timeout at X time, or at box level if more appropriate
Somebox.20.FileWatch2 - ""
Somebox.30.FileWatch3 - ""

D'no, lots of options - additionally I'm not sure what was bipinajith's simple ls solution. Assuming you have event manager component from workload manager alerts will autofire off based on whatever rules you defined (Otherwise, add a job whose condition is status failure or terminated to email).