I have this list of files . Now I will have to pick the latest file based on some condition

Shell Programming and Scripting
1 
 
 3679   Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log
   0    Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log
   3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log
     22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log
      0 Jul 20 05:50 belk_rpo_error_**po9999992**_07202014.log
    411 Jul 21 06:30 belk_rpo_error_**po9999992**_07212014.log
    742 Jul 21 07:30 belk_rpo_error_**po9999991**_07212014.log
      0 Jul 23 2014 belk_rpo_error_**po9999991**_07232014.log

First column represents Size in Bytes.

For a PATRICULAR Order_No(Marked with ** **)

  1. If the latest file is 0 kB then we will discard it (rest of the files with same Order_no as well)
  2. if the latest file is non Zero then I will take it.(Only the latest one)

Then append the contents in a txt file . My expected output would be ::

 
 411    Jul 21 06:30 belk_rpo_error_**po9999992**_07212014.log
   3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log
    22   Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log

I am at my wits end here. I cant seem to figure out how to compare dates in Unix. Any help is very appreciated.

2

Dear LoneRanger,
I have a few to questions pose in response first:-

  • Is this homework/assignment? There are specific forums for these.
  • What have you tried so far?
  • What output/errors do you get?
  • Are the ** shown actually part of the file name or just to highlight the important part?
  • How are you generating the listing? We might be able to process it in-line rather than writing and processing a file.
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)

Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.

We're all here to learn and getting the relevant information will help us all.

3

Is this homework/assignment? There are specific forums for these.
What have you tried so far? No It's not.
What output/errors do you get? I am not getting expected output.
Are the ** shown actually part of the file name or just to highlight the important part? Just to highlight the important part
How are you generating the listing? We might be able to process it in-line rather than writing and processing a file.

I am generating it using

find . -name "*.log" -mtime -8

What OS and version are you using? I am not sure. Some flavor some Linux.
What are your preferred tools? (C, shell, perl, awk, etc.) Shell
What logical process have you considered? (to help steer us to follow what you are trying to achieve) I am trying to pick the filename , date part ,size and the order_number part and insert all into a DB table from shell script. do the comparison there an pick the right file name and pass it back into shell script for further processing.

My code so far :: it's not giving me expected output though-

 
 export LOGDIR=/home/rmsbatch/testdata
 
# TEMPDIR : Path where all non zero files need to be moved after search.
export TEMPDIR=/home/rmsbatch/testdata/TEMP
 # RESULTFILE : Path of the Final File 
export RESULTFILE=${TEMPDIR}/belk_rpo_error.txt
 ###############################################################################
#                                     MAIN                                    #
###############################################################################
set -x
if [ $# -ne 1 ]
  then
    echo "Please Provide Date range"
    echo "Example: ./belk_rpo_shell_script.ksh "
    exit 1
fi
 DATE_RANGE=$1
echo $DATE_RANGE
echo " SET ECHO OFF
SET NEWPAGE 0
SET SPACE 0
SET PAGESIZE 0
SET FEEDBACK OFF
SET HEADING OFF
SET TRIMSPOOL ON
SET TAB OFF
           select to_char(sysdate,'YYYYMMDD')
                                  from dual ;" > date.sql
 END_DATE=`sqlplus -s $UP <<EOF @date.sql
exit;
EOF`
 echo " SET ECHO OFF
SET NEWPAGE 0
SET SPACE 0
SET PAGESIZE 0
SET FEEDBACK OFF
SET HEADING OFF
SET TRIMSPOOL ON
SET TAB OFF
           select to_char(sysdate-$DATE_RANGE,'YYYYMMDD')
                  from dual ;" > date1.sql
 START_DATE=`sqlplus -s $UP <<EOF @date1.sql
exit;
EOF`
 echo $START_DATE
echo $END_DATE
 cd $LOGDIR
 if [ ! -d "$LOGDIR/tmp" ]; then
  mkdir $LOGDIR/tmp
fi
 touch -t ${START_DATE}0000 ./tmp/NEWSTARTDATE
touch -t ${END_DATE}2359 ./tmp/NEWENDDATE
 find . -type f ! -size 0 \( -newer ./tmp/NEWSTARTDATE -a \! -newer ./tmp/NEWENDDATE \) -print |grep '.log'|
 while read fname
do
cp -p $fname  $TEMPDIR/${fname}
  
  PONUM=$(echo "${fname:17:9}")
  find . -type f  -size 0 \( -newer ./tmp/NEWSTARTDATE -a \! -newer ./tmp/NEWENDDATE \) -print |grep $PONUM |  
   
  while read fn
  do
    rm -f $TEMPDIR/$fn
    rm -f $TEMPDIR/${fname}
  done 
done
  
 cd $TEMPDIR
 
if test -f $RESULTFILE; then
  rm -rf $RESULTFILE;
fi
# Create an empty $RESULTFILE
touch -t ${START_DATE} $RESULTFILE
  
 for f in `ls *`; do
  cat $f >> $RESULTFILE;
done
 
nawk '!x[$0]++' $RESULTFILE > temp
mv temp $RESULTFILE
 
sed '/@0PACKAGE_ERROR@1ORA/d' $RESULTFILE > temp
mv temp $RESULTFILE
 
if [ -d "$LOGDIR/tmp" ]; then
  rm -rf $LOGDIR/tmp
fi
 
#for f in `ls belk_rpo_error_po* 2>/dev/null`
#do
#  rm -rf $f
#done
4

Something like

#! /bin/bash
ls -ctl | while read a b c d SIZE f g h NAME 
do
if [[ ${SIZE} -ne 0 ]] 
then
  echo  ${NAME},${SIZE}
  exit
fi
done
5

For

find . -name "*.log" -mtime -8

..... do you mean

find . -name "*.log" -mtime -8 -ls

That could give the listing you have.

For the listing, am I okay to assume that it's better to show it as this:-

3679   Jul 21 23:59 belk_rpo_error_po9324892_07212014.log
   0    Jul 22 23:59 belk_rpo_error_po9324892_07222014.log
   3679 Jul 23 23:59 belk_rpo_error_po9324892_07232014.log
     22 Jul 22 06:30 belk_rpo_error_po9324267_07012014.log
      0 Jul 20 05:50 belk_rpo_error_po9999992_07202014.log
    411 Jul 21 06:30 belk_rpo_error_po9999992_07212014.log
    742 Jul 21 07:30 belk_rpo_error_po9999991_07212014.log
      0 Jul 23 2014 belk_rpo_error_po9999991_07232014.log

.... where the red highlights the selection bit and the bold are those to be picked up? Note that find does not give you a listing that is sorted in any way in particular. It just reads the directories and gives them to you in the order written, which may or may not be what you need.

Okay, now what about the rest of the questions?

  • Is this homework/assignment? There are specific forums for these.
  • What output/errors do you get?
  • What OS and version are you using? Run uname -a and paste the result in here in CODE tags.
  • What are your preferred tools? (C, shell, perl, awk, etc.)

As for "What logical process have you considered?" , well :eek:!

Okay let's make this simpler. From what you wrote, I think you want to take (maybe read in, maybe loop for all) the order number which in your examples is po........ and get the most recent file that is not empty.

Am I correct?

If so, perhaps something more like this might help:-

ls -1t *${order_no}* | while read file       # For each file matching the search in a simple list, newest first
do
   if [ -s $file ]            # Test if it's a file with bytes in it
   then
      use_file=$file
      break
   else
      echo "Ignoring empty file $file"
   fi
done

echo "I selected $use_file"

Let me know what you think and the answers to the other requests, if you could be so kind. There may be limitations based on what you have so that may set our choices.

Robin

6

Robin , Thanks .

I will try your suggestion tomorrow and get back to you..