Weird date difference problem

Shell Programming and Scripting
1

I am trying to find the difference in days between 2 dates.
I have to extract the 1st date from a filename, which i did using the awk command.
I have to compare this date to today's date and if the difference is greater than 30 days, do something, else do something else.

This is what i wrote


INVDATE=`echo "SHIP_606400_2008233202_20080206_070308083544.xml"|awk -F_ '{print $4}'`
echo $INVDATE
TODAY=`date +"%Y%m%d"`
echo $TODAY
(( diff=$TODAY-$INVDATE))
echo $diff
if [ $diff -gt 30 ]; then
     #do a
else
  #do b
fi

The result that I got was

20080206
20080311
105

The difference between the dates is not 105 days, but 34 days.

What am i doing wrong - I am pretty new to Shell programming and am learning through this site.

Thanks in advance.

2

It's because it's subtracting one number from the other. The subtraction you're doing has no concept of dates.

There are many ways to go about this. What exactly are you trying to do?

A couple of possibilities:

A find command using the -mtime and -exec options (you can search for all files over 30 days old and do something to them).

A short script which returns the difference in days between a file's last-modified time and the current system time, to be embedded in your script.

As usual, providing as much information as possible will get you the best results. Also, try searching this site -- your question has probably been answered many times. It depends on exactly what you're trying to do.

ShawnMilo

3

Shawn,
Thanks for the quick response. This is what I am trying to do - these XML files are shipment files and these shipments need to be loaded into a database, dependent on some conditions.
This XML can be created any day(hence the last modified date is not an option), but it has to be loaded into the database only if the shipment is for within the last 30 days, otherwise it has to be archived to a different folder, without loading into the datatbase.
Is there any way to convert these numbers to date in UNIX (a very warped up way would be to connect to the database and do the calculation there - very unsophisticated)
Thanks again for your help.

4

Okay, I think I get it now. You have a directory with any number of XML files. You need to read them to find out the date (stored as text) from a certain position in a record in the XML file. Depending on whether that date falls within a certain range, you either want to process the file into your database, or move the whole XML file to another directory, where it will wait to be checked later.

Is that correct?

Can there be multiple orders per XML file? If so, what if there are different dates for the different orders?

Please post an example of the XML file (if it's small enough), or at least the line(s) with the date you are trying to check. If there are multiple lines in the XML file which do not have dates, explain how the program should figure out if a particular line should be checked or ignored.

ShawnMilo

5

This is a bit similar to an older post: http://www.unix.com/shell-programming-scripting/56451-days-difference-between-two-dates.html

I would suggest convert both the dates to "epoch" seconds, then do a awk operation to convert the epoch diff secs to num days.

something like:

$ cat days.sh
D1=`echo "SHIP_606400_2008233202_20080206_070308083544.xml"|awk -F_ '{print $4}'`
E1=`date +%s -d"$D1"`
E2=`date +%s` #present time
((diff_sec=E2-E1))
echo - | awk -v SECS=$diff_sec '{printf "Number of days : %d",SECS/(60*60*24)}'

$ sh days.sh
Number of days : 35

//Jadu

6

Jadu -
That requires GNU date, so you should mention that not all UNIX boxes have it as what you show "exceeds" POSIX standards.

7

HI Shawn,
NO- the date is in the filename itself - SHIP_606400_2008233202_20080206_070308083544.xml - in this case "20080206".
Also, there is only 1 shipment per XML file for the date mentioned in the file name.

Thanks for your help. Is there any way to convert a string to a Julian date?

8

Hi Jadu,
I did see your previous post yesterday and tried doing this - but I do not have GNU date and E1 just returns me today's date.
IS there any other way - the deadline for this is fast approaching.:slight_smile:
Thanks...

9

Okay, play with this:

$ date -d "20080206" +%s
1202274000

$ date +%s
1205330860

In that case, subtract 1202274000 from 1205330860. The result is 3056860. Divide that by 86,400 (24 * 60 * 60) to convert seconds to days. In this case, it's 35.38 days.

The point being that if you can extract the date from your filename (which you have already done), you can use the "+%s" formatting (seconds since epoch) to compare the current date/time (in epoch) to it. Take the difference and divide it by days (24 * 60 * 60) and see if that's less than 30.

You can probably modify the script you originally posted with this information fairly quickly and be up and running.

Please post the result to the thread when you get it working (or nearly so).

ShawnMilo

10

Hi Shawn,
OK i tried this - and this is the error that I got -

date: illegal option -- d
usage:  date [-u] mmddHHMM[[cc]yy][.SS]
        date [-u] [+format]
        date -a [-]sss[.fff]

Is it a version thing?

Thanks for all your help.

11

What's your Unix/Linux version? I'm on Ubuntu Linux, myself. Worst case, we can do this is a Python or Perl script. But I think we're already close to solving this.

Type:

man date

and see if there's an option you can use to pass it a date as a string instead of having it default to the current system time.

ShawnMilo

12

Thanks Shawn
The unix version i am using is

SunOS retailtest 5.9 Generic_122300-13 sun4u sparc SUNW,Sun-Fire-V240

tried getting help on the date command and the only 2 options available are -a and -u.

Also, have never written a Perl script in my life.... :o

Thanks again for all your help

13

Let's try it in Python.

 $ ./timeComparison.py SHIP_606400_2008233202_20080206_070308083544.xml 
35.4838506172

Code:

#!/usr/bin/env python

import time
import sys

one_day = 24 * 60 * 60

#Assume the first argument is the file to check
file = sys.argv[1]

#split the filename into an array
sections = file.split("_")

#the date we want is the fourth section of the filename
file_date = sections[3]

file_date = int(time.mktime(time.strptime(file_date, '%Y%m%d')))

now = time.time()


age_in_days = (now - file_date) / one_day

print age_in_days
14

Thanks Shawn - but having problems with this too.
I created a new file, timeComparison.py using the vi editor, (actually just copied your code there) and then trid running the commant, and it gave me an error:

/usr/bin/env: No such file or directory

I am creating this in the bin directory, where i usually create my shell scripts:

/u01/app/rms9.0/prd/oracle/proc/bin

Thanks again for your help

15

Also, I checked the /usr/bin directory and this is what i got

retailtest:RMST90:/u01/app/rms9.0/prd/oracle/proc/bin>ls /usr/bin/en*
/usr/bin/enable  /usr/bin/env
16

Probably you haven't installed Python. Anyways, i would not suggest installing it just because the way to solving your problem is not obvious.

Probably this is what you really need.

I hope this helps.

bakunin

17

Hopefully you have no problem. Try running the script this way:

python scriptname.py filename.xml

That should work. If it does not, run

which python

If there is no result, then you don't have Python on that system. In which case, I will be sad.

To fix my script (assuming you can run it by calling Python explicitly), try doing this:

which env

Whatever comes up is what you should replace the first line in my file with.

ShawnMilo

18

okay - bad news - no python on the system.
good news - there is perl on the system - so maybe we can write a perl script to do the same - right now I am sitting with a perl book in my hands...:eek:

19

Okay, here are the guts. See if this executes on your system:

perl -e 'use Time::Local; $fileDate = timelocal(0, 0, 0, 12, 2, 2008); $now = time(); $diff = ($now - $fileDate) / (24*60*60); print "The file is $diff days old.\n"'

Note: The month must be decremented by one, because timelocal uses months 0-11 instead of 1-12.

If that command works on your system, then you should be able to piece the rest of the Perl script together around it without too much trouble.

ShawnMilo

20

:)yippeee!!!
this worked - it gave me the following output :

The file is 0.603263888888889 days old.

so i guess this was testing against 03/12/2008 00:00:00 - so I can pass my date that i extract to the file as parameters to this perl program and return the difference back to my shell script, right??

Thanks again for your help - I might bug you again if i have trouble with writing the perl program...:slight_smile: