How to insert text within a file?

Hi,

I am trying to check for missing dates in a file and would want to insert the missing date into the file.

Currently the script is as below

#!/bin/ksh

dates="dates"
cat ${dates} | grep -v "^#"

curr_month=`date '+%m`
curr_day=`date '+%d`
curr_year=`date '+%Y`

#curr_month=02
#curr_month=2016
#leap=${curr_year}; math=`echo "$leap%4" | bc`; [ ! -z $leap ] && [ $math -eq 0 ] && echo "$leap is a leap year!" || echo "$leap isn't a leap year";
leap=${curr_year}; math=`echo "$leap%4" | bc`; [ ! -z $leap ] && [ $math -eq 0 ] && leapyear="Y" || leapyear="N";

case "${curr_month}" in
   01|03|05|07|08|10|12 )
      max_days="31"
      ;;
   04|06|09|11 )
      max_days="30"
      ;;
   02 )
      case "${leapyear}" in
         Y )
            max_days=29
            ;;
         N )
            max_days=28
            ;;
      esac
      ;;
esac

echo
echo "curr_month = ${curr_month}"
echo "curr_day   = ${curr_day}"
echo "curr_year  = ${curr_year}"
echo "leapyear   = ${leapyear}"
echo "max_days   = ${max_days}"
echo

check_day=${curr_day}
while [[ ${check_day} -le ${max_days} ]]
do
   next_date="${curr_year}-${curr_month}-${check_day}"

   if [[ -z "`grep ${next_date} ${dates}`" ]] ; then
      echo "- Missing date ---> ${next_date}"
   fi

   let check_day=${check_day}+1
done

A sample run of the script gives output as below:

$ ./x.ksh
2015-05-01
2015-05-04
2015-05-05
2015-05-06
2015-05-07
2015-05-08
2015-05-12
2015-05-14
2015-05-15
2015-05-18
2015-05-27
2015-06-02
2015-06-03
2015-06-04
2015-06-05
2015-06-06
2015-06-07
2015-06-08
2015-06-09
2015-06-10
2015-06-11
2015-06-12
2015-06-13
2015-06-14
2015-06-15
2015-06-18
2015-06-22
2015-06-23
2015-06-24
2015-06-26
2015-06-30
2015-07-01

curr_month = 06
curr_day   = 18
curr_year  = 2015
leapyear   = N
max_days   = 30

- Missing date ---> 2015-06-19
- Missing date ---> 2015-06-20
- Missing date ---> 2015-06-21
- Missing date ---> 2015-06-25
- Missing date ---> 2015-06-27
- Missing date ---> 2015-06-28
- Missing date ---> 2015-06-29

I want to insert the missing date into the file in the right ascending order.

Any advice will be much appreciated. Thanks in advance.

Given this script called, say, list_days

#!/bin/bash

MDarr=(0 31 28 31 30 31 30 31 31 30 31 30 31)
function mthdays() { printf "%d\n" $((MDarr[$1] + (M!=2?0:!($2%4) && ($2%100) || !($2%400)))); }

IFS=- read Year Month  Day SecEpoch R <<< $(date -d"$1" +%F-%s)

seq 0 $(($(mthdays $Month $Year) - Day )) |  while read TMP; do printf "%(%F)T\n" $((SecEpoch + 86401 * TMP)); done

(which, admittedly, takes advantage of some recent bash features that don't necessarily exits in other shells),

./list_days 2015-6-18 | sort -u - dates

should do what you requested: print the porous list that you gave and fills in the holes from the start date (parameter 1) till EOMonth. While the mthdays function should be leap year proof, I added the 1 sec in the loop to make it leap second proof. On second thought, as this usually happens on the last day of a month, it will not be relevant...

Hi,

Thanks for your reply. Unfortunately, I don't have bash on this Solaris server.

However, I did try your script on one of the Linux server and it is giving errors. Note below.

$: ./listdays 2015-6-18 | sort -u - dates
date: illegal option -- d
date: illegal option -- 2
date: illegal option -- 0
date: illegal option -- 1
date: illegal option -- 5
date: illegal option -- -
date: illegal option -- 6
date: illegal option -- -
date: illegal option -- 1
date: illegal option -- 8
usage:  date [-u] mmddHHMM[[cc]yy][.SS]
        date [-u] [+format]
        date -a [-]sss[.fff]
./listdays: line 4: MDarr[]: bad array subscript
./listdays: line 4: MDarr[]: bad array subscript
./listdays: line 4: MDarr[] + (M!=2?0:!(%4) && (%100) || !(%400)): syntax error: operand expected (error token is "%4) && (%100) || !(%400))")
./listdays: line 8: seq: command not found
# e.g. 2014-10-01
2015-05-01
2015-05-04
2015-05-05
2015-05-06
2015-05-07
2015-05-08
2015-05-12
2015-05-14
2015-05-15
2015-05-18
2015-05-27
2015-06-02
2015-06-03
2015-06-04
2015-06-05
2015-06-06
2015-06-07
2015-06-08
2015-06-09
2015-06-10
2015-06-11
2015-06-12
2015-06-13
2015-06-14
2015-06-15
2015-06-18
2015-06-22
2015-06-23
2015-06-24
2015-06-26
2015-06-30
2015-07-01

OS version?
bash version?
date version?
As stated, must be recent.

Hi,

Run on another server and gives the error below:

 
 $ ./listdays 2015-6-18 | sort -u - dates
./listdays: line 8: printf: `(': invalid format character
2015-05-01
2015-05-04
2015-05-05
2015-05-06
2015-05-07
2015-05-08
2015-05-12
2015-05-14
2015-05-15
2015-05-18
2015-05-27
2015-06-02
2015-06-03
2015-06-04
2015-06-05
2015-06-06
2015-06-07
2015-06-08
2015-06-09
2015-06-10
2015-06-11
2015-06-12
2015-06-13
2015-06-14
2015-06-15
2015-06-18
2015-06-22
2015-06-23
2015-06-24
2015-06-26
2015-06-30
2015-07-01
# e.g. 2014-10-01
$ bash --version
GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.
$ date --version
date (GNU coreutils) 5.97
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software.  You may redistribute copies of it under the terms of
the GNU General Public License <The GNU General Public License v3.0 - GNU Project - Free Software Foundation>.
There is NO WARRANTY, to the extent permitted by law.
 Written by David MacKenzie.
$ uname -a
Linux [hostname] 2.6.18-404.el5 #1 SMP Sat Mar 7 04:14:13 EST 2015 x86_64 x86_64 x86_64 GNU/Linux

 

Well, it needs bash 4 . date seems to work on the second server.

1 Like

Hi,

Sorry RudiC, I really need a ksh version instead :(-

I've managed to insert the missing date string using sed and the script 'almost' works like how I want it with just a 'little' bit of problem. I will explain further as we get along.

The script now looks like below:

 
 #!/bin/ksh
 dates="dates"
 date
echo ""
echo "- CONTENTS of $dates"
echo ""
cat ${dates}
echo ""
 curr_month=`date '+%m`
curr_day=`date '+%d`
curr_year=`date '+%Y`
 #curr_month=02
#curr_month=2016
#leap=${curr_year}; math=`echo "$leap%4" | bc`; [ ! -z $leap ] && [ $math -eq 0 ] && echo "$leap is a leap year!" || echo "$leap isn't a leap year";
leap=${curr_year}; math=`echo "$leap%4" | bc`; [ ! -z $leap ] && [ $math -eq 0 ] && leapyear="Y" || leapyear="N";
 case "${curr_month}" in
   01|03|05|07|08|10|12 )
      max_days="31"
      ;;
   04|06|09|11 )
      max_days="30"
      ;;
   02 )
      case "${leapyear}" in
         Y )
            max_days=29
            ;;
         N )
            max_days=28
            ;;
      esac
      ;;
esac
 echo
echo "curr_month = ${curr_month}"
echo "curr_day   = ${curr_day}"
echo "curr_year  = ${curr_year}"
echo "leapyear   = ${leapyear}"
echo "max_days   = ${max_days}"
echo
 check_day=${curr_day}
while [[ ${check_day} -le ${max_days} ]]
do
   next_date="${curr_year}-${curr_month}-${check_day}"
    if [[ -z "`grep ${next_date} ${dates}`" ]] ; then
      let prev_day=${check_day}-1
      prev_date="${curr_year}-${curr_month}-${prev_day}"
      echo "- Missing date ---> ${next_date} ... INSERT THIS BEFORE ${prev_date}"
       # --------------------------------------------------------------------
      # - Unfortunately, if the ${prev_date} does not exist
      #   then the sed below fails too.
      #   So ideally, ${prev_date} should be checking for the last date
      #   previous to the current date
      # --------------------------------------------------------------------
      sed "
/${prev_date}/ a\\
${next_date}
" ${dates} > /tmp/dates.tmp
      cp -p /tmp/dates.tmp ${dates}
   fi
    let check_day=${check_day}+1
done
 

The content of the dates file that I am checking is as below:

 
 # e.g. 2014-10-01
2015-05-01
2015-05-04
2015-05-05
2015-05-06
2015-05-07
2015-05-08
2015-05-12
2015-05-14
2015-05-15
2015-05-18
2015-05-27
2015-06-02
2015-06-03
2015-06-04
2015-06-05
2015-06-06
2015-06-07
2015-06-08
2015-06-09
2015-06-10
2015-06-11
2015-06-12
2015-06-13
2015-06-14
2015-06-15
2015-06-18
2015-06-22
2015-06-23
2015-06-24
2015-06-26
2015-06-30
2015-07-01
 

The portion of the script that does the insert is as below:

 
 while [[ ${check_day} -le ${max_days} ]]
do
   next_date="${curr_year}-${curr_month}-${check_day}"
    if [[ -z "`grep ${next_date} ${dates}`" ]] ; then
      let prev_day=${check_day}-1
      prev_date="${curr_year}-${curr_month}-${prev_day}"
      echo "- Missing date ---> ${next_date} ... INSERT THIS BEFORE ${prev_date}"
       # --------------------------------------------------------------------
      # - Unfortunately, if the ${prev_date} does not exist
      #   then the sed below fails too.
      #   So ideally, ${prev_date} should be checking for the last date
      #   previous to the current date
      # --------------------------------------------------------------------
    sed "
/${prev_date}/ a\\
${next_date}
" ${dates} > /tmp/dates.tmp
      cp -p /tmp/dates.tmp ${dates}
 fi
    let check_day=${check_day}+1
done
 

The problem is like this when I am running the script. Today is 2015-06-20 and 2015-06-20 is missing from the dates file.

Unfortunately, the sed portion that appends the missing date assumes that 2015-06-19 exists and unfortunately, that date string does not exist either, so the insert of 2015-06-20 fails.

So the 'logic' that I need to apply is if date-1 does not exist in the dates file, I need to check for the last known date string prior to the current date that I am checking and use that as the prev_date to insert the missing date from. In this case, prev_date should be 2015-06-18 instead.

Basically, I do not really what to insert all the missing date into the dates file, only the ones from current date when the script was run until the end day of the current month.

At the moment, I am writing that logic as deducting one day at a time, grepping the dates file until I find a previous date that match and then do the sed, a very tedious and long winding routine but works. Maybe someone know of a better way of doing this :o

Not sure if I've explained it clearly :o

---------- Post updated 06-20-15 at 02:45 AM ---------- Previous update was 06-19-15 at 01:23 PM ----------

Hi,

Now this is very, very, very embarrassing. After thinking about it, I don't actually have to check for previous date, all I have to actually do is append the missing dates into the dates file and then do a range sort of the file?

So, now my problem is how do I do a range sort of the file?

For example, for the file below, how do I sort the date string in it and keeping the comments in its place at the same time.

# Some comment
# More comment
# e.g. 2014-10-01
2015-05-01
2015-05-04
2015-05-05
2015-05-06
2015-05-07
2015-05-08
2015-05-12
2015-05-14
2015-05-15
2015-05-18
2015-05-27
2015-07-01
2015-06-20
2015-06-30
2015-06-21

The one below works but then I lose the comments? :frowning:

cat dates | grep -v "^#" | sort

Try:

{ grep ^# dates; grep -v ^# dates | sort;} > dates.sorted

or try:

 awk '/^#/; !/^#/{print | "sort"}' dates > dates.sorted