Create file every year and log data to it

Shell Programming and Scripting
1

Hi there,
Here is what I currently have and trying to come up with a logic to automatically append logs to a respective year file. The log files might get generated everyday or any other day, irrespective of which I want to copy all of the logs for that year into one file /tmp/<year>.log and then copy the logs to new location /tmp/<newyear>.log when a newyear comes and so on.
Thanks for looking.

#!/bin/ksh
cat /tmp/*.log.old >> /tmp/2016.log

rm /tmp/*.log.old > /dev/null 2>&1

ls /tmp/*.log > /tmp/system_logs

for i in `cat /tmp/system_logs`
do
  echo "New log file moving to old is $i \n"
  cp $i $i.old
  if [ $? -eq 0 ]
  then
     rm $i
  fi
done
2

Would something like this get you in the direction?

year=$(date "+%Y")
for f in /tmp/*.log; do
  cat "$f" >> /tmp/${year}.log
  rm "$f"
done
3

Aia - A belated thanks as I was away for sometime and didn't had a chance to look at the code, I'll post back if there are any issues.

4 
$ ./test.sh
++ date +%Y
+ year=2016
+ for f in '/tmp/*.log'
+ cat /tmp/2016.log
cat: /tmp/2016.log: input file is output file
+ rm /tmp/2016.log
+ for f in '/tmp/*.log'
+ cat /tmp/aaa.log
+ rm /tmp/aaa.log
+ for f in '/tmp/*.log'
+ cat /tmp/bbb.log
+ rm /tmp/bbb.log
+ for f in '/tmp/*.log'
+ cat /tmp/ccc.log
+ rm /tmp/ccc.log

This doesn't seem to work as expected, it is removing the dated file 2016.log in this case after giving message "input file is output file'whereas I'm looking to append to it. I'm trying to figure out how to append to multiple dated log files for eg...all aaa.log files should append to one aaa.2016.log, bbb.log files in bbb.2016.log file and so on.

---------- Post updated at 11:30 PM ---------- Previous update was at 06:42 PM ----------

#!/bin/ksh
YEAR=$(date "+%Y")
cd /tmp
if [ ! -d "${YEAR}_LOGS" ]; then
  echo " ${YEAR}_LOGS doesn't exist. Creating dir..." > /dev/null 2>&1
  mkdir /tmp/${YEAR}_LOGS > /dev/null 2>&1
else
  echo " ${YEAR}_LOGS directory exists" > /dev/null 2>&1
fi

for i in $(ls /tmp/ |grep old | awk -F "." '/aaa|bbb|cccc|dddd/ {print $1}'); do
  cat /tmp/${i}.log.old >> /tmp/${YEAR}_LOGS/${i}_${YEAR}.log
  rm /tmp/${i}.log.old > /dev/null 2>&1
done

for i in /tmp/*.log; do
  echo "Log being moved to old is $i ... \n"
  cp $i $i.old
  if [ $? -eq 0 ]; then
     rm $i
  fi
done

This is what I came up with and wondering if this could be made simpler or enhanced further, pls advise.

5

Although untested, this would seem to be a more efficient version of your latest script, producing the same outputs:

#!/bin/ksh
YEAR=$(date "+%Y")
cd /tmp

mkdir -p "${YEAR}_LOGS"

for i in aaa bbb cccc dddd
do
  if [ -f "$i.log.old" ]
  then
    cat "$i.log.old" >> "${YEAR}_LOGS/${i}_$YEAR.log" && rm -f "$i.log.old"
  fi
done

for i in *.log
do
  printf 'Log being moved to old is %s ... \n\n' "$PWD/$i"
  mv "$i" "$i.old"
done

Note, however, that, by definition, /tmp is a directory intended to hold temporary files. You haven't said what operating system you're using, but on many systems, ALL files under /tmp are removed every time the system is rebooted. So, attempting to archive data from multiple years in files under /tmp is extremely dangerous. I strongly suggest that you choose a different directory as the base of your active and archived log files.

2 Likes