Script to Tar file in a specific Directory

Shell Programming and Scripting
1

I'm trying to write a Unix script that will go to a specific directory (/tmp/Sanbox/logs) and tar.gz all the log files in that directory and delete the original files that are older than 2 days. So far I have this but it doesn't work. Any help would be appreciated.

 
 #!/bin/bash

AGE_TO_COMPRESS="172800" # 172800 seconds = 2 days
LOG_FILES="export/home/H0166015/Sandbox/logs"

# Any file older than EDGE_DATE must be compressed
NOW=$( date +%s )
EDGE_DATE=$(( NOW - AGE_TO_COMPRESS ))

for file in $LOG_FILES ; do
    # check if file exists
    if [ -e "$file" ] ; then

        # compare "modified date" of file to EDGE_DATE
        if [ $( stat -c %Y "$file" ) -gt ${EDGE_DATE} ] ; then

            # create tar file of a single file
            tar -cvzf $file.tar.gz $file --remove-files
        fi
    fi
done
2

Welcome to the forum.

You'll concede that "but it doesn't work" doesn't really help people help you. WHAT "doesn't work", and/or how? Please give people something to work upon.

What I can see from staring at your code ist that older-than-two-days-files' modification time should be LESS than the threshold value you constructed from NOW. Which by itself is a running point in time. Having different results running the script at different times of day may not be what you really need.

3

Another possible issue is with:

for file in $LOG_FILES ; do

This will not iterate through all the files/directories under /export/home/H0166015/Sandbox/logs as the script seems to expect.

Can I suggest as a starting point:

 #!/bin/bash

AGE_TO_COMPRESS="172800" # 172800 seconds = 2 days
LOG_FILES="/export/home/H0166015/Sandbox/logs"

# Any file older than EDGE_DATE must be compressed
NOW=$( date +%s )
EDGE_DATE=$(( NOW - AGE_TO_COMPRESS ))

for file in ${LOG_FILES}/* ; do
    # check if file exists
    if [ -f "$file" ] ; then

        # compare "modified date" of file to EDGE_DATE
        if [ $( stat -c %Y "$file" ) -gt ${EDGE_DATE} ] ; then

            # create tar file of a single file
            tar -cvzf "${file}.tar.gz" "$file" --remove-files
        fi
    fi
done
4

Thank you Chubler_XL works great. Sorry for being so vague

The previous code will tar up the files in "logs" directory but what if I want to tar up the files in multiple directories. I tried putting it in an array but it doesn't want to increment to the next element in the array and I don't know why???

Here's my code:

 
 #!/bin/bash
 AGE_TO_COMPRESS="172800" # 172800 seconds = 2 days
LOG_FILES=( "/Sandbox/logs1" "/Sandbox/logs2" "/Sandbox/logs3" "/Sandbox/logs4" )
 # Any file older than EDGE_DATE must be compressed
NOW=$( date +%s )
EDGE_DATE=$(( NOW - AGE_TO_COMPRESS ))

 for file in ${LOG_FILES[$file]}/* ; do
    # check if file exists
    if [ -f "$file" ] ; then
         # compare "modified date" of file to EDGE_DATE
        if [ $( stat -c %Y "$file" ) -gt ${EDGE_DATE} ] ; then
             # create tar file of a single file
            tar -cvzf "${file}.tar.gz" "$file"
        fi
    fi
done
5

You don't need an array here.

LOG_FILES="/Sandbox/logs1/* /Sandbox/logs2/* /Sandbox/logs3/* /Sandbox/logs4/*"
for file in $LOG_FILES
do
    ...

Or go for one more loop.

LOG_DIRS="/Sandbox/logs1 /Sandbox/logs2 /Sandbox/logs3 /Sandbox/logs4"
for log_dir in $LOG_DIRS
do
  for file in "$log_dir"/*
  do
     ...
2 Likes
6

I tried you code, "MadeInGermany" but it doesn't work.

Here's the working code. The @ symbol means to go through all the element in the array.

 
 #!/bin/bash
 LOG_FILES=(
        "/Sandbox/logs1"  "/Sandbox/logs2" "/Sandbox/logs3" "/Sandbox/logs4"
 
)
 for file in ${LOG_FILES[@]}; do
   gzip "${file}"/*;
done
7

In what way does it not work? I would say that the suggestions (two of them) were good.

Please post exactly what you have run and the output when you run it with the trace option.

I'm not sure that the @ quite means it that way, rather that it expands the array variable to list every element and therefore the loop has a long list to work through. I would expect both options from MadeInGermany to do the same.

Robin

8

Your last sample can be written as

#!/bin/bash 
LOG_DIRS="/Sandbox/logs1 /Sandbox/logs2 /Sandbox/logs3 /Sandbox/logs4"
for log_dir in $LOG_DIRS
do
  for file in "$log_dir"/*
  do
    # ensure it is a file and not yet gzipped
    if [ -f "$file" ] && [[ $file != *.gz ]]
    then
      gzip "$file"
    fi
  done
done