Hi Folks,
I've put together a script for sorting my backup files into sub folders to be run from a cron job. Each file is named username.tar.gz and the file /etc/trueuserowners contains all users and their owner in the format "user: owner".
The script works fine identifying users and their owners and is as follows:
#!/bin/bash
_w="/path/to/backups"
_wb="$_w/*.tar.gz"
if [ -d "$_w" ]; then
for f in $_wb
do
_f=$(basename $f)
_user=${_f%%.*}
_o=`grep "^$_user:" /etc/trueuserowners`
_owner=${_o#* }
if [ ! -d "$_w/$_owner" ]; then
`mkdir $_w/$_owner/`
fi
`mv $f $_w/$_owner/`
done
fi
Say I have the following:
/path/to/backups/userabackup.tar.gz
usera is owned by resellera.
The first time the script runs, the script will create the following directory and move userabackup.tar.gz into it:
/path/to/backups/resellera
The script has an if in it to ensure that /path/to.backups/resellera is only created if it does not already exist as you can see. My problem is that the mv command appears to be backing up the directory and creating a new one instead of moving the file into the existing directory and replacing the file if it is already there.
For example:
The second time the script runs, /path/to/backups/resellera will be renamed to /path/to/backups/resellera.1 and a new /path/to/backups/resellera is created.
The third time, /path/to.backups/resellera.1 is renamed /path/to.backups/resellera.2, /path/to.backups/resellera to /path/to.backups/resellera.1 and a new directory created.
I shouldn't need to have the script remove the directory first and the mv command does not create backup directories like this when run from the console. The VERSION_CONTROL environment is not set.
Is there something specific that makes mv act like this when run from a cron job? Do I have to tell it to over-write instead of create these backups?
That seems to have done the trick, no more backed up directories. Cheers
It is an envirmont variable that can be set instead of the --backup option for mv. Documented in man mv, I only mentioned it so that noone suggested it as a possible explanation.
I already did, I was referring to mv "backing up" the current destination directory as dirname.1 instead of simply overwriting any files in the existing directory.
Actually I did make a slight mistake, the numbering of the backups starts at 0, not 1
Apparently this didn't fix it. For reference I have "alias mv='mv -i'" in my .bashrc so I've change the mv to mv -f and am hoping that'll sort it out tonight.
Please post the current version of the script and mention what Operating System and what Shell you have.
We would expect changes to these original lines because of out-of-context backticks:
There are ambiguty issues with $_w and $_wb . Try using "${_w}" and "${_wb}" .
Do any of your filenames or directory names contain space characters or characters special to Shell (e.g. colon)? If so, the "for" statement needs to be a "while" statement.
Btw. If there are no subdirectories under /path/to/backups the script can be much improved.
This is the full script. I have made an interesting observation however. I have a number of resellers with differename usernames. For some reason, the problem does not affect the folder for the reseller root. The folders don't have anything to do with usernames other than being named after a user - there's no permissions differences across any of the files yet for some reason the one called root somehow works.
#!/bin/bash
_d="/home/scadmin/cpbackup/daily"
_db="$_d/*.tar.gz"
for f in $_db
do
_f=$(basename $f)
_user=${_f%%.*}
_o=`grep "^$_user:" /etc/trueuserowners`
_owner=${_o#* }
if [ ! -d "$_d/$_owner" ]; then
mkdir $_d/$_owner/
fi
mv -f $f $_d/$_owner/
done
# uname -a
Linux fred.southportcomputers.co.uk 2.6.18-238.12.1.el5 #1 SMP Tue May 31 13:22:04 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
Running Centos 5.6 x86_64.
The script runs from the root crontab and root's shell is /bin/bash.
There are no spaces or special characters in the path, only lowercase letters.
I'll try it with curly brackets as you suggest, the only bit I haven't got the curlys in it the for statement itself.
#!/bin/bash
_d="/home/backup/daily"
_db="$_d/*.tar.gz"
for f in $_db
do
_f=$(basename $f)
_user=${_f%%.*}
_o=`grep "^$_user:" /etc/trueuserowners`
_owner=${_o#* }
if [ ! -d "${_d}/${_owner}" ]; then
mkdir ${_d}/${_owner}/
fi
mv -f $f ${_d}/${_owner}/
done
With set -x and "echo" you can see very easy what's wrong with your script.
#!/bin/bash
set -x
d="/home/scadmin/cpbackup/daily"
db="$d/*.tar.gz"
for f in $db
do
f=$(basename $f)
user=${f%%.*}
o=`grep "^$user:" /etc/trueuserowners`
owner=${o#* }
if [ ! -d "$d/$owner" ]; then
echo mkdir $d/$owner/
fi
echo mv -f $f $d/$owner/
done
You don't need at all all this "_". You don't export these variables, so they can't interfere with anything. The shell is noisy enough to add more.
Well the script ran wrong again last night regardless of the changes. Removing the _ also broke the mv statement as $_f and $f were two different values. What I can see is that the if statement checks for the directory without a trailing slash whilst the mkdir statement runs with a trailing slash. I've added the trailing slash to see if that makes a different.
Its annoying that the script works fine when you run it from the console but screws up from cron overnight.
This is still being a pig. It works perfectly fine when run from the command line. With set -x on I get the output emailed over and when run from cron it does the check to see if the directory exists and then goes right ahead and creates the directory regardless.
Next try, removing the double quotes from the if statement..
