flags to avoid restart

hello all,

i have the below script that we used to backup our DB using oracle's utility called RMAN. This has been working fine, but the issue is when the backup fails and we re-start it, it backups the whole thing again. Example.

lets say i have 5 database on my system(db1,db2,db3,db4,db5) and we use below script which loops for a file called /etc/oratab which has my DB names in it. If the script completes successfully we have no issue. But lets say db1 and db2 backup was successful and db3 backup fails and db4 and db5 backup was good. NOW LETS SAY i want restart by backup as DB3 backup failed...it is going to backup db1, db2, db3,db4,db5 all of them all again and that i was i want to avoid...the script needs to put some flag or something were it knows only db3 has failed and that is the only one that needs to be backed up...

how can i accomplish that? below is the script....

umask o+w
LogMsg()
{
echo "`date '+%Y%m%d.%H%M%S'` :: $*" >> $RMAN_LOG_FILE
}

##########################################################################
# Calls LogMsg and exits with failure
##########################################################################
LogAndFail()
{
LogMsg "$*"
exit 1
}

TODAY=`date '+%C%y_%m_%d_%H_%M'`; export TODAY
export ORATAB=/etc/oratab

###START OF LOOP
for db in `egrep -i ":NBU" $ORATAB | grep -v "^#" | grep -v "\*" | cut -d":" -f1 | sort | uniq`
do
  echo "processing database: $db"
  export ORACLE_SID=$db

  export RMAN_LOG_FILE=/oracle/sqlutils/logs/RMAN_bkp_${ORACLE_SID}_${TODAY}.log

  export ORACLE_HOME=`grep -i $ORACLE_SID: ${ORATAB}|grep -v "^#" | cut -f2 -d:`
  export PATH=$ORACLE_HOME/bin:$PATH

$ORACLE_HOME/bin/rman <<EOF >> $LOG
connect catalog rman/password@reocverycatalog
connect target /
run
{
allocate channel t1 type disk;
allocate channel t2 type disk;
backup database plus archivelog;
RELEASE CHANNEL t1;
allocate channel for maintenance type disk;
show all;
report obsolete;
delete noprompt expired backup;
delete noprompt obsolete;
crosscheck backup;
crosscheck archivelog all;
release channel;
}
EOF
exit

 RSTAT=$?

# ---------------------------------------------------------------------------
# Log the completion of this script.
# ---------------------------------------------------------------------------

  if [ "$RSTAT" = "0" ]
  then
    LOGMSG="Backup successfully $ORACLE_SID"
    echo $LOGMSG|mailx -s "BACKUP Status ..." abc@company.com
  else
    LOGMSG="Backup error $ORACLE_SID"
    echo $LOGMSG|mailx -s "BACKUP Status ..." abc@company.com
  fi

  echo >> $RMAN_LOG_FILE
  echo Script $0 >> $RMAN_LOG_FILE
  echo ==== $LOGMSG on `date` ==== >> $RMAN_LOG_FILE
  echo >> $RMAN_LOG_FILE

######   end of loop #######
done

exit $RSTAT

I'm missing something here. You have:

 ... ... ...
export ORATAB=/etc/oratab

###START OF LOOP
for db in `egrep -i $ORATAB | grep -v "^#" | grep -v "\*" | cut -d":" -f1 | sort | uniq`
do
 ... ... ...
done

which means that you are asking egrep to look for the string "/etc/oratab" (matching uppercase or lowercase letters) from lines it reads from standard input. What input are you feeding into this script?

If instead you are grepping /etc/oratab and just didn't show us the ERE your looking for in /etc/oratab, please tell us what the ERE is and show us the contents of /etc/oratab.

yes you are totaly right, I was editing part of the script...and i missed out something...so the egrep is like below...its looking for a word called NBU at end of each line...if there is one then it obviously greps it...i will edit the original post as well...thanks...

egrep -i ":NBU" $ORATAB | grep -v "^#" | grep -v "\*" | cut -d":" -f1 | sort | uniq

---------- Post updated at 01:57 PM ---------- Previous update was at 01:53 PM ----------

here is output of /etc/oratab

test1:/oracle/ORAHOME/112_64:N:NBU
ptst:/oracle/ORAHOME/REP:N:NBU
alpd:/oracle/alpd/REP:Y
dsd:/oracle/ORAHOME/112_64:Y:NBU
yyd:/oracle/ORAHOME/112_64:Y:NBU
u1u:/oracle/ORAHOME/u1u:Y:NBU

To keep the forums high quality for all users, please take the time to format your posts correctly.

First of all, use Code Tags when you post any code or data samples so others can easily read your code. You can easily do this by highlighting your code and then clicking on the # in the editing menu. (You can also type code tags

```text
 and 
```

by hand.)

Second, avoid adding color or different fonts and font size to your posts. Selective use of color to highlight a single word or phrase can be useful at times, but using color, in general, makes the forums harder to read, especially bright colors like red.

Third, be careful when you cut-and-paste, edit any odd characters and make sure all links are working property.

Thank You.

The UNIX and Linux Forums

Any ideas or thought on how i can achive this ?

You need to keep track of successfully completed backups in a file when they finish. Then, in your main loop, you need to check to see if you are about to start a backup that has already been completed, and, if so, skip it.

I think I can provide changes to your script that will do that for you, but I have other things I need to do first. I expect to be able to complete something for you before dinner time tonight (US, Pacific time zone).

That would be awesome, i am going to try what you said as well and see what i come up with...

OK. So, this message is suggesting updates to the code provided in the first message i this thread. The basic idea is to create a log file (Completed.YYYYMMDD) that contains a list of databases successfully backed up on the date specified by the filename extension and skip any database backup if that log file indicates that the database has already been backed up.
I also modified the commands to process $ORATAB to make it a little more efficient (calling awk once and sort once instead of calling grep three times, sort once, and uniq once).

Now for the changes. First, change:

export ORATAB=/etc/oratab

###START OF LOOP
for db in `egrep -i ":NBU" $ORATAB | grep -v "^#" | grep -v "\*" | cut -d":" -f1 | sort | uniq`
do
  echo "processing database: $db"

to:

export ORATAB=/etc/oratab

### Look for a list of databases successfully backed up today.  Create if it doesn't exist.
comp=$(date "+Completed.%Y%m%d")
if [ ! -f $comp ]
then    touch $comp
fi

###START OF LOOP
for db in $(awk -F: '$0 ~ /:[Nn][Bb][Uu]/ && $1 !~ /^#/ && $0 !~ /[*]/ {print $1}' $ORATAB | sort -u)
do
  echo "processing database: $db"
   if [ $(grep -c -e "^$db\$" $comp) -gt 0 ]
   then   printf "Skipping %s: It has already been backed up.\n" "$db"
          continue
   fi

and then change:

# ---------------------------------------------------------------------------
# Log the completion of this script.
# ---------------------------------------------------------------------------

  if [ "$RSTAT" = "0" ]
  then
    LOGMSG="Backup successfully $ORACLE_SID"
    echo $LOGMSG|mailx -s "BACKUP Status ..." abc@company.com
  else

to:

# ---------------------------------------------------------------------------
# Log the completion of this script.
# ---------------------------------------------------------------------------

  if [ "$RSTAT" = "0" ]
  then
    LOGMSG="Backup successfully $ORACLE_SID"
    echo $LOGMSG|mailx -s "BACKUP Status ..." abc@company.com
    echo $db >> $comp
  else

Crazy_max, how does this compare to what you came up with?

Thanks Don.

I was trying for something too, but it did not work out...i did what you mentioned created touch entry for keeping track of successful DB(/tmp/checkifdbbackwassuccesful. and was trying something in term of this in my main for loop...

for db in `egrep -i ":NBU" $ORATAB | grep -v "^#" | grep -v "\*" | cut -d":" -f1 | sort | uniq`
do

########################################################
# Check to make sure previous backup was successful    #             
# if it was not, only backup that specific DB          # 
########################################################

if [ -f /tmp/checkifdbbackwassuccesful ]; then
   for success_db in `cat /oracle/tmp/testsuccess|grep $db`; do
if test $db = $success_db; then
 :
else
RUN MY BACKUP CMD
fi

but obviously that did not work.... and will still working on it

THANK YOU For you reply on this...couple of questions i have for you....

how would we deal with something like...let say my backup started at around 11pm today(it will touch/create a file called Completed.20120926) and my backup ended tomorrow around 2am....now that it is a different day and if i try to restart my backup job(if it failed) wouldn't it look for filed called Completed.20120927(sept 27/next day) and will obviously not find it and process all the DB again ?

Yes. I was assuming that you'd start your backups a little after midnight and have all day to retry the backups if the log shows that one or more of the backups failed.

You could also make the name of your successful backup log file be an operand to your shell script and use $1 instead of setting $comp using the current date. (If you do this, you'll need to verify that an operand is present when you start the script. Or, you could use $1 to initialize the value of $comp instead of using the date command only when an operand is present. The concept still works, you just need to choose a filename for your log file that works for your situation.)

Thanks Don, i am not able to understand what exactly do you mean when you say i can make the name of my successful backup log file be an operand to my shell script? Do you have an example of it for my situation ?

---------- Post updated at 10:22 AM ---------- Previous update was at 08:29 AM ----------

to avoid the day/date confusion...i modified it to be something like below...just creating a simple file

export comp=/oracle/completed_backup
if [ ! -f /oracle/completed_backup ]
then touch $comp
fi

then at the end i check to make sure ## of count i want backed up and # of count db backup was successful..if it match remove
file otherwise do nothing(so if restart the backup it know which one to backup)

######   end of loop #######
done

## Check to make sure oratab entry and sucessful backup match
## if it dose then remove the file that captures success entry($comp)

actual_db=`egrep -i ":NBU" $ORATAB | grep -v "^#" | grep -v "\*" | cut -d":" -f1 | sort | uniq|wc -l`
success_db=`cat /oracle/sqlutils/completed_backup|wc -l`

if [ $actual_db = $success_db ]; then
    rm /oracle/sqlutils/completed_backup
	else
	:  #DO NOTHING if value dose not match
	fi

exit $RSTAT

THANK YOU FOR YOUR HELP

I was suggesting that you could change:

comp=$(date "+Completed.%Y%m%d")

in the first set of changes suggested in message sequence #8 in this thread to:

comp=${1:-$(date "+Completed.%Y%m%d")}

Then it will behave exactly as it did before if you invoke your backup shell script with no arguments. I don't know the name of your script, but for this discussion, suppose that it is called backupdbs. If you issued the command:

backupdbs

late last night, it would have created Completed.20120926 as the list of successfully completed backups. If you looked at the log files this morning and found that one or more backups failed, you could use the command:

backupdbs Completed.20120926

to use the list created last night instead of creating a new file for today (Completed.20120927).

In case someone is reading this doesn't understand the command:

comp=${1:-$(date "+Completed.%Y%m%d")}

It sets the the variable comp to $1 (the 1st argument to yoiur shell script) if that argument is set and is not an empty string; otherwise, it expands $(date "+Completed.%Y%m%d") and sets comp to that value (i.e., Completed.YYYYMMDD where YYYY is the current year, MM is the current two-digit month, and DD is the current two-digit day within the current month).

If you don't like my proposed naming convention for the list of successfully completed backups, change $(date "+Completed.%Y%m%d") to whatever you want. Or just make up a new name every day and always invoke your shell script with that name as its operand. (Just be sure that the name you choose doesn't contain any whitespace characters nor any characters that would be treated specially by your shell such as *, ?, $, etc.)