Shell script to execute Oracle procedure and trigerring email on success and failure

Here is the shell script which need to trigger a stored procedure and when the record count is 0 then it should capture in the log that 0 duplicate records are found and if it's more than 0 then log with no of records. Also if there is any sqlerror then it should write the same in the log file and it should send the communication in all scenarios. We have funct file which has declared with all directories and we are referring here for the log file path directory. But when i tried to execute it getting this error message not found [No such file or directory] - ./local/dir1/funct.sh . Also can you help me out in the code logic as well?

PS - I have all the privileges to this file

#!/bin/ksh

    ./local/dir1/funct.sh

    sqlplus -s "${ORAUSER}"/"${ORAPASSWD}"@"${ORASRVC}" <<EOF > "$LOG_TEXT"

    set head off

    set serveroutput on

    SELECT TO_CHAR(SYSDATE,'MMDDRRRRHH24MISS') FROM DUAL;

    EOF   

TIME_STAMP=$(cat "${LOG_TEXT}")

LOG_FILE_NAME='FileStatus'${TIME_STAMP}'.log'

LOG_FILE=${LOGFILEDIR}'/FileStatus/'${LOG_FILE_NAME}

write_log "Database Timestamp is $TIME_STAMP"

write_log "Executing FileStatus now..."

chmod 777 "${LOG_FILE}"

sqlplus -s "${ORAUSER}"/"${ORAPASSWD}"@"${ORASRVC}" <<EOF >> "${LOG_TEXT}"

set feedback off

set heading off

DECLARE

v_count NUMBER;

BEGIN

select count(*) INTO v_count from (select col1,col2,count(col3) from Tab1 group by col1,col2 having count(col3)>1);

whenever sqlerror exit -1

whenever oserror exit -1

execute FileStatus;
END;
EOF
if [[ ${v_count} -eq 0 ]]; then

        write_log "FileStatus executed successfully."

    write_log "No Duplicate records Found: $LOG_FILE"

         mailx -s "No Duplicate records Found" email< "${success}"

elif [[ ${v_count} -gt 0 ]];
then
        write_log "FileStatus  executed successfully."
    write_log "Number of duplicate records found::$v_count:$LOG_FILE"
        mailx -s "Number of duplicate records found:$v_count" email < "${success}""
else
        Write_log "FileStatus Failure."
         mailx -s "FileStatus" email < "${failure}""
fi 
cleanup
exit 0;

Hello senmng, and welcome.

Firstly I would suggest that you move the credentials to within the here document like this:

:
:
sqlplus -sn  <<EOF > "$LOG_TEXT"
connect "${ORAUSER}"/"${ORAPASSWD}"@"${ORASRVC}"
set head off
:
:

The way you have it exposes the credentials to anyone who manages to run ps whilst the sqlplus command is running each time.

You can even neaten this further by avoiding the output file:

:
:
TIME_STAMP="$(sqlplus -sn <<EOF                                       # Start the sub-process with "$(
   connect "${ORAUSER}/${ORAPASSWD}@${ORASVC}"                        # Just double quote the whole thing here rather than each little bit
   set head off
   set serveroutput on
   SELECT TO_CHAR(SYSDATE,'MMDDRRRRHH24MISS') FROM DUAL;
EOF)"                                                                 # End of the sub-process with the )"

LOG_FILE_NAME="FileStatus${TIME_STAMP}.log"                           # Just double quote the whole 

LOG_FILE="${LOGFILEDIR}/FileStatus/${LOG_FILE_NAME}"                  # ... and again

## Or perhaps go straight to LOG_FILE="${LOGFILEDIR}/FileStatus/FileStatus${TIME_STAMP}.log"

write_log "Database Timestamp is ${TIME_STAMP}"                       # Braces added to be consistent

write_log "Executing FileStatus now..."

chmod 777 "${LOG_FILE}"
:
:

As to your actual question over the 'not found' response (for which I've wrapped it in ICODE tags in your post) this is probably because the file is relative to your current working directory. If you doa pwd then think about where that file is relative to where you are thne it might become clearer. Can you set an absolute path (i.e. starts with a leading slash for the root directory) and try again? Does this get you anywhere further?

I hope tha these suggestions help,
Robin

I tried with the same code by giving the exact path for funct.sh. Now that error not found [No such file or directory got rectified but it's not connecting with Oracle and not executing the proc. Moreover am getting email message that No Duplicate records Found: though the result of this query fetches 800 records.

select count(*) INTO v_count from (select col1,col2,count(col3) from Tab1 group by col1,col2 having count(col3)>1);

Also I tried with other version of code with your suggested changes and got the error message like this

Usage: CONN[ECT] [{logon|/|proxy} [AS {SYSDBA|SYSOPER|SYSASM}] [edition=value]]
where <logon> ::= <username>[/<password>][@<connect_identifier>]
      <proxy> ::= <proxyuser>[<username>][/<password>][@<connect_identifier>]
SP2-0157: unable to CONNECT to ORACLE after 3 attempts, exiting SQL*Plus'.log: cannot create [No such file or directory]

Also , i'd like to have an accurate mechanism to store the return value of procedure to the shell script. Here my requirement is to execute a stored procedure from this shell script once done i need to trigger the emails based on their success with the record counts and if not return with sql error if any.Could you please help me on this?

Please can you post the code you ran to generate the error. It might be that the comments get in the way, so perhaps strip them out and try again too.

An alternate might be to return a complex string listing the return code and data values, then afterwards your script can separate out which means which. I do this for commands driven remotely by ssh connections. It's not pretty, but it works quite reliably.

Regards,
Robin

Thanks for your reply Robin. Here is the modified code. Also could you assist me in how to store the return value of procedure to the shell script.Here the variable V_COUNT value is not returning to the shell script. Could you please give an example on this?

#!/bin/ksh
 ./local/dir1/funct.sh
TIME_STAMP="$(sqlplus -sn <<EOF                                      
   connect "${ORAUSER}/${ORAPASSWD}@${ORASVC}"                        
   set head off
   set serveroutput on
   SELECT TO_CHAR(SYSDATE,'MMDDRRRRHH24MISS') FROM DUAL;
EOF)"

LOG_FILE_NAME="FileStatus${TIME_STAMP}.log"

LOG_FILE="${LOGFILEDIR}/FileStatus/${LOG_FILE_NAME}"  

write_log "Database Timestamp is $TIME_STAMP"

write_log "Executing FileStatus now..."

chmod 777 "${LOG_FILE}"

ssqlplus -sn  <<EOF > "$LOG_TEXT"
connect "${ORAUSER}"/"${ORAPASSWD}"@"${ORASRVC}"
set head off

DECLARE

v_count NUMBER;

BEGIN

select count(*) INTO v_count from (select col1,col2,count(col3) from Tab1 group by col1,col2 having count(col3)>1);

whenever sqlerror exit -1

whenever oserror exit -1

execute FileStatus;
END;
EOF
if [[ ${v_count} -eq 0 ]]; then

        write_log "FileStatus executed successfully."

    write_log "No Duplicate records Found: $LOG_FILE"

         mailx -s "No Duplicate records Found" email< "${success}"

elif [[ ${v_count} -gt 0 ]];
then
        write_log "FileStatus  executed successfully."
    write_log "Number of duplicate records found::$v_count:$LOG_FILE"
        mailx -s "Number of duplicate records found:$v_count" email < "${success}""
else
        Write_log "FileStatus Failure."
         mailx -s "FileStatus" email < "${failure}""
fi 
cleanup
exit 0;

Hello senmng,

  • Your first active line calls another script. Is that meant to load functions in for you? I'd suggest preceding it with the word source else any variables set and the functions defined will be lost when the script exits.
  • Line 18 exectuing chmod 777 "${LOG_FILE}" looks a little dubious. Would you really want anybody with access to the OS to be able to read your log (maybe), edit your log (not very likely) and potentially execute your logfile (almost certainly not)?
  • Does line 20 have a spelling mistake with ssqlplus?
  • Is line 36 calling a stored procedure? What's inside that? Does it end with a zero return code?
  • How do you know that there has been a failure, or is that just zero rows counted?

Outside of the embedded SQL, you refer to ${v_count} which only seems to be set within the embedded SQL, i.e. will only exist when the SQL process is running. It may be as simple as exiting from the SQL using that.

Have you considered finishing the SQL part with exit &v_count perhaps? It might be just as simple as that and you shell script will get the $? as the value of &v_count unless you abort early and return the value -1 , of course.

Kind regards,
Robin

Hi - Thanks for your reply.

  1. - Yes, that issue has been fixed and it's now reading that file which has username,password and parameters
  2. - Not necessarily but we need the full access for the log file
  3. - In the actual script, i have the correct word but while copying this error had occurred.
  4. - No,it's not returning any code,simply merging statement.
  5. - The actual value of that query is 890(i ran that query alone in the DB)

Yes i tired with that option also it's returning 0.Here i removed the exec proc since i'd like to check the output of the query alone.

ssqlplus -sn  <<EOF > "$LOG_TEXT"
connect "${ORAUSER}"/"${ORAPASSWD}"@"${ORASRVC}"
set head off
DECLARE
v_count NUMBER;
BEGIN
select count(*) INTO v_count from (select col1,col2,count(col3) from Tab1 group by col1,col2 having count(col3)>1);
END;
EOF
exit  &v_count;
echo$?

Since you are using Oracle PL/SQL, it has DBMS Output availability to write to logs and UTL_SMTP to send mails.
Do everything in PL/SQL procedure and make your life easier.

At least, do not use $? , this is an abuse of exit call and will not give results you might think it will.
Valid range is from 0 to 255, with 0 meaning success, everything else (some) type of error.

So using exit code variable filled with count from rows is just wrong, with all those pl/sql database utilities at your disposal.

Hope that helps
Regards
Peasant.

What's the other option to return the value to the shell script and use the same in the IF condition?

Can you please show some examples using UTL_SMTP and UTL_mail to send emails on the success?

Your exit &v_count statement needs to at least be run before the EOF closes the here document else this is left to the shell. It will run the shell exit in the background and then try to run the command v_count, which probably won't be found. It will then confuse you.

I agree with Peasant that this probably isn't the way to go for non-trivial counts. Some shells might let you get away with it, but best to avoid it. Perhaps an SQL statement like prompt "v_count=&v_count" ; might work, but I haven't got a server I can test this statement on at all, so it will probably have an error in it, however if you can generate an obviously tagged output that tell you the value you need to pass, then perhaps you can get at that in the shell script part.

Maybe someone else can jump in here to suggest a clean way to pass the value out. I've got a few options but I'd expect better ones. Given you are writing all the output to a log file, you might get away with:-

:
:
EOF

v_count_line="$(grep 'v_count='  "${LOG_TEXT}")"            # Grab the crafted line from the output file

v_count="${v_count_line#*=}"          # Set the value in the shell script to be the part of the line after the equals sign by trimming off everything in front (including the equals sign)

printf "The duplicate lines count is %s\n" "${v_count}"
:
:

Like I say, there's probably a better way.

Hi - Apologize for the delayed response. Here is the code which i tired where it passes the value to the shell script(in echo command) but not able to get the variable value in the IF condition. Am getting this output even the query output is 0.But at same time i tried with query with more than 0 records and am getting the output as expected.Since the value has leading spaces the IF condition is not matching with the values. Also i tried with cat test.out| sed -e 's/^[ \t]*//' IF condition works if the value is 0 but not for value>0 it returns with error No such file or dir Can you help me to get the proper trimming command for the values 0 or more than 0(max 4 digits)

status proc  executed successfully

Num of Duplicate records Found:          0

where the actual output should be

status proc  executed successfully

No Duplicate records Found:          0
#!/bin/ksh  
outvar=0  
sqlplus -S /nolog <<EOF>test.out  
   "${ORAUSER}"/"${ORAPASSWD}"@"${ORASRVC}"
   set echo off termout off feedback off  
   set pagesize 0  
   set trimspool on  
   select to_char(count(*)) from (select col1,col2,count(col3) from Tab1 group by col1,col2 having count(col3)>1)  
 EOF  
outvar=`cat test.out`  
echo " status proc  executed successfully"  
if [[ $outvar = 0 ]]; then  
   echo "No Duplicate records Found: $outvar"  
else  
   echo "Num of Duplicate records Found: $outvar"  
fi 

The outvar=`cat test.out` is not great from a few things:-

  • The backticks ` are deprecated and can make complex statements difficult to read. You would be better to wrap such code with $( and )
  • The cat command is a waste of a process. You can just do this - outvar=$(<test.out)

The content of outvar will contain the entire file, so leading spaces, any headers (I see you've turned them off) and any end of line/end of file marker so the numerical comparison is a bit awkward. Your test in [[ & ]] uses the = operator, but this is a string comparison, and I'm not sure if this structure is supported in true ksh either. This is more of a bash syntax. ksh might also accept it, but I think it is only looking for file information instead. Your test (assuming that the file always just contains a numeric value) could probably be written as:

outvar=$(<test.out)
if [ $outvar -eq 0 ]
then
   echo "No Duplicate records Found: $outvar"  
else  
   echo "Num of Duplicate records Found: $outvar"  
fi

If it gives a bit of trouble, it is likely that the other characters read in from the file are the problem. You might try adding a line after the read such as typeset -i outvar=$outvar to see if that strips all the other things away and just assigns it the integer value. There are various typeset options for true ksh that have not transferred into bash or other pseudo-ksh so I'm hoping you have a true ksh. I can't test the syntax properly because I've not got a true ksh available so this bit is from memory of about 5 years ago.

Do either of these suggestions help?

Robin

1 Like

Awesome! It's working as expected now by assigning the variable value in this approach.

outvar=$(<test.out)
if [ $outvar -eq 0 ]

I'm gald this got you moving again. You should really test this hard too and work out how to handle errors in creating the file that might mean you:-

  • have multiple lines
  • have multiple values on a single (or multiple) line(s)
  • have non-numerics
  • an empty file

.....just to be careful.

Kind regards,
Robin

Sure Robin. In this case, we are handling only 0 and 3 digit numeric values . But in other cases like you mentioned (multiple lines,empty files) can we use sed(for displaying nth line) and grep(for finding the particular text )?

Yes, the value of outvar will just hold all the content of your file. You might prefer to process the variable or create another by reading the file in a different way. It's up to you, but you can use many ways to read specific things. For example:-

  • To read the 3rd line, this would do sed -n3p This could be written as third_line="$(printf "%s" "${outvar}" | sed -n3p)" or third_line="$(sed -n3p filename)" depending
  • To set a value if your file is null, you could [ ! -s filename ] && variable=0
  • To set a value if your variable is null, you could [ -z "${outvar}" ] && outvar=0
  • To scan for non-numerics and set the value to zero, you could attack it in several ways, e.g. delete the digits from the string and see if there's anything left; try to add zero and check the return code (writing all output/error messages to /dev/null to keep it clean) etc. etc. It depends what makes sense to you.

Does this help you sanity check your data?

Robin

1 Like

Dear Robin-That will really help me to do the sanity check.