Need to select files larger than 500Mb from servers

donpasscal · May 29, 2018, 11:09am

I need help modifying these two scripts to do the following:

print files in (MB) instead of (KB)
only select files larger than 500MB -> these will be mailed out daily
Select all files regardless of size all in (MB) -> these will be mailed out once a week

this is what i have so far and it is now working well...help will be appreciated.

[set v_alert_command_db002 "; echo '            Alert Log (12cLNX): /oracle/UB1/saptrace/diag/rdbms/test/TEST/trace/alert_TEST.log - ' | tr '\\n' ' '; du -sk '/oracle/TEST/saptrace/diag/rdbms/test/TEST/trace/alert_TEST.log' | awk '{ print \$1 }' | tr '\\n' ' '; echo '(KB)' "]

[# add listener command for db003 server]
 [set v_listener_db003 "; echo '            Listener Log' | tr '\\n' ':'; echo '' | tr '\\n' ' ' ; lsnrctl status LISTENER_TEST01 | grep Log| awk '{ print \$4 }' | sed 's/alert\\/log.xml/trace\\/listener_test01.log/g' | tr '\\n' ' '; echo '-' | tr '\\n' ' ' ; du -sk `lsnrctl status LISTENER_TEST01 | grep Log | awk '{ print \$4 }' | sed 's/alert\\/log.xml/trace\\/listener_test01.log/g'` | awk '{ print \$1 }' | tr '\\n' ' ' ;  echo '(KB)'"]

apmcd47 · May 30, 2018, 4:08am

Could you tell us what operating system you are using? The version of shell (bash3, bash4, ksh88, ksh93) would be useful too.

For instance, if I knew you were using a version of Linux I could tell you that you could replace

du -k

with

du -m

but if you are using Solaris that won't work. You could use wc -c to get the size of the file in bytes and then divide by 1048576 to convert to megabytes.

So in each case you want to suppress the output if the size of the file in question is smaller than 500Mb?

Andrew

donpasscal · May 30, 2018, 9:02am

shell- #!/bin/ksh
operating system - 5.11 11.3 sun4v sparc sun4v
I would like the format for getting the size the same as in the example because thats how mgmt wants it and its part of an expect script. Yes i will like to suppress the output if the size of the file in question is smaller than 500Mb and -> these will be mailed out daily and again to:

Select all files regardless of size all in (MB) -> these will be mailed out once a week
thanks for your input so far

apmcd47 · May 31, 2018, 4:49am

Do you have gdu on your system? It is the GNU version of du . If so, try this:

[set v_alert_command_db002 "; file='/oracle/TEST/saptrace/diag/rdbms/test/TEST/trace/alert_TEST.log'; size_M=$(gdu -m $file | nawk '{ print \$1 }'); (( size_M > 500 )) && printf \"            Alert Log (12cLNX): %s - %s (MB)\n\" $file $size_M"]
[set v_listener_db003 "; file=$(lsnrctl status LISTENER_TEST01 | nawk '/Log/ { print $4 }' | sed 's^alert/log.xml^trace/listener_test01.log/' ); size_M=$(gdu -m $file | nawk '{ print \$1 }'); (( size_M > 500 )) && printf \"            Listener Log: %s - %d (Mb)\n\" ${file} ${size_M}"]

You may have to add backslashes to some of the backslashes and other special characters. I don't know the tool you are using there so don't know the rules for escaping strings.

I've removed several pointless tr s and grep s and replaced the echo s with printf s. Do you really need that

sed 's^alert/log.xml^trace/listener_test01.log/'

I don't think you do.

I haven't tested this as I cannot.

If you don't have gdu try replacing

size_M=$(gdu -m $file | nawk '{ print \$1 }')

with

size_M=$(( $(wc -c < ${file}) / 1048576 ))

Andrew

donpasscal · May 31, 2018, 11:58am

[I've removed several pointless trs and greps and replaced the echos with printfs. Do you really need that!]

Thank you for the help i will test what you have provided and update the thread. I guess i do need them (trs) to stay in place.... i am just trying to mod the script that my lead gave me and he would not like that many changes made.

---------- Post updated at 10:58 AM ---------- Previous update was at 10:02 AM ----------

@apmcd47 ...Please see if you can help me figure this out....please.....!!!
This expect script is suppose to connect to SERVER DB01 that hosts two databases and pull the alert and listener log files sizes. The problem i am having is that the correct alert log/size file is pulled for both databases but the wrong listener log/size is pulled for testdb2...because in the output (below) the sid is set to testdb1 instead of testdb2. I need help in directing the script ot set the correct sid and to pull from the correct path for testdb2. I will also like the output to be in MB because i only want to print out the log files larger that 500MB and mail them out every day , then every friday i mail out the complete output.


[# command for gathering hostname
set v_host_command "echo hostname | tr '\\n' ':' ;hostname"]
[# Exception hostname
set v_host_null "echo''"]

[# command for gathering Listener Log
set v_listener_command "; echo ' Listener Log' | tr '\\n' ':'; echo '' | tr '\\n' ' ' ; lsnrctl status | grep Log | awk '{ print \$4 }' | sed 's/alert\\/log.xml/trace\\/listener.log/g' | tr '\\n' ' '; echo '-' | tr '\\n' ' ' ; du -sk `lsnrctl status | grep Log | awk '{ print \$4 }' | sed 's/alert\\/log.xml/trace\\/listener.log/g'` | awk '{ print \$1 }' | tr '\\n' ' ' ; echo '(KB)'"]


# add env setting command for db01 server TEST DB
[#set v_command_db01_testdb1 "export PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/home/users/oraclet/bin:/usr/bin/X11:/sbin:/usr/ccs/bin:/bin:/usr/local/bin:.:/oraclet/120/bin:/oraclet/120;export ORACLE_HOME=/oraclet/120;export ORACLE_SID=TESTDB1;"]

[# add env setting command for db01 server Catalog
set v_command_db01_testdb2 "export PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/home/users/oraclet/bin:/usr/bin/X11:/sbin:/usr/ccs/bin:/bin:/usr/local/bin:.:/oraclet/120/bin:/oraclet/120;export ORACLE_HOME=/oraclet/120;export ORACLE_SID=TESTDB2;"]

[# add env listener command for db01 server TEST DB
set v_listener_db01_1 "; echo ' Listener Log' | tr '\\n' ':'; echo '' | tr '\\n' ' ' ; echo '/oraclet/diag/tnslsnr/db01/listener/trace/listener.log' | tr '\\n' ' '; echo '-' | tr '\\n' ' ' ; du -sk /oraclet/diag/tnslsnr/db01/listener/trace/listener.log | awk '{print \$1 }' | tr '\\n' ' ' ; echo '(KB)'"]

[# add listener command for db01 server RMAN Catalog
set v_listener_db01_testdb1 "; echo ' Listener Log' | tr '\\n' ':'; echo '' | tr '\\n' ' ' ; echo '/oraclet/diag/tnslsnr/db01/listener_testdb2/trace/listener_TESTDB2.log' | tr '\\n' ' '; echo '-' | tr '\\n' ' ' ; du -sk /oraclet/diag/tnslsnr/db01/listener_testdb2/trace/listener_TESTDB2.log | awk '{print \$1 }' | tr '\\n' ' ' ; echo '(KB)'"]

[# now connect to remote UNIX box
} elseif {$v_server == $v_host_db01 && $v_user == $v_user_oraclet} {
spawn ssh $v_user@$v_server $v_command_db01_testdb1$v_host_command$v_exc_alert_command$v_listener_db01_1$v_end_command
match_max 100000]
[} elseif {$v_server == $v_host_db01 && $v_user == $v_user_oraclet} {
spawn ssh $v_user@$v_server $v_command_db01_testdb2$v_host_command$v_exc_alert_command$v_listener_db01_b$v_end_command
match_max 100000]

[# Look for passwod prompt
expect "*?assword:*"]

[# Send password
send "$v_password\r"]

[# send blank line (\r) to make sure we get back to gui
send "\r"
expect eof]


[running ssh oraclem@db01...
spawn ssh oraclem@db01 export PATH=/oraclet/120/bin:/usr/bin:/usr/ccs/bin:/usr/vac/bin:/usr/sbin:/sbin:/usr/bin/X11:/bin:/usr/local/bin:/usr/lbin:/usr/ucb:PATH:.;export ORACLE_HOME=/oraclet/120;export ORACLE_SID=TESTDB1;echo hostname | tr '\n' ':' ;hostname; echo ' Alert Log' | tr '\n' ':'; echo '' | tr '\n' ' ' ;echo /oraclet/diag/rdbms/testdb1/TESTDB1/trace/alert_TESTDB1.log | tr '\n' ' '; echo '-' | tr '\n' ' ' ; du -sk /oraclet/diag/rdbms/testdb1/TESTDB1/trace/alert_TESTDB1.log | awk '{ print $1 }'| tr '\n' ' ' ; echo '(KB)'; echo ' Listener Log' | tr '\n' ':'; echo '' | tr '\n' ' ' ; lsnrctl status | grep Log | awk '{ print $4 }' | sed 's/alert\/log.xml/trace\/listener.log/g' | tr '\n' ' '; echo '-' | tr '\n' ' ' ; du -sk `lsnrctl status | grep Log | awk '{ print $4 }' | sed 's/alert\/log.xml/trace\/listener.log/g'` | awk '{ print $1 }' | tr '\n' ' ' ; echo '(KB)';echo] [----------------------------------------------------------------------------------------------------------------------------------]
[Password:
hostname:db01
Alert Log: /oraclet/diag/rdbms/TESTDB1/TESTDB1/trace/alert_TESTDB1.log - 3 (KB)
Listener Log: /oraclet/diag/tnslsnr/db01/listener/trace/listener.log - 154255 (KB)]
[----------------------------------------------------------------------------------------------------------------------------------]
[running ssh oraclem@db01...(in this output its setting and connecting the the wrong sid ...testdb1...needs to be connecting to testdb2)
spawn ssh oraclem@db01 export PATH=/oraclet/120/bin:/usr/bin:/usr/ccs/bin:/usr/vac/bin:/usr/sbin:/sbin:/usr/bin/X11:/bin:/usr/local/bin:/usr/lbin:/usr/ucb:PATH:.;export ORACLE_HOME=/oraclet/120;export ORACLE_SID=TESTDB1;echo hostname | tr '\n' ':' ;hostname; echo ' Alert Log' | tr '\n' ':'; echo '' | tr '\n' ' ' ;echo /oraclet/diag/rdbms/testdb2/TESTDB2/trace/alert_TESTDB2.log | tr '\n' ' '; echo '-' | tr '\n' ' ' ; du -sk /oraclet/diag/rdbms/testdb2/TESTDB2/trace/alert_TESTDB2.log | awk '{ print $1 }'| tr '\n' ' ' ; echo '(KB)'; echo ' Listener Log' | tr '\n' ':'; echo '' | tr '\n' ' ' ; lsnrctl status | grep Log | awk '{ print $4 }' | sed 's/alert\/log.xml/trace\/listener.log/g' | tr '\n' ' '; echo '-' | tr '\n' ' ' ; du -sk `lsnrctl status | grep Log | awk '{ print $4 }' | sed 's/alert\/log.xml/trace\/listener.log/g'` | awk '{ print $1 }' | tr '\n' ' ' ; echo '(KB)';echo]
[----------------------------------------------------------------------------------------------------------------------------------]
[Password:
hostname:db01
Alert Log: /oraclet/diag/rdbms/testdb2/TESTDB2/trace/alert_TESTDB2.log - 770 (KB)
Listener Log: /oraclet/diag/tnslsnr/db01/listener/trace/listener.log - 154255 (KB)]

[Listener Log: /oraclet/diag/tnslsnr/db01/listener_testdb2/trace/listener_TESTDB2.log - 154255 (KB)]

apmcd47 · June 1, 2018, 4:53am

Actually that question was aimed at the sed command, not the use of tr to suppress newline generation.

I'm sorry, I know nothing about expect and don't really know enough about Oracle to help you. My advice at this time would be to log onto the server and figure out how to get this to work interactively and then put what you have learnt into the expect script. If they won't let you do this, because it's a production server, then ask to use the development server.

I gave you two possible ways of getting the file size in MB in my last post; can you use either of them in your script?

And please, when posting code, use the CODE tags. It makes it easier to understand the code.

Andrew