Hello,
I am basic level shell script developer. I have developed the following script. The shell script basically tracking various files containing certain strings. I am finding options to make the script run more faster. Any help/suggestion would be appreciated
#! /bin/bash
# Greps for all the patterns
# Trap Ctrl-C
trap Ctrl_C SIGINT
Ctrl_C() {
echo "CTRL-C CAUGHT"
read -p "Are you sure you want to quit? [y/n] " ans
if [[ "$ans" = "y" || "$ans" = "Y" ]]; then
echo "Quitting"
exit 1
fi
}
echo "Enter the ID:"
read Id
#touch $HOME/path/$Id.txt
result_log="$HOME/path/$Id.txt"
echo "================================================================================" >> "$result_log"
echo "Grep Starts"
echo "Greps Begins" >> "$result_log"
echo "========================================" >>"$result_log"
echo -n "X73" >> "$result_log"
echo >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
sudo grep -inr "\bx73\b" "<path>" >> "$result_log"
echo "========================================" >>"$result_log"
echo -n "Base64_decode" >> "$result_log"
echo >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
sudo grep -inr "base64_decode" "<path>" >> "$result_log"
echo >> "$result_log"
echo "========================================" >> "$result_log"
echo "GLOBALS" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
sudo grep -inr "GLOBALS" "<path>" >> "$result_log"
echo >> "$result_log"
echo "========================================" >>"$result_log"
echo "ASSERT" >>"$result_log"
echo -n "========================================" >>"$result_log"
echo >> "$result_log"
sudo grep -inr "assert" "<path>" >> "$result_log"
echo >>"$result_log"
echo "========================================" >>"$result_log"
echo "Eval" >>"$result_log"
echo -n "========================================" >>"$result_log"
echo >> "$result_log"
sudo grep -inrP "\beval\b" "<path>" >> "$result_log"
echo >> "$result_log"
echo "========================================" >> "$result_log"
echo "GZINFLATE" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
sudo grep -inrP "gzinflate" "<path>" >> "$result_log"
echo >> "$result_log"
echo "========================================" >> "$result_log"
echo "STRREV" >> "$result_log"
echo -n "========================================" >> "$grep_log"
echo >> "$result_log"
sudo grep -inrP "strrev" "<path>" >> "$result_log"
echo >> "$result_log"
echo "========================================" >> "$result_log"
echo "MOVE_UPLOAD_FILE" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
sudo grep -inr "move_uploaded_file" "<path>" >> "$result_log"
echo "========================================" >> "$result_log"
echo "SHELL_EXEC" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
sudo grep -inr "shell_exec" "<path>" >> "$result_log"
echo >> "$result_log"
echo "========================================" >> "$result_log"
echo "ROT13" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
sudo grep -inr "rot13" "<path>" >> "$result_log"
echo >> "$result_log"
echo "Common Malware keywords"
echo "==============Common Malware keywords=================" >> "$result_log"
echo "==============FilesMan=================" >> "$result_log"
sudo grep -inr "FilesMan" "<path>" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
echo "==============FilesGirl=================" >> "$result_log"
sudo grep -inr "FilesGirl" "<path>" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
echo "==============FSCOKOPEN=================" >> "$result_log"
sudo grep -inr "fsockopen" "<path>" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
echo "========================================" >> "$result_log"
echo "Error_Reporting" >> "$result_log"
echo -n "========================================" >> "$result_log"
echo >> "$result_log"
sudo grep -inr "error_reporting" "<path>" >> "$result_log"
echo >> "$result_log"
echo "shellscript.sh completed."
read -p "Do you wish to check the grep logs? [y/n] " ans
if [[ "$ans" = "y" || "$ans" = "Y" ]]; then
less $result_log | more
fi
echo "shellscript.sh logs: '$result_log'"
ha.. your script is big and not in readable format.. sorry for that...
can you post sample input contents and the logic you are using and the expected output
how many files inside your <path> ?
your grep is using -i -r ( time consuming based on the file size and number of files ).
you can write a small function and call the function for grep activity
#! /bin/bash
# Greps for all the patterns
# Trap Ctrl-C
trap Ctrl_C SIGINT
Ctrl_C() {
echo "CTRL-C CAUGHT"
read -p "Are you sure you want to quit? [y/n] " ans
if [[ "$ans" = "y" || "$ans" = "Y" ]]; then
echo "Quitting"
exit 1
fi
}
echo "Enter the ID:"
read Id
result_log="$HOME/path/$Id.txt"
path="/tmp/"
echo "================================================================================" >> "$result_log"
echo "Grep Starts"
echo "Greps Begins" >> "$result_log"
function GREP_LOGS(){
PATTERN="$1"
echo "========================================" >>"$result_log"
echo "${PATTERN}" >> "$result_log"
echo "========================================" >> "$result_log"
sudo grep -inr "${PATTERN}" "<path>" >> "$result_log"
}
GREP_LOGS "X73"
GREP_LOGS "base64_decode"
echo "shellscript.sh completed."
read -p "Do you wish to check the grep logs? [y/n] " ans
if [[ "$ans" = "y" || "$ans" = "Y" ]]; then
less $result_log | more
fi
echo "shellscript.sh logs: '$result_log'"
1 Like
Ok. As per requested here is the logic
#! /bin/bash
echo "Enter the ID:" // Say server ID e.g 123456
read Id
result_log="$HOME/path/$Id.txt" // Log file in which the result will be stored
sudo grep -inr "\bx73\b" "<path>" >> "$result_log" // Create a log file for that server ID and write the grep result one by one
sudo grep -inr "base64_decode" "<path>" >> "$result_log"
.
.
.
.
Grep will be run for various patterns and the output of the grep will be appended in the result_log file one by one
echo "shellscript.sh completed."
read -p "Do you wish to check the grep logs? [y/n] " ans
if [[ "$ans" = "y" || "$ans" = "Y" ]]; then
less $result_log | more
fi
echo "shellscript.sh logs: '$result_log'"
======================================================
Ouput will be like
========================================
X73
========================================
/path/filename.php:20: return str_replace(array('x73/x74', '_'), array('+', '/'), $s));
/path/admin.php:15: $existing_notice = x73/x74
========================================
Base64_decode
========================================
/path/filename.php:420: return base64_decode(str_replace(array('-', '_'), array('+', '/'), $s));
/path/admin.php:15: $existing_notice = base64_decode( $existing_notice );^M
.
.
result of every pattern
The problem with this script is it takes lot of time if the server conatins lots of files.
---------- Post updated at 04:38 AM ---------- Previous update was at 04:32 AM ----------
Thanks for guidance.. I will surely try it out.
itkamaraj:
ha.. your script is big and not in readable format.. sorry for that...
can you post sample input contents and the logic you are using and the expected output
how many files inside your <path> ?
your grep is using -i -r ( time consuming based on the file size and number of files ).
you can write a small function and call the function for grep activity
#! /bin/bash
# Greps for all the patterns
# Trap Ctrl-C
trap Ctrl_C SIGINT
Ctrl_C() {
echo "CTRL-C CAUGHT"
read -p "Are you sure you want to quit? [y/n] " ans
if [[ "$ans" = "y" || "$ans" = "Y" ]]; then
echo "Quitting"
exit 1
fi
}
echo "Enter the ID:"
read Id
result_log="$HOME/path/$Id.txt"
path="/tmp/"
echo "================================================================================" >> "$result_log"
echo "Grep Starts"
echo "Greps Begins" >> "$result_log"
function GREP_LOGS(){
PATTERN="$1"
echo "========================================" >>"$result_log"
echo "${PATTERN}" >> "$result_log"
echo "========================================" >> "$result_log"
sudo grep -inr "${PATTERN}" "<path>" >> "$result_log"
}
GREP_LOGS "X73"
GREP_LOGS "base64_decode"
echo "shellscript.sh completed."
read -p "Do you wish to check the grep logs? [y/n] " ans
if [[ "$ans" = "y" || "$ans" = "Y" ]]; then
less $result_log | more
fi
echo "shellscript.sh logs: '$result_log'"
Thanks
RudiC
January 5, 2017, 8:04am
4
Every single redirection opens, reads / writes, and closes the target file. So, for every loop of your script you're handling roughly 80 file operations (times 2, for open and close). Try redirecting the entire script's output, e.g. like
./script > result.log
.grep s per loop, each causing a process creation with its resource consumption, - try running it once with either an alternation search pattern ( "PAT1|PAT2|PAT3" ) or a pattern file ( -f option), which would simplify some later treatment, and deal with your pretty print later with e.g. a single sort and / or a single awk command.
The following boosts your function
GREP_LOGS(){
echo "========================================"
PATTERN="" pattern=""
for pat
do
PATTERN="$PATTERN -e '$pat'"
pattern="$pattern '$pat'"
done
echo "$pattern"
echo "========================================"
sudo grep -inr ${PATTERN} "<path>"
}
# search all in one stroke
GREP_LOGS "X73" "base64_decode" >> "$result_log"
echo "shellscript.sh completed."
rovf
January 6, 2017, 3:42am
6
@MadeInGermany : You forgot the $1 and $2 in your GREP_LOGS (i.e. you pass the parameters, but you don't fetch them). Please update your posting accordingly.
for pat cycles through $1 $2 ...for pat in "$@"
1 Like
