How to convert my /bin/sh script with cgi and html to run it on browser!??

Hello,

I want to run this script on my CentOS 6 via browser :
________________________________________________________________________________________________

#!/bin/sh
echo Username?
read MY_NAME
echo Provisional file name?
read MY_FILE
echo File NAME you want to save?
read MY_FILE2
cat /opt/SUNWappserver/nodeagents/ins1/logs/* | grep $MY_NAME > $MY_FILE.txt
sed 's/INFO.*;|//g' $MY_FILE.txt > $MY_FILE2.txt

________________________________________________________________________________________________

For that I have created 2 files : /var/www/html/test.html and / var/www/cgi-bin/script.sh :

  1. I have created one file test.html
    ________________________________________________________________________________________________
<HTML>
<TITLE>LOGS</TITLE>
<BODY>
<b>LOGs Instacne</b><p>
<form action="cgi-bin/script.sh" method="post">
<br>
<input type="submit" value="LOG Button">
</form>
</BODY>
</HTML>
  1. And /var/ww/cgi-bin/script.sh
#!/bin/sh -f
echo "Content-type: text/html"
echo ""
echo '<html>'
echo '<head>'
echo '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">'
echo '<title>Inscance LOGs</title>'

#Step 1
echo "<form method=GET action=\"${SCRIPT}\">"\
     '<table nowrap>'\
         '<tr><td>UserName</TD><TD><input type="text" name="val_x" size=12></td></tr>'\
         '</tr></table>'
#Step 2
echo "<form method=GET action=\"${SCRIPT}\">"\
        '<table nowrap>'\
                 '<tr><td>File Name not cleaned?</TD><TD><input type="text" name="val_y" size=12></td></tr>'\
                '</tr></table>'
#Step 3
echo "<form method=GET action=\"${SCRIPT}\">"\
        '<table nowrap>'\
                 '<tr><td>File NAME cleaned</TD><TD><input type="text" name="val_a" size=12></td></tr>'\
                '</tr></table>'

echo '<br><input type="submit" value="Search">'\
       '<input type="reset" value="Reset"></form>'

  if [ -z "$QUERY_STRING" ]; then
        exit 0
  else
     XX=`echo "$QUERY_STRING" | sed -n 's/^.*val_x=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`
     YY=`echo "$QUERY_STRING" | sed -n 's/^.*val_y=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`
     AA=`echo "$QUERY_STRING" | sed -n 's/^.*val_a=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`
     echo "UserName: " $XX
     echo '<br>'

     echo "File Not_Cleaned: " $YY
     echo '<br>'

     echo "File Cleaned: " $AA
     echo '<br>'


echo "cat /opt/SUNWappserver/nodeagents/ins1/logs/*" | grep $XX > $YY.txt

          browser="`echo $XX | cut -d\  -f1`"
          echo '<br>'
          echo "Browser: " $browser

BB="`echo $browser | sed 's/INFO.*;|//g' $YY.txt > $AA.txt`"
     echo '<br>'
     echo '<br>'
     echo "Result: " $BB
     echo '<br>'
fi
echo '</body>'
echo '</html>'
exit 0

________________________________________________________________________________________________

When I open in browser: localhost/test.html click on button "LOG Button" it redirects me on localhost/cgi-bin/script.sh .
I fill in the required fields and click on the search button to show the data, but the cat dhe sed command is not executed ( echo "cat /opt/SUNWappserver/nodeagents/ins1/logs/*" | grep $XX > $YY.txt) and (sed 's/INFO.*;|//g' $YY.txt > $AA.txt ).

Can you give me any suggestions to execute the script shell above in the browser and if i have chosen the wrong way how can i execute this script?

Regards,
Juta2020

One important thing to keep in mind with CGI. It doesn't run as you, it runs as the web server's user. Which means a CGI script doesn't -- AND SHOULDN'T! -- have permissions to write to the current folder, whatever it happens to be. You should create a file in a known writable location, like /tmp/$$-query.txt or some such. $$ is the script's PID, which is a quick and dirty way of getting a unique file name. (Once you have stuff working, see mktemp for a fancier, safer option.) And yes, you'll need to delete the file when you're done!

What's 'echo cat' for? If you want to cat, run cat. But you don't actually need cat here, grep (and most other commands) is fully capable of reading files without cat's help.

Also, you should be putting variables inside double quotes, and not putting * inside double quotes.

# Catch error messages so you see them in the web browser
exec 2>&1
# Check log files for 
grep "$XX" /opt/SUNWappserver/nodeagents/ins1/logs/* > /tmp/$$.txt 

...

rm /tmp/$$.txt

Lastly, does your web server have read-access to /opt/SUNWappserver/nodeagents/ins1/logs/* ? It'll need it.

Also, a quick and simple way to handle QUERY_STRING in bash is:

OLDIFS="$IFS"
IFS="&=" # Split strings upon ampersand and equals
# If query_string="a=b&c=d, this sets $1=a, $2=b, $3=c, $4=d
set -- ${QUERY_STRING}
IFS="$OLDIFS" # Set splitting back to normal

while [ "$#" -gt 0 ]
do
        case "$1" in
        val_x) XX="$2" ; shift ;;
        val_y) YY="$2" ; shift ;;
        val_z) ZZ="$2" ; shift ;;
        *) ;; # Ignore unknown option
        esac
        shift # Throw away $1, move $2 etc down
done

Faster than running sed three times.

There's lots of stuff that's not being handled by using bash, like %20 encoding for spaces, etc.

Hello Corona688.

I have added both your suggestion on my file script.sh:

________________________________________________________________________________________________________

#!/bin/sh -f
echo "Content-type: text/html"
echo ""
echo '<html>'
echo '<head>'
echo '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">'
echo '<title>Inscance LOGs</title>'

#Step 1
echo "<form method=GET action=\"${SCRIPT}\">"\
     '<table nowrap>'\
         '<tr><td>UserName: </TD><TD><input type="text" name="val_x" size=12></td></tr>'\
         '</tr></table>'
#Step 2
echo "<form method=GET action=\"${SCRIPT}\">"\
        '<table nowrap>'\
                 '<tr><td>File Name not cleaned: </TD><TD><input type="text" name="val_y" size=12></td></tr>'\
                '</tr></table>'
#Step 3
echo "<form method=GET action=\"${SCRIPT}\">"\
        '<table nowrap>'\
                 '<tr><td>File NAME cleaned: </TD><TD><input type="text" name="val_a" size=12></td></tr>'\
                '</tr></table>'

echo '<br><input type="submit" value="Search">'\
       '<input type="reset" value="Reset"></form>'

  if [ -z "$QUERY_STRING" ]; then
        exit 0
  else
     XX=`echo "$QUERY_STRING" | sed -n 's/^.*val_x=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`
     YY=`echo "$QUERY_STRING" | sed -n 's/^.*val_y=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`
     AA=`echo "$QUERY_STRING" | sed -n 's/^.*val_a=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`
     echo "UserName: " $XX
     echo '<br>'

     echo "File Not_Cleaned: " $YY
     echo '<br>'

     echo "File Cleaned: " $AA
     echo '<br>'

# Catch error messages so you see them in the web browser
exec 2>&1
# Check log files for
grep "$XX" /var/www/html/** > /tmp/$$.txt

rm /tmp/$$.txt


OLDIFS="$IFS"
IFS="&=" # Split strings upon ampersand and equals
# If query_string="a=b&c=d, this sets $1=a, $2=b, $3=c, $4=d
set -- ${QUERY_STRING}
IFS="$OLDIFS" # Set splitting back to normal

while [ "$#" -gt 0 ]
do
        case "$1" in
        val_x) XX="$2" ; shift ;;
        val_y) YY="$2" ; shift ;;
        val_z) ZZ="$2" ; shift ;;
        *) ;; # Ignore unknown option
        esac
        shift # Throw away $1, move $2 etc down
done

fi
echo '</body>'
echo '</html>'
exit 0

________________________________________________________________________________________________________

But, on browser I receive this error message:

UserNaname: juta
File Name not cleaned:  juta_not_clean  	
File NAME cleaned: juta_ok	

UserName: juta
File Not_Cleaned: juta_not_clean
File Cleaned: juta_ok
grep: /var/www/html/**: No such file or directory

How can I solve this ???
So , With my script I want through web browser "to find files that contain a text string" for example on /var/www/html/* filling in the fields: username, File Name not cleaned, File NAME cleaned to take results and save results in file.txt after that do this command:

sed 's/INFO.*;|//g' $YY.txt > $AA.txt

, in this case $YY.txt it is file.txt.
I want to use sed to clean to clear unnecessary information through the rows that will appear, a command that works fine when running it through the terminal in the script below:

#!/bin/sh
echo Username?
read MY_NAME
echo Provisional file name?
read MY_FILE
echo File NAME you want to save?
read MY_FILE2
cat /opt/SUNWappserver/nodeagents/ins1/logs/* | grep $MY_NAME > $MY_FILE.txt
sed 's/INFO.*;|//g' $MY_FILE.txt > $MY_FILE2.txt

This sed command is important for me!

So, can you help me with any script to run through the web browser and perform the above functions?

Best Regards,
Juta2020

You missed the point. set and case is a replacement for XX=`echo "$QUERY_STRING" | sed -n 's/^.*val_x=\([^&]*\).*$/\1/p' | sed "s/%20/ /g"`

As for "no such file or directory", I'll hazard a wild guess and say there's no such file or directory. You want one *, not two.

1 Like

Ok. Thank you Corona688! BUt I have anothere question. After I will grep and save file grep -r "$XX" /var/www/html/ > /tmp/$YY.txt how can I sed it with command :

sed 's/INFO.*;|//g' $YY.txt > $ZZ.txt            

??

Best Regards,
Jurgen

The same as before, except you must write files inside /tmp/ instead of the current directory.

on my script.sh after line:

 grep -r "$XX" /var/www/html/ > /tmp/$YY.txt

this line:

sed 's/INFO.*;|//g' /tmp/$YY.txt > /tmp/$ZZ.txt

but it just create file /tmp/$YY.txt , it does not perform the sed command to clear the rows in
/tmp/$YY.txt file in a new /tmp/$ZZ.txt file. I don't know what I did wrong! Please help me to resolve this issue.

Regards
Juta2020

I have no way of knowing if your sed expression will do what you want or not! Show the input you have and the output you want.

You also haven't told me what $XX, $YY, or $ZZ are supposed to be, and what the output from your CGI script was, if anything.

Also, show your script itself. I have no way of knowing if it's correct either without actually seeing it.

Hello again Corona688. Please take a look to my script.sh:

#!/bin/sh -f
echo "Content-type: text/html"
echo ""
echo '<html>'
echo '<head>'
echo '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">'
echo '<title>Instance LOG's</title>'

#Step 1
echo "<form method=GET action=\"${SCRIPT}\">"\
     '<table nowrap>'\
         '<tr><td>UserName</TD><TD><input type="text" name="val_x" size=12></td></tr>'\
         '</tr></table>'
#Step 2
echo "<form method=GET action=\"${SCRIPT}\">"\
        '<table nowrap>'\
                '<tr><td>FileName not_cleaned</TD><TD><input type="text" name="val_y" size=12></td></tr>'\
                '</tr></table>'
#Step 3
echo "<form method=GET action=\"${SCRIPT}\">"\
        '<table nowrap>'\
                 '<tr><td>FileName cleaned</TD><TD><input type="text" name="val_z" size=12></td></tr>'\
                '</tr></table>'

echo '<br><input type="submit" value="Search">'\
       '<input type="reset" value="Reset"></form>'


OLDIFS="$IFS"
IFS="&=" # Split strings upon ampersand and equals

# If query_string="a=b&c=d, this sets $1=a, $2=b, $3=c, $4=d

set -- ${QUERY_STRING}
IFS="$OLDIFS" # Set splitting back to normal

while [ "$#" -gt 0 ]
do
        case "$1" in
        val_x) XX="$2" ; shift ;;
        val_y) YY="$2" ; shift ;;
        val_z) ZZ="$2" ; shift ;;
        *) ;; # Ignore unknown option
        esac
        shift # Throw away $1, move $2 etc down
done

# Catch error messages so you see them in the web browser

exec 2>&1

# Check log files for

grep -r "$XX" /opt/SUNWappserver/nodeagents/ins1/logs/ > /tmp/$YY.txt

sed 's/INFO.*;|//g' /tmp/$YY.txt > /tmp/$ZZ.txt

sendmail -t my_mail@mydomain.com -s "LOG's instance for $XX" -a=/tmp/$ZZ.txt

rm /tmp/$ZZ.txt

echo '</body>'
echo '</html>'
exit 0

I have

chown apache:apache /opt/SUNWappserver/nodeagents/ins1/logs/

, grep command works perfectly, but this command desn't work :

sed -n 's/INFO.*;|//g' /tmp/$YY.txt > /tmp/$ZZ.txt

to clear in each line INFO.*; . One example of my logs in /tmp/$YY.txt (FileName not_cleaned):

/opt/SUNWappserver/nodeagents/ins1/logs/server.log_2018-10-05T09-05-18:[#|2018-10-05T06:22:37.640+0200|INFO|sun-appserver2.1|web.test.bean.alert.NominalsMyThread|_ThreadID=1610;_ThreadName=Thread-33935;|My Search: Username -> Test, Test, 123456789|#]

I want to delete the bold ones in rows, every line that starts with : /opt and end with [# ,also each line thats starts with INFO and end with ; (second one works with sed -n 's/INFO.*;|//g' /tmp/$YY.txt > /tmp/$ZZ.txt if I run via command line but on my script.sh desn't work). What I want to

What I want to get as a result in /tmp/$ZZ.txt (FileName cleaned) is:

|2018-10-05T06:22:37.640+0200||My Search: Username -> Test, Test, 123456789|#]

So, deleted :

/opt/SUNWappserver/nodeagents/ins1/logs/server.log_2018-10-05T09-05-18:[#

and

INFO|sun-appserver2.1|web.test.bean.alert.NominalsMyThread|_ThreadID=1610;_ThreadName=Thread-33935;

After that I want to attach /tmp/$ZZ.txt and sent into my e-mail via sendmail (

sendmail -t my_mail@mydomain.com -s "LOG's instance for $XX" -a=/tmp/$ZZ.txt

desn't work )and and after send it, remove /tmp/$ZZ.txt from server.

Regards,
Juta2020

Couple of things.

How do you know that /tmp/$ZZ.txt isn't being created? You're deleting it!

If you're not getting error messages in your browser, sed is running. So I'd check that $ZZ is what you actually think it is. Just a line somewhere that does

echo "XX=$XX YY=$YY ZZ=$ZZ" so you can see it in the browser and rule out the obvious.

That's a really bad idea. You need to put back whatever permissions you had before. If you don't know them, restore from backup.

It's a bad idea for two reasons. #1, it gives the web server permission to change the files! And the web server could be being used by anyone!
#2, it only works right now but will break whenever new log files are created. New log files won't have the right permissions.

The proper way to do it depends on what's creating those files, and what permissions they were in the first place. You might have been able to do it by adding apache to some group or other and restarting apache, but without knowing it's impossible to say.

I've added down the line:

sed -n 's/INFO.*;|//g' /tmp/$YY.txt > /tmp/$ZZ.txt

this row :

echo "XX=$XX YY=$YY ZZ=$ZZ"

to see it in the browser, but nothing appears. No clean file is created ($ ZZ.txt), only un cleaned file is created ($ YY.txt). To test it I have comented the line: rm /tmp/$ZZ.txt . I haven't receive any e-mail on my_mail@mydomain.com!

Please take a look :

grep -r "$XX" /opt/SUNWappserver/nodeagents/ins1/logs/ > /tmp/$YY.txt
sed 's/INFO.*;|//g' /tmp/$YY.txt > /tmp/$ZZ.txt
sendmail -t my_mail@mydomain.com -s "LOG's instance for $XX" -a=/tmp/$ZZ.txt
echo "XX=$XX YY=$YY ZZ=$ZZ"

#rm /tmp/$ZZ.txt

Regards,
Juta2020

That you see nothing at all in the browser, not even errors hints that your program isn't running -- you're only seeing files left by previous attempts.

To be clear: We cannot pull a rabbit out of our hat and give you perfect code. You've given us too many unknowns and change everything we give you far too much. What I am trying to do is show you how to troubleshoot. The basics are:

  1. SIMPLIFY.
    [list]
  2. Strip the webpage out of your script to remove that complication.
  3. If that doesn't help, remove EVERYTHING until it starts working. Pare it down to 'hello world'. Then add things back, one by one, until something breaks.
    [/list]
  4. IGNORE ASSUMPTIONS.
    [list]
  5. You think grep is working. Do you know that for a fact? You're using the same name every time, it could have created that file last week.
    [/list]
  6. TEST.
    [list]
  7. The error could be in something 90 lines up. You're getting NO OUTPUT when you should be getting something. That hints the program is breaking early.
    [/list]

Zero feedback tells you nothing. Try this:

#!/bin/bash

# Make sure errors get shown in web browser.  DO THIS FIRST!
exec 2>&1

# Make sure web browser shows everything raw
echo "Content-Type:  text/plain"
echo

echo "Text output test"
echo "Error output test" >&2
echo
echo "Query is: ${QUERY_STRING}"

IFS="&=" ; set -- ${QUERY_STRING}

while [ "$#" -gt 0 ]
do
        echo "Got variable $1=$2"

        case "$1" in
        val_x) XX="$2" ; shift ;;
        val_y) YY="$2" ; shift ;;
        val_z) ZZ="$2" ; shift ;;
        *) ;; # Ignore unknown option
        esac
        shift
done

echo "got XX=$XX YY=$YY ZZ=$ZZ"

grep "$XX" /var/www/html/* > /tmp/a.txt
echo "grep returned $?"
sed 's/INFO.*;|//g' /tmp/a.txt > /tmp/b.txt
echo "sed returned $?"

I tried it via browser, but desn't:

UserName: name
FileName not_cleaned: juta_not_clean
FileName cleaned: juta

Text output test Error output test Query is: val_x=name&val_y=juta_notclean&val_z=juta Got variable val_x=name Got variable val_y=juta_notclean Got variable val_z=juta got XX=name YY=juta_notclean ZZ=juta

It's create file /tmp/juta_notclean.txt but don't create /tmp/juta.txt !!

My script.sh is like:

#!/bin/sh -f
echo "Content-type: text/html"
echo ""
echo '<html>'
echo '<head>'
echo '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">'
echo '<title>Instance LOG's</title>'

#Step 1
echo "<form method=GET action=\"${SCRIPT}\">"\
     '<table nowrap>'\
         '<tr><td>UserName</TD><TD><input type="text" name="val_x" size=12></td></tr>'\
         '</tr></table>'
#Step 2
echo "<form method=GET action=\"${SCRIPT}\">"\
        '<table nowrap>'\
                '<tr><td>FileName not_cleaned</TD><TD><input type="text" name="val_y" size=12></td></tr>'\
                '</tr></table>'
#Step 3
echo "<form method=GET action=\"${SCRIPT}\">"\
        '<table nowrap>'\
                 '<tr><td>FileName cleaned</TD><TD><input type="text" name="val_z" size=12></td></tr>'\
                '</tr></table>'

echo '<br><input type="submit" value="Search">'\
       '<input type="reset" value="Reset"></form>'

# Make sure errors get shown in web browser.  DO THIS FIRST!
exec 2>&1

# Make sure web browser shows everything raw

echo "Text output test"
echo "Error output test" >&2
echo
echo "Query is: ${QUERY_STRING}"

IFS="&=" ; set -- ${QUERY_STRING}

while [ "$#" -gt 0 ]
do
        echo "Got variable $1=$2"

        case "$1" in
        val_x) XX="$2" ; shift ;;
        val_y) YY="$2" ; shift ;;
        val_z) ZZ="$2" ; shift ;;
        *) ;; # Ignore unknown option
        esac
        shift
done

echo "got XX=$XX YY=$YY ZZ=$ZZ"

grep -r "$XX" /opt/SUNWappserver/nodeagents/ins1/logs/ > /tmp/$YY.txt
echo "grep returned $YY.txt"
sed 's/INFO.*;|//g' /tmp/$YY.txt > /tmp/$ZZ.txt
echo "sed returned $ZZ.txt"

     echo '<br>'

echo '</body>'
echo '</html>'
exit 0

You have followed none of my suggestions. I suggest you follow them and tell me what happens then.

I run your script via browser :

#!/bin/bash

# Make sure errors get shown in web browser. DO THIS FIRST!
exec 2>&1

# Make sure web browser shows everything raw
echo "Content-Type: text/plain"
echo

echo "Text output test"
echo "Error output test" >&2
echo
echo "Query is: ${QUERY_STRING}"

IFS="&=" ; set -- ${QUERY_STRING}

while [ "$#" -gt 0 ]
do
echo "Got variable $1=$2"

    case "$1" in
    val_x\) XX="$2" ; shift ;;
    val_y\) YY="$2" ; shift ;;
    val_z\) ZZ="$2" ; shift ;;
    *\) ;; \# Ignore unknown option
    esac
    shift

done

echo "got XX=$XX YY=$YY ZZ=$ZZ"

grep "$XX" /var/www/html/* > /tmp/a.txt
echo "grep returned $?"
sed 's/INFO.*;|//g' /tmp/a.txt > /tmp/b.txt
echo "sed returned $?"

The result is :

Text output test
Error output test

Query is:
got XX= YY= ZZ=
grep returned 0
sed returned 0