Strange Script behaviour with Grep

mohtashims · May 3, 2016, 6:45pm

Here is my script

LOGDATE=`date '+%Y-%m-%d %k:%M' | cut -c1-15`

echo $LOGDATE
echo "grep '$LOGDATE' /tmp/logs/vel.log >10min_log"
grep '$LOGDATE' /tmp/logs/vel.log>10min_log
grep '$LOGDATE' /tmp/logs/vel.log

Here is the output of the script

-rw-r--r--   1 wluser    wluser          0 May  3 17:35 10min_log

You see 0 bytes.

But when i type this manually grep '2016-05-03 17:3' /tmp/logs/vel.log i see 1000s of lines like below.

Like you can see when i manually type the grep command i see 1000s of line but when the same command goes in the script the redirected output to 10min_log shows 0 bytes and no data.

Why ?
Note: when i change grep '$LOGDATE' /tmp/logs/vel.log>10min_log to grep $LOGDATE /tmp/logs/vel.log>10min_log i.e remove the single quotes'' then it works. But i want to grep 2016-05-03 17:3 as one text hence i need the single quotes or if there is another solution ??
Note: I was able to overcome the issue by using double quotes ""
Note: i m with the same user and grp as the directory i.e there is no permission issues.

Please suggest.

RavinderSingh13 · May 3, 2016, 7:01pm

Hello mohtashims,

Could you please change ' to " as follows and let me know how it goes then.

grep "$LOGDATE" /tmp/logs/vel.log>10min_log
grep "$LOGDATE" /tmp/logs/vel.log

Thanks,
R. Singh

mohtashims · May 3, 2016, 8:48pm

ravindersingh13:

Hello mohtashims,

Could you please change ' to " as follows and let me know how it goes then.
grep "$LOGDATE" /tmp/logs/vel.log>10min_log
grep "$LOGDATE" /tmp/logs/vel.log
Thanks,
R. Singh

Like i said that works but why not single quotes ?

Don_Cragun · May 3, 2016, 10:58pm

Because shell variables are not expanded inside single-quoted strings. (And, note that in your echo statement:

echo "grep '$LOGDATE' /tmp/logs/vel.log >10min_log"

single-quote characters are not special inside a double-quoted string; so in that case $LOGDATE is expanded because it is in a double-quoted string; not inside a single-quoted string.)

mohtashims · May 4, 2016, 2:05am

don cragun:

Because shell variables are not expanded inside single-quoted strings. (And, note that in your echo statement:
echo "grep '$LOGDATE' /tmp/logs/vel.log >10min_log"
single-quote characters are not special inside a double-quoted string; so in that case $LOGDATE is expanded because it is in a double-quoted string; not inside a single-quoted string.)

Don the problem i m reporting is this line ->

grep '$LOGDATE' /tmp/logs/vel.log>10min_log

and it does not have double quotes.

Also, like i said when i manually copy paste this it gives me the output. It fails only when executed from inside the script !!

RudiC · May 4, 2016, 2:41am

man bash :

QUOTING
Quoting is used to remove the special meaning of certain characters or words to the shell. Quoting can be used to disable special treatment for special characters, to prevent reserved words from being recognized as such, and to prevent parameter expansion.
.
.
.

There are three quoting mechanisms: the escape character, single quotes, and double quotes.

A non-quoted backslash (\) is the escape character. It preserves the literal value of the next character that follows, with the exception of <newline>. If a \<newline> pair appears, and the backslash is not itself quoted, the \<newline> is treated as a line continuation (that is, it is removed from the input stream and effectively ignored).

Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. . . .

The last paragraph shows that ' will be preserved literally and NOT used for quoting.

Now, does Don Cragun's post#4 become clearer?

Difficult to believe. Please execute the exactly identical command from the command line and from within a script, and compare the results.

Don_Cragun · May 4, 2016, 3:11am

mohtashims:

Don the problem i m reporting is this line ->
grep '$LOGDATE' /tmp/logs/vel.log>10min_log
and it does not have double quotes.

Also, like i said when i manually copy paste this it gives me the output. It fails only when executed from inside the script !!

Sorry. I just do not believe you.

If I run the following commands sequentially in an interactive shell that uses Bourne shell syntax:

$ XYZ=abc
$ echo "$XYZ"
abc
$ echo '$XYZ'
$XYZ
$ echo "XYZ is set to '$XYZ'"
XYZ is set to 'abc'
$

I get the output shown in bold text from those commands. And, if I put the commands in a script:

XYZ=abc
echo "$XYZ"
echo '$XYZ'
echo "XYZ is set to '$XYZ'"

and execute it, I get the output:

abc
$XYZ
XYZ is set to 'abc'

If you use a shell that uses csh syntax instead of Bourne shell syntax, you still get the same results if you change the 1st command from:

XYZ=abc

to:

set XYZ=abc

The only way that the command:

grep '$XYZ' file

and the command:

grep "$XYZ" file

produce the same, non-empty output is if the variable XYZ has been set with something like:

XYZ='$XYZ'

(assuming Bourne shell syntax) or:

set XYZ='$XYZ'

(assuming csh shell syntax) and the file named file contains the literal string $XYZ unless you have a function named grep , an alias for grep , or a non-standard version of the grep utility that is found in your PATH environment variable before the standard version of the grep utility.

mohtashims · May 4, 2016, 3:24am

don cragun:

Sorry. I just do not believe you.

If I run the following commands sequentially in an interactive shell that uses Bourne shell syntax:
$ XYZ=abc
$ echo "$XYZ"
abc
$ echo '$XYZ'
$XYZ
$ echo "XYZ is set to '$XYZ'"
XYZ is set to 'abc'
$ 
I get the output shown in bold text from those commands. And, if I put the commands in a script:
XYZ=abc
echo "$XYZ"
echo '$XYZ'
echo "XYZ is set to '$XYZ'"
and execute it, I get the output:
abc
$XYZ
XYZ is set to 'abc'
If you use a shell that uses csh syntax instead of Bourne shell syntax, you still get the same results if you change the 1st command from:
XYZ=abc
to:
set XYZ=abc
The only way that the command:
grep '$XYZ' file
and the command:
grep "$XYZ" file
produce the same, non-empty output is if the variable XYZ has been set with something like:
XYZ='$XYZ'
(assuming Bourne shell syntax) or:
set XYZ='$XYZ'
(assuming csh shell syntax) and the file named file contains the literal string $XYZ unless you have a function named grep , an alias for grep , or a non-standard version of the grep utility that is found in your PATH environment variable before the standard version of the grep utility.

That helped ... The issue is resolved but i will dwell more into this in the future. Thanks Again.