UNIX Shell Scripting (Solaris) for File Checking

That is not what grep -v BRE file... does. That grep command gives you a list of every line in each of the given file operands that does NOT contain a string matching the basic regular expression specified by BRE .

If you want to list all files in the directory home/solaris/unix/samplefiles/ which is a relative pathname of a sub-sub-sub-subdirectory of your current working directory, you could use something like:

cd home/solaris/unix/samplefiles/ && \
for file in *.txt
do	if ! grep -q davehandsome "$file"
	then	printf 'File "%s" does not contain the string "davehandsome"\n' "$file"
	fi
done
1 Like

hi don cragun,

i tried using your code but it resutls to

grep: illegal option -- q
Usage: grep -hblcnsviw pattern file . . .
File "*.edi" does not contain the string "davehandsome"

may be its because of the limitations of my unix..

thank you and sorry but can you check this script of mine,

Hello daveaztig14,

Could you please try following and let me know if this helps.

find . -type f -exec grep -H -c 'FTRORD' {} \+ | grep 0$ | cut -d':' -f1

I have checked above in BASH environment.

Thanks,
R. Singh

1 Like

HI R. Sing

What you have shown us is that there are four lines in each of the files:

VZW997.03.26.2010:06:10:12.000
VZW997.03.26.2010:06:33:46.007
VZW997.03.26.2010:06:37:47.004
VZW997.03.26.2010:06:54:36.003

that do not contain the string FTRORD . There is no indication that there is not another line in one or more of these files does contain that string.

Try using my script again, changing grep to /usr/xpg4/bin/grep .

1 Like

hi don,

Don, grep -q ... is not portable, and if ! does not work in a Bourne shell.
Instead one needs grep ... >/dev/null ,
and one of the following workarounds

grep ...
if [ $? -ne 0 ]
then
  echo "not matched"
fi

or

if grep ...
then
  :
else
  echo "not matched"
fi

or

grep ... || {
  echo "not matched"
}

The following is a modification of my post #7, and greps the words in the file data.

basedir=/home
log=$basedir/dummyevents.log

for word in banana apple orange
do
  dir=$basedir/$word
  badfile=0
  
  for fname in $dir/*
  do
    if [ -f "$fname" ]
    then
      grep -w "$word" "$fname" >/dev/null
      if [ $? -ne 0 ]
      then
        echo "No '$word' in '$fname'"
        # We redirect the standard output of the whole loop
        badfile=1
      end
    fi
  done >> $log
  
  if [ $badfile -eq 1 ]
  then
    echo "Invalid file in $dir, see $log"
  fi
done
1 Like

You stated twice you were using ksh but the sample script posted here is using the legacy bourne shell /bin/sh which you should never use.

You'd rather start your scripts with one of:

#!/bin/ksh

or

#!/usr/xpg4/bin/sh
1 Like

hi made in germany,

thank you and i am trying your code right now. but can you check my script at post #26?. what went wrong or what is my mistake? super thank you so much for the help.

The directories you are searching, the strings you are matching, the names of the files you are processing in the directories you are searching, and the shell you are using seems to change with every post you make. We can't keep up with all of the changes you are making and we still do not have a clear description of what you are trying to do. (Especially whether some of the pathnames you are supplying are the names of files to be processed or the names of directories [AKA folders] containing files to be processed.)

But some problems with your code in post #26:

#list all folders for FTR
folderFTR="/files05/home/chgtpst/GT/outb/CHTR/FTR/9/archive_ia
/files05/home/chgtpst/GT/outb/CHTR/FTR/9/send_ia
/files05/home/chgtpst/GT/outb/CHTR/FTR/9/send"

#list all folders for VRZ
folderVRZ="/files05/home/chgtpst/GT/outb/CHTR/VRZ/9/archive_ia
/files05/home/chgtpst/GT/outb/CHTR/VRZ/9/send
/files05/home/chgtpst/GT/outb/CHTR/VRZ/9/send_ia/success"


#Main program for FTR
for f in $folderFTR
do
    if [ 'grep FTRORD -l "$f"' ]
    then
        echo "invalid file in $f" >> /home/abainzd/logfile.txt

    else
       exit 1 
    fi
done


#Main program for VRZ
for f in $folderVRZ
do
    if [ 'grep CHTRVRZPRD -l "$f"' ]
    then
        echo "Invalid file in $f" >> /home/abainzd/logfile.txt

    else
       exit 1 
    fi
done

include (but might not be limited to):

  1. You have said you are using the Korn shell, but there is nothing in this script that specifies that. Make the 1st line of your script be #!/bin/ksh .
  2. We can't tell from what you have shown us whether the three pathnames you assigned to the shell variable folderVRZ are the names of directories (which you call folders) or are pathnames of three regular files in the directory /files05/home/chgtpst/GT/outb/CHTR/VRZ/9 .
  3. The test command [ 'grep FTRORD -l "$f"' ] ALWAYS evaluates to true because the argument being evaluated 'grep FTRORD -l "$f"' is a non-empty fixed string. It does not execute grep and it does not expand "$f" (shell variables are not expanded when they are surrounded by single quotes) to the name of a file (directory or regular file). And, assuming (based solely on the name of your variables) that you intend for $f to expand to the name of a directory, the grep utility looks for matches to a regular expression in the text file(s) named as its file operand(s); it does NOT search all of the files in a directory named as a file operand and it does not try to match filenames in a directory to your regular expression. The behavior of grep is only defined when the pathnames of the files you ask it to process are text files (and directories are not text files)!

I apologize for not noticing that you used /bin/sh when you ran my script instead of /bin/ksh ) after you had told us at least three times that you were using the Korn shell.

I apologize for not mentioning that you would have to use /usr/xpg4/bin/grep instead of /usr/bin/grep to use the -q option.

I have absolutely no explanation for why you modified the script I suggested in post #21 to process the four files you described in post #20:

cd home/solaris/unix/samplefiles/ && \
for file in *.txt
do	if ! grep -q davehandsome "$file"
	then	printf 'File "%s" does not contain the string "davehandsome"\n' "$file"
	fi
done

to:

cd $EDI_ROOT/GT/outb/CHTR/VRZ/9/send && \
for file in *.edi
do      if ! /usr/xpg4/bin/grep  -q CHTRVRZPRD "$file"
        then    printf 'File "%s" does not contain the string "CHTRVRZPRD"\n' "$
file"
        fi
done

and ran it in a directory where there are no filenames that end with the string .edi when you said you wanted to look for a different string in one of four existing files in a different directory.

If you had told us that some of the directories you want to search do not contain any files, we would have suggested different code, perhaps something more like:

#!/bin/ksh
EDI_ROOT="/absolute/path/to/directory"
# The absolute pathname of the directory named by EDI_ROOT must contain a
# subdirectory accessible by the relative pathname "BT/outb/CHTR/VRZ/9/send".

cd $EDI_ROOT/GT/outb/CHTR/VRZ/9/send && \
for file in *.edi
do      if [ -f "$file" ] && ! /usr/xpg4/bin/grep  -q CHTRVRZPRD "$file"
        then    printf 'File "%s" does not contain "CHTRVRZPRD"\n' "$file"
        fi
done

Note that case matters in Solaris and other UNIX systems. The string "UNIX" and the string "unix" are NOT the same. (Case matters!)

If you were asked (as an EE) to create a circuit that is given +5V DC power and instead of +5V DC power the user connected it to 120V AC power, should we be disappointed if your circuit exploded? If you tell us you want to search for a string in a bunch of files that match a certain filename matching patter in a directory, don't be surprised if a grep command that we suggest doesn't do what you want if you ask it to process files matching a pattern that does not match any files in that directory. If you tell us you want to search for a string in a bunch of files in a directory, don't be surprised if a grep command that we suggest doesn't do what you want if you give it the name of a directory to process instead of a list of one or more files in that directory to process.

Computers can do some things very quickly. They usually do exactly what you ask them to do even if what you ask them to do is dangerous. In CS we have a frequently referenced acronym: GIGO (Garbage In, Garbage Out). If your specifications are not clear (GI), the results you get from running code written to meet what the programmer guesses you meant are likely to give you GO.

Please help us help you!

1 Like

hi don,

i understand and im refelcting all of the things right now.. and im so sorry.. for all the change of post and directories and variable and all.. thanks a lot for your help.

Thank you for the much clearer specification. But, there are still a few things missing in your specification:

  1. You didn't mention the pathname of the log file you want to create. (This script uses /files05/home/chgtprd/logfile.txt .) Note that the directory in which the log file is located MUST NOT be the directory named by the variable dir !
  2. You didn't mention whether this script should recreate the log file each time it runs, or would just add information to an existing log file. (This script recreates the log file each time it runs. To append instead of recreate, change both occurrences of > "$log" to >> "$log" .)
  3. This code assumes that you only want to look at regular files in the named directory (not at all regular files in the file hierarch rooted in the named directory).

With the assumptions stated above, the following script should do what you have requested on a Solaris system (although I did take a few liberties with the messages written to your log file). I assume that you will be able to adjust the text in the printf statements if you don't like the text I used.:

#!/bin/ksh
bad_found=0
dir='/files05/home/chgtprd/MXS/inb/CHTR/FTR/9/'
log='/files05/home/chgtprd/logfile.txt'

cd "$dir" && for file in *
do	if [ -f "$file" ] && ! /usr/xpg4/bin/grep -q FTRORD "$file"
	then	printf 'Invalid file "%s" in "%s"\n' "$file" "$dir"
		bad_found=1
	fi
done > "$log"
if [ $bad_found -eq 0 ]
then	printf 'There are no invalid files in "%s"\n' "$dir" > "$log"
fi

hi don,

1) sorry i forgot to mention that the log file should be created in

/home/abainzd

thank you

Sorry,
Although it won't affect the operation of the script, the last two lines in your script should not be indented.

I had a copy and paste error that dropped the last line of the script (which has now been corrected in post #32). Either add the line:

fi

to the end of the script, or change the last two lines of the script from:

if [ $bad_found -eq 0 ]
then    printf 'There are no invalid files in "%s"\n' "$dir" > "$log"

to:

[ $bad_found -eq 0 ] && printf 'There are no invalid files in "%s"\n' "$dir" > "$log"

Hi don,,

i added

fi

on the last line and it works perfecrtly.. tahnk you so much..

Hello daveaztig14,

A simple way could be put all the directories information into a Input_file and you could read it from there like for an example as follows.

cat Input_file
/files05/home/chgtprd/MXS/inb/CHTR/FTR/9/
/files05/home/chgtprd/GT/outb/CHTR/VRZ/9/send
/files05/home/chgtprd/GT/outb/CHTR/VRZ/9/send_ia/success
/files05/home/chgtprd/GT/outb/CHTR/FTR/9/send
/files05/home/chgtprd/GT/outb/CHTR/FTR/9/send_ia
/files05/home/chgtprd/GT/outb/CHTR/FTR/997/send
/files05/home/chgtprd/GT/outb/CHTR/VRZ/997/send

Then following will be code changes for previously code shown by Don, please try this and let us know how it goes then.

#!/bin/ksh
bad_found=0
log='/home/abainzd/logfile.txt'
while read dir
 do
cd "$dir" && for file in *
do      if [ -f "$file" ] && ! /usr/xpg4/bin/grep -q FTRORD "$file"
         then    printf 'Invalid file "%s" in "%s"\n' "$file" "$dir"
                bad_found=1
         fi
done > "$log"
        if [ $bad_found -eq 0 ]
        then    printf 'There are no invalid files in "%s"\n' "$dir" > "$log"
         fi
done < "Input_file"
 

NOTE: I haven't tested above code.

Thanks,
R. Singh

1 Like

Hi R.Singh,

i have tried your steps.

Hello daveaztig14,

My sincere apologies, somehow I have not written do after while , could you please try following.

#!/bin/ksh
bad_found=0
log='/home/abainzd/logfile.txt'
while read dir
do
cd "$dir" && for file in *
do      if [ -f "$file" ] && ! /usr/xpg4/bin/grep -q FTRORD "$file"
         then    printf 'Invalid file "%s" in "%s"\n' "$file" "$dir"
                bad_found=1
         fi
done > "$log"
        if [ $bad_found -eq 0 ]
        then    printf 'There are no invalid files in "%s"\n' "$dir" > "$log"
         fi
done < "Input_file"
 

Thanks,
R. Singh

1 Like

hi R.sings,

Hello daveaztig14,

Could you please change Input_file to /home/abainzd/Input_file and let me know how it goes.

#!/bin/ksh
bad_found=0
log='/home/abainzd/logfile.txt'
while read dir
do
cd "$dir" && for file in *
do      if [ -f "$file" ] && ! /usr/xpg4/bin/grep -q FTRORD "$file"
         then    printf 'Invalid file "%s" in "%s"\n' "$file" "$dir"
                bad_found=1
         fi
done > "$log"
        if [ $bad_found -eq 0 ]
        then    printf 'There are no invalid files in "%s"\n' "$dir" > "$log"
         fi
done < "/home/abainzd/Input_file"
 

Thanks,
R. Singh

1 Like