search all files and sub directory

Shell Programming and Scripting
1

I wanted to search in all the sub directories under /vob/project (recurse) in everything inside /vob/project.

search.run

for x in `cat search.strings`
do
find /vob/project -type f -print | xargs grep -i $x > ~/$x.txt
done

search.string
hello
whoami

I am getting the error
Maximum line length of 2048 exceeded.

Thanks for any help

2

The message is giving you a clue here. Is there any line in any file under /vob/project that contains a line longer than 2048 chars, which is a limit for some Unix commands?

Try this to count the length of lines in your files

#!/bin/sh

for file in "$@"; do
  echo -e "\nFILE: $file"
  awk '{printf( "%5d: %s\n", length( $0 ), $0 ) }' $file
done

Save this as linelength.sh, make it executable, and run it as
$ linelength /vob/project/*

Cheers
ZB

3

Thank you Bob.

Yes, it looks like there are some files, it returned a whole bunch of results.

Is there any way i can still search by ignoring these.

i ma new to unix.

thank you
siva

4

The script will post the length of all lines in all files, regardless of length. To find the lines longer than 2047 chars, try

#!/bin/sh

for file in "$@"; do
  awk -v f=$file '{ if ( length( $0 ) > 2047 ) {
         printf( "%s %5d: %s\n", f, length( $0 ), $0 );
         }
       }' $file

done

and invoke it as above.

If you have a newer version of wc you can try wc -L /vob/project/* (captial L) to show the length of the longest line in each file.

So, assuming your wc supports this,

#!/bin/sh

MAX_LINE=2047
TEMPFILE=/tmp/out.$$.txt

# first, create temp file with list of files + max line lengths
find . -type f -print | xargs wc -L | grep -v total >> ${TEMPFILE}

while read term
do
   while read line
   do
      max_line=$(echo $line | awk '{print $1}')
      filename=$(echo $line | awk '{print $2}')
      if [ "$max_line" -lt "$MAX_LINE" ]; then
         echo "$filename:" >> ~/$term.txt
         grep -i "$term" $filename >> ~/$term.txt
      fi
   done < ${TEMPFILE}
done < search.strings

rm -f ${TEMPFILE}

Will do what you want, excluding any file from the search that contains a line longer than 2047 chars.

Cheers
ZB

5

Thanks a lot Bpb.

Looks like the wc -L (-L) option is not there in this version.

just wondering how should i do.

thank you
siva

6

Substitute

find . -type f -print | xargs wc -L | grep -v total >> ${TEMPFILE}

with

for file in $(find . -type f -print)
do
linecount=`awk 'BEGIN { f=0 } {if ( length( $0 ) > f ) { f = length( $0 ); }} END { print f}' $file`
string="$linecount $file"
echo "$string" >> ${TEMPFILE}
done

Obviously, replace the "." in the find command with the directory you are reading, /vob/project, in your case

Cheers
ZB