UNIX Shell Scripting (Solaris) for File Checking

Hi guys,

I'm sorry but i badly need your help. I am assigned to do a basic shell script in my job but sadly i don't have any idea on what it is because i am an electronics engineer, but i googled all of it, ask my friends but i cant finalize my scripts. so do please help me.

The requirement is i hve to do a shell script in unix solaris that will check invalid files in the directory. when there is an invalid file. it will echo "there is an invalid file in this directory(=the directory itself) please see dummyevents.log." then is no invalid file there. it will echo"there is no invalid file in this directory(=the directory itself)". then it will not produce dummyevents.log

how will you knwo that it is invalid file? ihave list of words like. if there is no "apple" word inside the file in the directory. it is invalid.

i hope you understand guys. and please see below draft of my scripts.

apple_DIR=/home/cfg/dirAPPLE.cfg
banana_DIR=/home/cfg/dirBANANA.cfg
orange_DIR=/home/cfg/dirORANGE.cfg
RESULT=/home/
apple_file=`cat $APPLE_DIR`
banana_file=`cat $BANANA_DIR`
orange_file=`cat $ORANGE_DIR`
#apple
for a in $apple_file
    do 
	cd $a
	touch Statuscheck.tmp
	ls -lrt | grep -l apple > Statuscheck.tmp
    FILE=`cat Statuscheck.tmp | wc -l`
    if [ $FILE -ne 0 ]
	then
	   echo "Invalid file in $a directory" > $RESULT/dummyevents.log
    fi
    rm Statuscheck.tmp
    done
#banana
for b in $banana_file
    do 
	cd $b
	touch Statuscheck.tmp
	ls -lrt | grep -l banana > Statuscheck.tmp
    FILE=`cat Statuscheck.tmp | wc -l`
    if [ $FILE -ne 0 ]
	then
	   echo "Invalid file in $b directory" > $RESULT/dummyevents.log
    fi
    rm Statuscheck.tmp
    done
#orange
for c in $orange_file
    do 
	cd $c
	touch Statuscheck.tmp
	ls -lrt | grep -l orange > Statuscheck.tmp
    FILE=`cat Statuscheck.tmp | wc -l`
    if [ $FILE -ne 0 ]
	then
	   echo "Invalid file in $c directory" > $RESULT/dummyevents.log
    fi
    rm Statuscheck.tmp
    done

Not sure if I understood correctly. Let me paraphrase:
You have a handful of directories with some files in each. You want to check if those files have exactly ONE keyword in them, each directory having a different keyword. While the files to be searched are supplied in a config file, one per directory, the keywords are NOT but are constants. Correct?

Some comments upfront:

  • *nix differentiates between cases, so apple_DIR does NOT equal APPLE_DIR .
  • you can't cat directories ( cat $APPLE_DIR ) - or is that variable name just misleading?
  • with identical code pieces (as shown for the different fruits), loops or subroutines/functions lend themselves for implementation.

Other optimization potential we can discuss once the requirements are clear.

EDIT: And, of course: Please use code tags as required by forum rules!

1 Like

i'm so sorry for the confusion.

in exact i have 7 directories that i need to check. let me give an example.

first directory /home/banana and this directory has files which has "BANANA" word in it. so what they want me to do is to detect the file in the directory WITHOUT "BANANA" word in it because it is considired an ivnalid file in the directory. when file(s) WITHOUT "BANANA" is/are detected it will echo "there is invalid files in this directory. please see dummyevents.log"

and the resutls of the script is the dummyevents.log which when you view it, will contain the directories who has a invalid file and the filename of the invalid file.

i hope you understand.

also you can diregard my draft script and suggest another script construction or scipt approach.. thanks.

Could you post an example of your config data, or are you happy with any (new) structure of config file(s)?
Any subdirectories under e.g. /home/banana?

1 Like

And, keyword always the same as the directory name? Should the search respect case or not?
Where should the config file reside?

1 Like

You failed to mention which shell you are using. With a not too old bash that has extended pattern matching with the extglob shell option, and a config file with directory and keyword in it like

./apple         tart
./orange        juice
./banana        banana

this might already do the trick:

while read DIR KW REST; do cd $DIR; echo invalid files in $DIR: $(ls !(.|..|$(grep -l $KW *))); cd ..; done < dave.cfg;
invalid files in ./apple: X y
invalid files in ./orange: Z
invalid files in ./banana: x

Of course, this is not yet bullet proof, as there's no error checking yet built in, and the positive message is missing, too.

1 Like

I hope this is not coursework/homework.
And I assume that you want to check the file names (not the file data)

basedir=/home
log=$basedir/dummyevents.log

for word in banana apple orange
do
  dir=$basedir/$word
  badfile=0
  
  for fname in $dir/*
  do
    if [ -f "$fname" ]
    then
      case "$fname" in
      *$word*) # these are ok
      ;;
      *) # all others
        echo "Invalid filename '$fname'" # >> $log
        # not redirected here, for efficiency we redirect the whole loop
        badfile=1
      ;;
      esac
    fi
  done >> $log
  
  if [ $badfile -eq 1 ]
  then
    echo "Invalid file in $dir, see $log"
  fi
done
1 Like

Hi MadeinGermany,

I;m so sorry but im looking inside of each file. not just the file name, but your codes has given me some idea. Thanks a lot and hope you could help me more. :smiley:

---------- Post updated at 03:06 AM ---------- Previous update was at 12:49 AM ----------

Hi RudiC,

i am open for other codes and scripts. :smiley:

lately this is my script.

BANANA_DIR=/home/dave/cfg/dirBANANA.cfg
ORANGE_DIR=/home/dave/cfg/dirORANGE.cfg
APPLE_DIR=/home/dave/cfg/dirAPPLE.cfg
RESULT=/home/dave/
banana_file=`cat $BANANA_DIR`
orange_file=`cat $ORANGE_DIR`
apple_file=`cat $APPLE_DIR`
#BANANA
for a in $banana_file
    do 
	cd $a
	touch Statuscheck.tmp
	ls -lrt | grep -l BANANA > Statuscheck.tmp
    FILE=`cat Statuscheck.tmp | wc -l`
    if [ $FILE -ne 0 ]
	then
	   echo "Invalid file in $a directory" > $RESULT/dummyevents.log
    fi
    rm Statuscheck.tmp
    done
#ORANGE
for b in $orange_file
    do 
	cd $b
	touch Statuscheck.tmp
	ls -lrt | grep -l ORANGE > Statuscheck.tmp
    FILE=`cat Statuscheck.tmp | wc -l`
    if [ $FILE -ne 0 ]
	then
	   echo "Invalid file in $b directory" > $RESULT/dummyevents.log
    fi
    rm Statuscheck.tmp
    done
#APPLE
for c in $ftrord_file
    do 
	cd $c
	touch Statuscheck.tmp
	ls -lrt | grep -l APPLE > Statuscheck.tmp
    FILE=`cat Statuscheck.tmp | wc -l`
    if [ $FILE -ne 0 ]
	then
	   echo "Invalid file in $c directory" > $RESULT/dummyevents.log
    fi
    rm Statuscheck.tmp
    done

then yyou are asking for the cfg. i dont know how to insert image because it is asking for the url. im sorry.. but my /home/dave/cfg contains

dirBANANA.cfg
dirORANGE.cfg
dirAPPLE.Cfg

which contains the directories where the script will grep for "BANANA" "ORANGE" and "APPLE" word.

home/dave/cfg also contains
grepBANANA.cfg
grepORANGE.cfg
grepAPPLE.cfg

which contains the word "BANANA" or "ORANGE" or "APPLE" itself,

So you corrected the most obvious errors in your script, which is good. You are "open for other codes and scripts", which is good, too. Did you consider the "loop advice"?

We still don't know what your config files' contents look like. And, if your grep*.cfg files contain the keywords, why are you looking for string constants in your script? What is for c in $ftrord_file , by the way?

I've shown you a very simplistic example of how one could (partly, admittedly) approach your request, given you're free in structuring the config files. But even with another configuration setup, there's many, simple ways to achieve what you need.

1 Like

Hi RudiC,

i have considered your script earlier which is

while read DIR KW REST; do cd $DIR; echo invalid files in $DIR: $(ls !(.|..|$(grep -l $KW *))); cd ..; done < dave.cfg;
invalid files in ./apple: X y
invalid files in ./orange: Z
invalid files in ./banana: x

but i forgot to ask what is the function of this

while read DIR KW REST;
for c in $ftrord_file

is supposed to be

for c in $appe_file

sorry that's my bad

yes. even my friends tell me that my requirements are so easy but im so sorry. i am just a super duper beginner in unix. and i have this script

#!/bin/sh

cd home/testfiles/banana

grep -v BANANA *

if [ $grep 
then 
echo "There is an unknown file in this directory."
elif [ $grep 
then
echo "There is no unknown file in this directory."
fi

but i cant complete this simple one bacuase lack of knowledge in loop commands if commands else commands. and i never continued this script becuase as i have said earlier i have 7 directories. so if i use this script it would be a long .sh file then. so my boss told me that you can create cfg files for your directories but i never really understood what he means. im so sorry

So, where do we start? Being an electronics engineer, is the assumption correct that you know what a specification is? Loosely phrased, the description of a target (contents, shape, structure, timing) and the collection of basic data (setup, sample input, ...) to start from, constraints and/or exceptions and/or boundary conditions, tools available (or forbidden), sometimes accompanied by ideas or even rules/instructions on how to get there.
Still online? Let's start with

The target: Paraphrased from your post#1: A list of directories with the information "contains valid/invalid files". Please refine to taste.

The basics: A number of subdirectories in your home directory (definition varies in above posts) with files to be searched for keywords. Please be very aware that we don't know anything about your computer/(file-) system/directory structure/anything. So some samples might help!
Additional info needed:

  • What flexibiltiy is required?

  • Do the subdirs change a lot?

  • Are there more subdirs?

    • How to define the interesting ones?
    • How to exclude the others?
    • Use a "white list" (= config file)?
  • Do the keywords change?

    • How to provide?
    • Are there more than one keyword per subdir?
  • config files to be used?

    • one? many?
    • structure of the config file(s)
    • up to the service provider?

Constraints/exceptions/boundary conditions:

  • not (yet) known

The tools: available *nix commands/versions (your Solaris have may some restrictions that other OS don't have)

  • pure shell (which do you use?)
  • text processing commands ( sed , awk , perl , ...)
  • up to the service provider?

I'll stop here with this non-exhaustive list. Feel free to complete and answer.

You now may say: "What an immense effort for a minor task like a mere 'file list for 7 directories' ". I'd request you to ponder the overall time it took for the two of us to type posts #1 to #10 compared to you making up your mind and put together a small set of basic data, stringent rules, and a precise, distinct result, with which a satisfactory solution would have been proposed within hours if not minutes.

I'd propose you take a step back and try to rephrase your request considering at least part of above...

1 Like

im so sorry for all the confusion and all my mistakes regarding my past posts. i hope you understand that im a beginner. but i am willing to learn and i am studying and learning these things.

target: a "dummyevents.log" that contains the directories that has "invalid files" in it.

flexibility: any
subdirs/dirs: only 7 directories to be checked
keywords does not change
1 word to be grep per directory

im so sorry regarding config files, we can use it or not depending on the script. the use of cfg file is a suggestion only to make my script short and simple

and im so sorry icant answer all you questions and i dont know the versions or model of my unix but you are correct i have some restrictions in commands
i am using UNIX solaris shell script when i perform showrev -a

Release: 5.8
Kernel architecture: sun4u
Application architecture: sparc
Hardware provider: Sun_Microsystems
Domain:
Kernel version: SunOS 5.8 Generic 117350-53 Jan 2008

OpenWindows version:
X11 Version 6.4.1 14 January 2008

Rephrased request: i have 7 directories, each directory as dafault files, those default files have 1 word in common in it.

home/dave/banana - default files has "BANANA" word in it
home/dave/orange - default files has "ORANGE" word in it
home/dave/orange1- default files has "ORANGE" word in it
home/dave/orange2- default files has "ORANGE" word in it
home/dave/apple - default files has "APPLE" word in it
home/dave/apple1 - default files has "APPLE" word in it
home/dave/apple2 - default files has "APPLE" word in it

i want a script which is as much as possible a short and a simple one,

for example there is a file in home/dave/banana without a "BANANA" word in it. so the script would reverse grep and will detect the files or files without "BANANA" in it then put a "dummyevent.log" in home/dave which when you view that "dumymevent.log" you will see the directory home/dave/banana.

then after that the script will go to home/dave/orange then reveser grep and detect the files without "ORANGE" in it then add home/dave/orange directory in the "dummyevents.log" located at home/dave.

and so on and so forth the remaining directories. but at the same time if the scriprt does not detect any invalid files (which means the all the files in the directory contains the common word) the sciprt will results also a "dummyevents.log" in home/dave which when you view it, it contains only "There is no invalid files in the given directories."

i hope you udnestand.. i cant send you a private message becuase sending a pm requires a minimum of 10 posts. Thanks.

What's your shell?

And: No PM please for technical questions/data.

1 Like

korn shell (ksh)

Ok ok. understood no pm to any moderator(s). noted.

OK, no bash isms. What happens if you create a dave.cfg file in your home directory containing

home/dave/banana        BANANA
home/dave/orange        ORANGE
home/dave/orange1       ORANGE
home/dave/orange2       ORANGE
home/dave/apple         APPLE
home/dave/apple1        APPLE
home/dave/apple2        APPLE

(are you sure, BTW, that those paths are relative and not absolute?) and run

while read DIR KW REST; do cd $DIR; echo invalid files in $DIR: $(ls > ../TMP; grep -l $KW * | grep -vf- ../TMP) >> ../de.log; cd ..;  done < dave.cfg; 

? Are you happy with de.logs contents, at least for the invalid files?

1 Like

hi rudic,

Please take one step at a time. What you presented in post#16 is NOT the directories from post#12. Use full absolute paths for a test, i.e. paths rooted at / and not using variables to be expanded.
Obviously your grep version behaves differently. There might be other versions installed on your node, like /usr/xpg4/bin/grep . Try with this one. If that fails, we need to look further.

1 Like

Hi rudic,

ok lets take it step by step..

im using unix solaros, kornshell.

for example i have a directory

home/dave/solaris/unix/samplefiles

the default files has 1 word in common and that is "UNIX". how can i display the fiesl without "UNIX" word in it using grep command?

thanks a lot for all the help

Hello daveaztig14,

You requirement is not clear but if you want to read/print the file's content on standard output in a specific directory then following may help you in same.

for file in home/solaris/unix/samplefiles/*
do
      grep -v "UNIX" $file
done

If your requirements are different from above code then please do let us know your requirement with complete details with sample Inputs and expected outputs.

Thanks,
R. Singh

1 Like

Hi R.Singh,

i used your code but it doest results to anything.

forexample in home/solaris/unix/samplefiles/ directory i have files

dave1.txt
dave2.txt
dave3.txt
dave4.txt
dave5.txt

dave1 to dave4 .txt has

davehandsome
~
~
~
~
~
~

in it.. and dave5.txt dont have..

so i would like to

grep -v davehandsome

so that my output would only be dave5.txt