Help with File Monitoring Script

I am getting errors when I try to run the script I just made, any suggestiongs would be helpful. Please be gentle, Im still a newbie and learning on the go at an entry level position.

#!/usr/bin/ksh
# PURPOSE: This script is going to view the top 10 largest home directory Folders, then go in each directory and list top 10 largest files.
#
#Author: Emmanuel Iroanya Jr
#Date: March 8,2012
#
##### DEFINE FILES AND VARIABLES HERE ####
WORKFILE="/tmp/homedf.work" # Holds filesystem data
>$WORKFILE # Initialize to empty
OUTFILE="/tmp/homedf.outfile" # Output display file
>$OUTFILE # Initialize to empty
THISHOST=`hostname` # Hostname of this machine
USER=`ls /home | sort -n -r | head -n 10` # Define User
counter=0 # Sets Counter to 0
######## START OF MAIN #############

cd /home
du -sk /home | sort -n -r | head -n 10 > $WORKFILE

while read $USERS

do 
du -a $USER | sort -n -r | head -n 10 > $OUTFILE
counter=counter+1
done

if [[ -s "$WORKFILE" && "$OUTFILE"]]
then
cat "$WORKFILE" && "$OUTFILE"
print
mailx -s "Home Folder Size Files" exxxxxxl_ixxxxxa@xxxxxx.com < "$WORKFILE" && "$OUTFILE"

fi

---------- Post updated at 12:13 PM ---------- Previous update was at 11:52 AM ----------

/usr/bin/ksh: syntax error: `newline or ;' unexpected <---- First error I recieved.

I'm horrible at searching through my own code. ANother pair of eyes would help alot. thanks

where does the $USERS come from? (or gets its value(s)... I havent found...)
then

if [[ -s "$WORKFILE" && "$OUTFILE" ]]

(missing space before ]])

Maybe I did it wrong, but $USERS should be getting its value from above where i make all my variables.

USER=`ls /home | sort -n -r | head -n 10`

---------- Post updated at 12:31 PM ---------- Previous update was at 12:30 PM ----------

well, thats one part wrong, I have a S on $USERS and not when I make the USERS variable.

Now you could:

ls /home | sort -n -r | head -n 10| while read USER
do
  <the rest...>
done

Did you add the extra missing space?

The test is false also...

if [[ -s "$WORKFILE" && "$OUTFILE" ]]

what are you trying to test?
Something like?

if [ -s "$WORKFILE" -a  -s "$OUTFILE" ]

Surely this will sort the name of the directory not the size of the contents of the directory.

The top ten directories sorted by the size of the contents is:

du -sk /home/* | sort -n -r | awk '{print $2}' | head -10

Thanks Methyl, Iwas trying to figure out all possible syntax errors, and did not look what it was doing (yet).. Good point!

Though $counter is actually irrelevant in this script, syntax for the arithmetic is wrong:

# Wrong
counter=0      
counter=counter+1
echo $counter
counter+1

# Right
counter=0
let counter=$counter+1
echo $counter
1

# Modern syntax
counter=0
counter=$(($counter + 1))
echo $counter
1

One of the many methods of finding the top 10 largest files in a directory is as follows:

find "${dir}" -type f -exec ls -lad {} \; | sort -n -r +4 | head -10
# If you have knowledge of the size of files you could only look at big files
#  -size +1000000c 

Imho the smallest unit of storage returned by "du" (one block) is too large for this exercise on files. However my method is totally useless on systems which return file sizes in "human readable" format.

1 Like

You guys are really helpful, allow me to make some changes to the code, I will post once more, and confirm that I understand what I should have changed

---------- Post updated at 01:48 PM ---------- Previous update was at 01:44 PM ----------

#!/usr/bin/ksh
# PURPOSE: This script is going to view the top 10 largest home directory Folders, then go in each directory and list top 10 largest files.
#
#Author: Emmanuel Iroanya Jr
#Date: March 8,2012
#
##### DEFINE FILES AND VARIABLES HERE ####
WORKFILE="/tmp/homedf.work"  # Holds filesystem data
>$WORKFILE                   # Initialize to empty
OUTFILE="/tmp/homedf.outfile" # Output display file
>$OUTFILE                   # Initialize to empty
THISHOST=`hostname`      # Hostname of this machine
USER=`du -sk /home/* | sort -n -r | awk '{print $2}' | head -10`    # Define User
counter=0                  # Sets Counter to 0
######## START OF MAIN #############

cd /home
du -sk /home | sort -n -r | head -n 10 > $WORKFILE 
ls /home | sort -n -r | head -n 10
while read $USER
do 
    du -a $USER | sort -n -r | head -n 10 > $OUTFILE
    let counter=$counter+1
    echo $counter
done 
if [[ -s "$WORKFILE" -a -s "$OUTFILE" ]]
    then
    cat "$WORKFILE" "$OUTFILE"
    print
    mailx -s "Home Folder Size Files" exxxxxxxl_ixxxxxa@xxxxxx.com[ < "$WORKFILE" && "$OUTFILE"
fi

Now if i understand, you said the counter portion is useless,, so after i take that out, I should be good to go correct?

---------- Post updated at 02:14 PM ---------- Previous update was at 01:48 PM ----------

One Last error when I post the edited code I get this error as well

ksh: syntax error: `-a' unexpected

I am assuming its coming from the line:

du -a $USER | sort -n -r | head -n 10 > $OUTFILE

Note :

. If he then need to scan the list, the awk '{print $2}' can be made avoided by then using | while read a a (still in ksh).
Indeed : the second assignment of the variable a will overwrite the first one so you will get the second
field only.

. the -10 is not necessary since it head already takes 10 lines by default.

So just as a quick shot:

#!/usr/bin/ksh
TMPF=mymail.tmp
>$TMPF
du -sk /home/* | sort -n -r | head | while read a a
do
# make sure the output entry refer to a directory
echo "=== $a ===" >>$TMPF
[[ -d "$a" ]] && \
find $a -type f -ls | sort -k 7n | head >>$TMPF
done
mailx -s "Top 10 Big Folders & Files in it" whoever@whereever.whatever <${TMPF} 

(assuming the size of file appear in the 7th column of the -ls of find command)

After changes

What I am getting not in an e-mail:

xmladm
williamb
tranadm
trainadm
tonyd
tombe
toddm
tiat
testuser
testadm
/usr/bin/ksh: $USRDIR: is not an identifier

What I am expecting in an e-mail:

1484373 krishnaa
160446  joe
148463  kimh
112804  michaelr
89522   craign
79895   solomond
78694   brandonp
63859   charlesc
63769   miker
39251   michael 
krishnaa:
2968746 .
2956268 ./bin
262688  ./bin/EVServices
256352  ./bin/Bureau
154288  ./bin/CharacteristicCitizens
137712  ./bin/CharacteristicReg
134048  ./bin/Initiator
132480  ./bin/CalculationSTD
129568  ./bin/CharacteristicCmp
127024  ./bin/VerificationService
etc...

CODE:

#!/usr/bin/ksh
#PURPOSE: This script is going to view the top 10 largest home directory Folders, then go in each directory and list top 10 largest files.
#
#Author: Emmanuel Iroanya Jr
#Date: March 8,2012
#
##### DEFINE FILES AND VARIABLES HERE ####
WORKFILE="/tmp/homedf.work"             # Holds filesystem data
>$WORKFILE                              # Initialize to empty
OUTFILE="/tmp/homedf.outfile"           # Output display file
>$OUTFILE         # Initialize to empty
THISHOST=`hostname`      # Hostname of this machine
USER=`du -sk /home/* | sort -n -r | head -10 | awk '{print $2}'`      # Defines User
counter=0           # Sets Counter to 0
######## START OF MAIN #############
cd /home
du -sk | sort -n -r | head -n 10 > $WORKFILE
ls /home | sort -n -r | head -n 10
for $USRDIR in $USERS
do  
 du -a $USRDIR | sort -n -r | head -n 10 >> $OUTFILE
 let counter=$counter+1
 echo $counter
done
if [[ -s "$WORKFILE" && -s "$OUTFILE" ]]
 then
    cat "$WORKFILE" "$OUTFILE"
    print
 mailx -s "Home Folder Size Files" (email address) < "$WORKFILE" && "$OUTFILE"
fi

Check your variable names ($USERS vs $USER) ?