Peak Ram Usage

grpdsku program allows user to check their group disk space in a server environment. The data in the dialog box queries a text file. Each text file is labeled with a current timestamp. Results output to a msgbox. Also, results output to a csv file. The csv file is sent to the user via email

When running the program, I am getting error } missing . I check each open and close bracket and found no issue. There are other issues with this program. The dialog forms aren't creating and generating output. I appreciate any help you can give me. I have attached a document that explains the program process in more detail.

#!/bin/bash
# grpdsku.sh - A shell script to display the form dialog on screen



 DIALOG_CANCEL=1
 DIALOG_ESC=255
 HEIGHT=0
 WIDTH=0

 display_result() {
   dialog --title "$1" \
    --no-collapse \
    --msgbox "$result" 0 0
 }

 while true; do



 Now=date + %M/%D/%Y
 groups=""
 date1="  "  
 n=""
 email=""

# open fd - fields can't be empty,
 
 error_message=
 IFS=$'\n'; set -f
 while [[ -z $groups || -z $date1 || -z $n|| -z $email ]]; do
   set $(dialog � --form="creating new $error_message" � --output-fd=3 3>&1 >/dev/tty)
   groups=$1 date1=$1 n=$1 email=$1 
   error_message="(fields must not be empty)"
   done
 unset IFS; set +f


# Store data to  variable - data entry form for users

 dialog --separate-widget $\n 
       --clear \
       --ok-label "Submit" \
       --cancel-label "Exit" \
       --backtitle "SAS Managment" \
       --title "Disk Space" \
       --form "Disk Optimization" \15 50 0 \
	"Enter groupname  :"                         1 1	"$groups"  1 10 30 0 \
	"Enter current date (09/30/2016):"           2 1	"$date1"   2 10 10 0 \
	"Enter the number records [1-30 or F]:"      3 1         "$n"      3 10 10 0 \
	"Enter email:"                               4 1	 "$email"  4 10 60 0 \
 2>&1 1>&3)| {
  read -r groups
  read -r date1
  read -r n
  read -r email

 
#-----read in the current timestamp text file----------
 
 DISKINFO=tail -$n "/var/LV/$disk_'date +%Y.%H.%d'.txt"   



#----check date format for example 09/30/2016---------------

 regex=/^[0-1][0-9]/[0-3][0-9]/[0-9]\{4\}$/


 if [[$regex =~ date -d "$date1"]]
 then
 echo $date1 NOT a valid mm/dd/yyy date
 fi

#---------------if group name isn't in text file then display error message--

 if ! grep -q '$group' $DISKINFO ; then
    echo $group Not found.
   ELSE
   continue
 fi

---------exit and continue shell scripting-----
 exit_status=$?

# close fd
  exec 3>&-

 case $exit_status in
    $DIALOG_CANCEL)
      clear
      echo "Program terminated."
      exit
      ;;

    $DIALOG_ESC)
      clear
      echo "Program aborted." >&2
      exit 1
      ;;
  Esac



# display values just entered

#match the group name the input to the group name in the text file, for example,  hasasvmi

 grp=(echo "$groups" | grep "$groups" "$DISKINFO")

#--------read in text file line by line.  Print and format date and time---------

  while IFS='' read -r f1 f2 f3 f4 f5 f6

 do

 f7=  date +%m/%d/%Y | awk  'BEGIN {FS=". _" OFS="/"} {print $6'}' 

 f8  = date +%T   "  | awk  'BEGIN {FS="_ ." OFS=":"} {print $f6}' 


#---if field f4 is null or has a dash then add 0.0%---------------------

 if [${f4} eq'' || ${f4} eq '-'] Then ${f4} eq'0.0%'

 fi
done

#----if the date from the input form matches the date  from the text file and group name then print the fields
fi
 do
 if[${date1}==${Date}] && [grp] THEN



  # display fields using f1, f2,..,f8 

  echo result =( printf '\tFileSystem%s\tSize%d%s\tAvail%d%s\tused(GB,KB,MB)%d %s\tuse% %s\ tMounted on %s\ tDate%s\ tTime%s\n' "$f1" "$f2" "$f3" "$f4" "$f5"  "$f6" "$f7" "$f8")
  fi




  display_result "Disk Space used by $group"

#------print result output to a csv file. CSV File is group label--------

"$result" > $group.csv

#---------csv file is sent via email to user-----------------

[ -s group.csv ] && mailx -s "subject text" -a  "$group.csv" "$email" < "Disk space used by $group "
done <"$DISKINFO"

Group name Group name current date: 09/30/2016

Example 2: output
from disk_2016.09.29.txt :

Below is how I want the output results to be displayed in
a csv file and on screen. The output must have headers. Date,and time must be
formatted. Columns must be aligned
correctly.

This:

2>&1 1>&3)| {

is opening a code block and there is no closing brace.

At the start of this block these variables are initialised through read statements from stdin that is piped ( | ) into the code blok from the dialog command :

  read -r groups
  read -r date1
  read -r n
  read -r email

Outside the code block these variables do not retain their values (this is because a pipe construction | is used and in most shells this is executed in the background).

Since one or more of them or their derivatives is used in the last mailx command, which is used in a loop, there should be a single closing brace after this loop:

  done <"$DISKINFO"
}
1 Like

OS is AIX

The first part of the program check if the input of a particular date and group name exist in a text file. The input values need to match on that entire record in the text file. This process is similar to SQL where clause statement.

The second part of the program display an error message and exit the loop if there isn't a match or else continue if there is a match. The date format needs to change from mm/dd/yyyy to mm/dd/yy in the output of results. The input variable n limit the number of records that are displayed.

I'm getting an error message when I run my code. Error message: too many errors to read.

a) Input variables in the code are

        group=hasasvmi
        date1=09/29/2016
         n=1 

b) The following is the data in the text file
Example 1: data in disk_2016.09.29.txt :

/dev/sasdwk/hasasvmi 128 1 127  0% /sasdwk/hasasvmi  2016.09.29_11.30.27

c) The following is an example of the results that I want to display:

Filesystem                 GBks Free Used used%    Mt                  DT         TM                                                                     
/dev/sasdwk/hasasvmi 128   1      127    0%  /sasdwk/hasasvmi 09/29/16  11:30       
group=hasasvmi
date1=09/29/2016
n=1
|{
read "$group"
read "$date1"
read "$n"

DISKINFO=tail $n" k_%Y.%M.%D.txt

  while IFS='' read $line

      do

        group=hasasvmi
        date1=09/29/2016

       f1='grep ^"f1" $line | awk '{print $1}''
       f2='grep ^"f2" $line | awk '{print $2}''
       f3='grep ^"f3" $line | awk '{print $3}''
       f4='grep ^"f4" $line | awk '{print $4}''
       f5='grep ^"f5" $line | awk '{print $5}''
       f6='grep ^"f5" $line | awk '{print $6}''

       If  grp= echo( $grep "$group" "$line") &&
       f7= echo ('grep " $date1" | date2= (date +%m/%d/%Y | gawk  "$line" 'BEGIN {FS=". _" OFS="/"} {print $7}'')
       && f8= (date +"%H:%M" | gawk "line" 'BEGIN {FS="_ ." OFS="."} {print $7}'')
       Then
         continue
            printf '\tFileSystem%s\tGBks%d%s\t Free%d%s\tused%d %s\tused%%s\ tMt %s\ tDate%s\ tTime%s\n'     
                   "$f1" "$f2" "$f3" "$f4" "$f5"  "$f6" "$f7" 
        else
             echo '$grp ' doesn't exist (exit)|| echo  '$f7' doesn't match the date on file for'$grp' (exit)

    fi

done <"$DISKINFO"

Hi,

seems lot of issues here.

 2>&1 1>&3) | {
if [[$regex =~ date -d "$date1"]]

has to be ( note the space after [[ and before ]].

if [[ $regex =~ `date -d "$date1"` ]]

it has to be small one. case sensitive.

  Esac

what is the aim here ?
Below codes are incorrect, IMO.

 grp=(echo "$groups" | grep "$groups" "$DISKINFO")
 f8 = date +%T   "  | awk  'BEGIN {FS="_ ." OFS=":"} {print $f6}' 
if[${date1}==${Date}] && [grp] THEN

and many more.

can you please have a close look line by line and go through basic tutorial such as thisone ?

After that you can come back with your questions later on, which will help you , when people explains about the issue/technical detail.

Yes.

I will point out the most obvious ones, but i wouldn't bet on having found all of them. The script is in a state which begs a rewrite anyway.

#!/bin/bash

You mentioned using AIX. The system shell is ksh, not bash. Use it!

Now=date + %M/%D/%Y

I suppose you want to use the date command to fill a variable there. This will fill the variable "Now" with the string "date" instead and the rest of the line will produce a syntax error. You do it this way (i have corrected the date-command too):

Now=$(date +"%M/%D/%Y")
while [[ -z $groups || -z $date1 || -z $n|| -z $email ]]; do

This is not ouright wrong, just dangerous. Suppose the variable groups has no content: this would leave the following command to be executed:

while [[ -z || -z $date1 || -z $n|| -z $email ]]; do

similar, if groups contains a space char:

while [[ -z foo bar || -z $date1 || -z $n|| -z $email ]]; do

and this, subsequently, will lead to a syntax error. If you query variables and their contents do it this way (notice the quoting):

while [[ -z "$groups" || -z "$date1" || -z "$n" || -z "$email" ]]; do

or, even more preferably, because being POSIX-compliant:

while [ -z "$groups" || -z "$date1" || -z "$n" || -z "$email" ] ; do

This will also not work:

 dialog --separate-widget $\n 
       --clear \

Notice the "n" at the end of the first line. When you continue a line with a backslash, it needs to be the absolutely last character in this line, because in fact you are escaping the following newline character with it. If there is another character (common problem: a space) after it, the backslash will escape that instead of the newline.

DISKINFO=tail -$n "/var/LV/$disk_'date +%Y.%H.%d'.txt"

Same as above, if you want a variable filled from a commands output you need to do it like this:

DISKINFO=$(tail -$n "/var/LV/$disk_$(date +"%Y.%H.%d").txt") 
regex=/^[0-1][0-9]/[0-3][0-9]/[0-9]\{4\}$/
if [[$regex =~ date -d "$date1"]]

This won't work at all. First, "date -d" is not available in AIX (and in no other POSIX-compliant UNIX, because "-d" is not part of the POSIX standard). Since i cannot figure out what you are trying to achieve no suggestion for an alternative here, but explain what it should do and we probably find a solution.

if ! grep -q '$group' $DISKINFO ; then

almost correct. The problem is that variables ("$group") will not be expanded inside single quotes, only double quotes. And "grep" only works on files or streams, not on fixed strings:

if ! echo "$DISKINFO" | grep -q "$group" ; then

Finally: capitalisation is not arbitrary in UNIX. If a keyword is "else", it is supposed to be "else", not "Else", "ELSE" or anything else either (sorry for the pun). Likewise with all the other keywords: do, done, if, while, then, ...

I hope this helps.

bakunin

1 Like

Hi Bakunin, this is not right. Again, this is mixing up shell keywords ( which [[ ... ]] are) and a shell utility (which is the case with [ ... ] ) . There really is no reason to use double quotes here, unless you want to combine whitespace separated fields (which is not the same as field splitting) or prevent pattern matching together with the == operator or regex matching together with the =~ operator (if available).

From man bash

and man ksh (both 88 and 93):

This syntax is incorrect and not part of POSIX. Correct POSIX syntax would be:

while [ -z "$groups" ] || [ -z "$date1" ] || [ -z "$n" ] || [ -z "$email" ]; do

Thank for your help. I want to get my code running. I realized I need to get the code working before adding the shell script code to zenity dialog i. Here is what I am trying to do

The first part of the code check if the input of a particular date and group name exist in a text file. The input values need to match on that entire record in the text file. This process is similar to SQL where clause statement.

The second part of the code display an error message and exit the loop if there isn't a match or else continue if there is a match. The date format needs to change from mm/dd/yyyy to mm/dd/yy in the output of results. The input variable n limit the number of records that are displayed.

I'm getting an error message when I run my code. Error message:

 too many errors to read

.

a) Input variables in the code are

group=hasasvmi
date1=09/29/2016
n=1 

b) The following is the data in the text file
Example 1: data in disk_2016.09.29.txt :

/dev/sasdwk/hasasvmi 128 1 127  0% /sasdwk/hasasvmi  2016.09.29_11.30.27

c) The following is an example of the results that I want to display:

Filesystem                 GBks Free Used used%    Mt                  DT         TM                                                                     
/dev/sasdwk/hasasvmi 128   1      127    0%  /sasdwk/hasasvmi 09/29/16  11:30

Here is my code:

/bin/ksh
group=hasasvmi
date1=09/29/2016
n=1
|{
read "$group"
read "$date1"
read "$n"

DISKINFO=DISKINFO=$(tail -$n "/var/LV/$disk_$(date +"%Y.%H.%d").txt")

 WHILE IFS='' read $line

DO

        
       f1='grep ^"f1" $line | awk '{print $1}''
       f2='grep ^"f2" $line | awk '{print $2}''
       f3='grep ^"f3" $line | awk '{print $3}''
       f4='grep ^"f4" $line | awk '{print $4}''
       f5='grep ^"f5" $line | awk '{print $5}''
       f6='grep ^"f6" $line | awk '{print $6}''
       Date= $(date +%m/%d/%Y | gawk  "$line" 'BEGIN {FS=". _" OFS="/"} {print $7}'')
       Time= $(date +"%H:%M" | gawk "line" 'BEGIN {FS="_ ." OFS="."} {print $7}'')

  if[${date1}==${Date}] && [grp] THEN'

 printf '\tFileSystem%s\tGBks%d%s\t Free%d%s\tused%d %s\tused%%s\ tMt %s\ tDate%s\ tTime%s\n'     
                   "$f1" "$f2" "$f3" "$f4" "$f5"  "$Date" "$Time" 
     ELSE
             echo '$grp ' doesn't exist (exit)|| echo  '$f7' doesn't match the date on file for'$grp' (exit)

    fi

DONE <"$DISKINFO"
(

Sorry to be blunt, but you first need to learn the basics of shell scripting, only then you can expect to get something - anything - running.

That's what i figured.

/bin/ksh

shebang missing

group=hasasvmi
date1=09/29/2016
n=1
|{
read "$group"
read "$date1"
read "$n"

This will not only not work, it is also superfluous. If you use a pipeline you connect processes: process1 | process2 means: start process1 and whatever comes out of its stdout output channel (the part usually printed on screen) put into process2 s stdin (normally connected to the keyboard). Since variable=value is not a process, there will be no output and hence nothing to feed your pipe.

WHILE
[...]
DO
[...]
ELSE
[...]
DONE

I have told you already that capitalisation matters and that keywords are not "WHILE", but "while". What do you expect the shell to do if you write gibberish (^=unknown commands/keywords)?

My suggestion is to get some introductory book on shell programming and start reading it. You cannot expect to guess your way to some working code without knowing (obviously) neither UNIX nor shell and while we are willing to answer your questions we will not write complete scripts for you. Anything less than that, i suppose, would not suffice to get working code, though.

I hope this helps.

bakunin

If there are "too many errors to read", split the script into executable blocks and test run those, block by block, line by line if need be.

I am getting one error in my program. The path and files do exist.

This is the error message after adding information to each prompt entries

./grpdsku.sh

Enter Group name:
hasasvmi

Enter Current Date:
10/25/2016

Enter number of records:
6


File: ./sas/scripts/OPTUMize/output/disk_utilization/disk_2016.10.25.txt not found. Exiting.

here are the path and example of files that are read in the program

pwd /sas/scripts/OPTUMize/output/disk_utilization/


-rwxr--r--    1 ffxsas  ffxsas     1462455 Mar 04 2016  disk_2016.03.04.txt
-rwxr--r--    1 ffxsas  ffxsas      698461 Oct 23 23:39 disk_2016.10.23.txt
-rwxr--r--    1 ffxsas  ffxsas      407624 Oct 24 13:35 disk_2016.10.24.txt
-rwxr--r--    1 ffxsas  ffxsas      407624 Oct 25 18:35 disk_2016.10.25.txt

Here is the code:

#!/bin/ksh

fpath=. /sas/scripts/OPTUMize/output/disk_utilization/

# get input
echo "Enter Group name: "
read gname

echo "Enter Current Date: "
read cdate
# format: 09/30/2016

# parse date
yyyy=$(echo ${cdate} | cut -d"/" -f3)
dd=$(echo ${cdate} | cut -d"/" -f2)
mm=$(echo ${cdate} | cut -d"/" -f1)

echo "Enter number of records: "
read nrec

# construct file name to parse
diskfile=${fpath}/disk_${yyyy}.${mm}.${dd}.txt

# check if file exists
if [ ! -f ${diskfile} ]
then
	echo File: ${diskfile} not found. Exiting.
	exit 1
fi

# generate output on screen
(echo "Filesystem	GB_blocks	Free	GB/%Used	%Iused	Mounted_on	Group	Date	Time" ; grep ${gname} ${diskfile} | \
awk -v gname=${gname} '{gsub(/_/," ",$7); print $1 " " $2 " " $3 " " $4 " " $5 " " $6 " " gname " " $7 " " $8}' | \
head -${nrec}) | column -t


The 3rd line in your script:

fpath=. /sas/scripts/OPTUMize/output/disk_utilization/

sets the variable fpath to the string . and places it in the environment of the command it will then try to execute:

/sas/scripts/OPTUMize/output/disk_utilization/

which will fail with one of two errors producing a diagnostic message similar to:

./grpdsku.sh: /sas/scripts/OPTUMize/output/disk_utilization/: not found

or:

./grpdsku.sh: /sas/scripts/OPTUMize/output/disk_utilization/:  cannot execute [Is a directory]

Since you didn't show us anything like that in the output of the script that you ran, you didn't show us the script that you ran.

If, instead of:

fpath=. /sas/scripts/OPTUMize/output/disk_utilization/

your script had contained the command:

fpath=./sas/scripts/OPTUMize/output/disk_utilization/

(with no space between the period and the slash and assuming that your current working directory when you ran your script was not / ), that would be likely to lead to the results you showed us. If instead of either of the above, your script had contained the command:

fpath=/sas/scripts/OPTUMize/output/disk_utilization/

(with neither the period nor the space), your script might do what you want. All of this is based on the assumption that the file you want to process is named /sas/scripts/OPTUMize/output/disk_utilization/disk_2016.10.25.txt and that there is no file named $PWD/sas/scripts/OPTUMize/output/disk_utilization/disk_2016.10.25.txt (where $PWD expands to your current working directory).

1 Like

thank for your help. I remove the period in the path of the program. Another error occurred.

input entries:

Enter Group name:
hasasvmi

Enter Current Date:
10/25/2016

Enter number of records:
8

here is the error message after the program ran:

./grpdsku.sh[39]: column:  not found.

Here is my script:

#!/bin/ksh

fpath=/sas/scripts/OPTUMize/output/disk_utilization/

# get input
echo "Enter Group name: "
read gname

echo "Enter Current Date: "
read cdate
# format: 09/30/2016

# parse date
yyyy=$(echo ${cdate} | cut -d"/" -f3)
dd=$(echo ${cdate} | cut -d"/" -f2)
mm=$(echo ${cdate} | cut -d"/" -f1)

echo "Enter number of records: "
read nrec

# construct file name to parse
diskfile=${fpath}/disk_${yyyy}.${mm}.${dd}.txt

# check if file exists
if [ ! -f ${diskfile} ]
then
	echo File: ${diskfile} not found. Exiting.
	exit 1
fi

# generate output on screen
(echo "Filesystem	GB_blocks	Free	GB/%Used	%Iused	Mounted_on	Group	Date	Time" ; grep ${gname} ${diskfile} | \
awk -v gname=${gname} '{gsub(/_/," ",$7); print $1 " " $2 " " $3 " " $4 " " $5 " " $6 " " gname " " $7 " " $8}' | \
head -${nrec}) | column -t




Is the column command available at all on your system?

1 Like

As it is, i suppose there isn't:

# uname -a
AIX myhostname 1 7 00F7CA254C00

# oslevel -s
7100-03-04-1441

# which column
no column in /usr/bin /etc /usr/sbin /usr/ucb /usr/bin/X11 /sbin /usr/local/bin /usr/local/sbin ~/bin

As an aside, i suggest you write ksh (instead of bash ) scripts when it comes to AIX system maintenance, because the systems default shell is ksh. I have seen nothing in your scripts so far which would require bash .

I hope this helps.

bakunin

1 Like