Scripting question

arun888 · September 4, 2015, 3:27am

hi all,

I am writing a script and beginner in shell scripting. I have tried the below script. could you please check and let me know whether the below scirpt is correct.

Unix details : HP Unix

Input file.

cat input.txt | tail -4
HTS40002.W1978.PROM
HTS40003.W1978.PROM
HTS40004.W1978.ADUS
HTS40003.W1978.PROM
HTS40004.W1978.ADUS

#!/bin/ksh
for i in `cat input.txt`;do
echo $i
if [ grep "PROM" $i ]
then 
echo "PROM FOUND"
elif [ grep "ADUS" $i ]
then 
echo "ADUS FOUND"
else
echo " NONE"
fi
done;

RavinderSingh13 · September 4, 2015, 3:40am

Hello Arun,

You need to check if word is present in complete file or not, following may help you in same then.

awk '{if($0 ~ /PROM/){A=1};if($0 ~ /ADUS/){B=1}} END{if(A){print "PROM found in file."} else {print "PROM NOT found."};if(B){print "ADUS found in file."} else {print "ADUS NOT found in file."};if(!A && !B){print "BOTH ADUS and PROM coulndnopt find in input file."}}'  Input_file

Following is output, I have used input_file as per your sample provided.

PROM found in file.
ADUS found in file.

On a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk .

Thanks,
R. Singh

Don_Cragun · September 4, 2015, 3:55am

You don't give enough details about what your script is supposed to do for anyone but you to say whether or not it is correct.

You should learn to indent your code so that the structure of your script is easy to see. Putting all of your code at the left margin makes it hard to see where loops start and end, where if statements start and end, etc.

Using:

for i in `cat file`
do	if ...
	then	echo ...
	elif ...
	then	echo ...
	fi
done

is almost always better written using:

while read -r i
do	...
done < file

If input.txt contains a list of filenames and you are trying to determine whether or not text in each of those files contain the string PROM and, if not, if each of those files contain the string ADUS , your script might be correct.

If you are trying to determine whether or not each line in input.txt ends with the string PROM or the string ADUS , that is NOT what your script does.

arun888 · September 4, 2015, 5:26am

hi ravinder,

Thanks for your help. Is there any possibilty to check using the grep statement.

---------- Post updated at 04:26 AM ---------- Previous update was at 02:59 AM ----------

I have tried for the below requirement. But i am not sure whether i am coding in right away. it would be great if you could help me on this.

Requirement is it should read the below text file and check the below path whether the file is exist or not.

Check :
PROM words is present in the input it should be checked in loki direcory whether the file is exist.
ADJUS words is present in the input it should be checked in loki direcory whether the file is exist.
 *PROM* /retail/market/loki/W1978 
 *ADUS* /retail/market/sali/W1978

Input file :

cat input.txt | tail -4
HTS40002.W1978.PROM
HTS40003.W1978.PROM
HTS40004.W1978.ADUS
HTS40003.W1978.PROM
HTS40004.W1978.ADUS

RudiC · September 4, 2015, 7:31am

I have to repeat what Don Cragun said: We cannot deduce what you need from what you say. Does the file "input.txt" contain file names to be found in a to-be-named directory? Should the files from "input.txt" in that unknown directory be checked for certain words? Which words? Or do you want to check the sheer existence of the file named in "input.txt"?

arun888 · September 4, 2015, 8:17am

I have the tried the below code for my requirment.

Need to read the file name from the input.txt and check whether the file exist in the below directory.

cat input.txt | tail -4
HTS40002.W1978.PROM
HTS40003.W1978.PROM
HTS40004.W1978.ADUS
HTS40003.W1978.PROM
HTS40004.W1978.ADUS
HTS40004.W1978.

#!/bin/ksh

grep 'PROM' projection.txt > prom.txt
grep 'ADUS' projection.txt >adus.txt
egrep -v 'PROM|ADUS' project.txt > none.txt
count=`cat project.txt | head -1 > pro.txt`
week=`awk '{print $3}' pro.txt`
input=w$week
echo $week

while read -r i
do
echo $i
if [ -s /retail/market/sali/$input/$i ]
then
echo " files is availbale"
else
echo " not available"
fi
done < prom.txt

while read -r i
do
echo $i
if [ -s /retail/market/sali/usmf/$input/$i ]
then
echo " files is availbale"
else
echo " not available"
fi
done < adus.txt

RavinderSingh13 · September 4, 2015, 9:17am

Hello aurn,

Following may help you in same.

###One liner form:
while read line; do ZERO=0; ls -l $line > /dev/null 2>&1; if [[ $? -eq $ZERO ]]; then echo "File named $line found."; else echo "File named $line NOT found."; fi; done < "Input_file"
  
OR
### NON one liner form:
while read line
do
    ZERO=0
    ls -l $line > /dev/null 2>&1
    if [[ $? -eq $ZERO ]]
    then
      echo "File named $line found."
    else
   echo "File named $line NOT found."
    fi
done < "check1213"

Output will be as follows.

File named HTS40002.W1978.PROM found.
File named HTS40003.W1978.PROM NOT found.
File named HTS40004.W1978.ADUS found.
File named HTS40003.W1978.PROp NOT found.
File named HTS40004.W1978.ADUS found.
File named HTS40004.W1978.PDnm NOT found.
File named HTS40002.W1978.PRjk NOT found.
File named HTS40003.W1978.PRyu NOT found.
File named HTS40004.W1978.ADdf NOT found.
File named HTS40003.W1978.PRqw found.
File named HTS40004.W1978.ADrt NOT found.
File named HTS40004.W1978.PDDT found.

NOTE: Input file used is as follows: Also I have placed script in same path where I need to look for files, if you want to setup script somewhere else then with ls command you can add complete path. Also I have created files named HTS40004.W1978.PDDT, HTS40003.W1978.PRqw, HTS40004.W1978.ADUS, HTS40002.W1978.PROM .

 cat check1213
HTS40002.W1978.PROM
HTS40003.W1978.PROM
HTS40004.W1978.ADUS
HTS40003.W1978.PROp
HTS40004.W1978.ADUS
HTS40004.W1978.PDnm
HTS40002.W1978.PRjk
HTS40003.W1978.PRyu
HTS40004.W1978.ADdf
HTS40003.W1978.PRqw
HTS40004.W1978.ADrt
HTS40004.W1978.PDDT

Thanks,
R. Singh

Don_Cragun · September 4, 2015, 2:06pm

arun888:

I have the tried the below code for my requirment.

Need to read the file name from the input.txt and check whether the file exist in the below directory.
cat input.txt | tail -4
HTS40002.W1978.PROM
HTS40003.W1978.PROM
HTS40004.W1978.ADUS
HTS40003.W1978.PROM
HTS40004.W1978.ADUS
HTS40004.W1978.

This is interesting... You have shown us six lines of output from a command that should produce no more than four lines of output.

And, you say you want to process a file named input.txt , but the code that follows never reads that file. Instead, it reads two other files named project.txt and projection.txt . And, with default fields separators, awk will see only one field in each of the above lines; but your code below is using awk to extract the 3rd field from the 1st line of the file named projection.txt . Does the 1st line in project.txt have a different format than the lines you have shown us above in input.txt ? If so, please show us the first few lines of input.txt instead of the last few lines.

Does the echo in your script print the value of $week that you were hoping to get? (If not; what does it print and what did you want it to print?)

arun888:

#!/bin/ksh

grep 'PROM' projection.txt > prom.txt
grep 'ADUS' projection.txt >adus.txt
egrep -v 'PROM|ADUS' project.txt > none.txt
count=`cat project.txt | head -1 > pro.txt`
week=`awk '{print $3}' pro.txt`
input=w$week
echo $week

while read -r i
do
echo $i
if [ -s /retail/market/sali/$input/$i ]
then
echo " files is availbale"
else
echo " not available"
fi
done < prom.txt

while read -r i
do
echo $i
if [ -s /retail/market/sali/usmf/$input/$i ]
then
echo " files is availbale"
else
echo " not available"
fi
done < adus.txt

You code reads from projection.txt and project.txt and produces the files prom.txt , adus.txt , none.txt , and pro.txt .

What is the relationship between the input.txt you showed us above and the projection.txt , and project.txt files read by your script?

When your script exits, which of the files it produces are needed for something else; and which are just temporary files only needed within the above script?

arun888 · September 7, 2015, 3:00am

hi ravi,

Thanks for your input for my requirement really it is useful and learned few things.

The file name mentioned in the input file, PROD file contrains in the directory.

*PROM* /retail/market/loki/

Other files are present in the /retail/market/sali directory.

please let me know how change the directory and check whether the file exist.

 
ls -l $line > /dev/null 2>&1

RavinderSingh13 · September 7, 2015, 3:13am

Hello Arun,

Following may help you in same, if you have any queries then please be more specific in your requirement with complete details so that we can help you. Here I am requesting user itself to enter the initial path.

echo "Enter the directory's path where files present. Example: /path/to/files"
read PA
 while read line
do
    ZERO=0
    ls -l $PA"/"$line > /dev/null 2>&1
    if [[ $? -eq $ZERO ]]
    then
      echo "File named $line found."
    else
   echo "File named $line NOT found."
    fi
done < "Input_file"

Thanks,
R. Singh

arun888 · September 7, 2015, 3:26am

hi ravi,
sorry for not giving requirement properly.
In the input file , there are two keywords present in the file , "PROD" and "ADUS".
Could you please let me know how o hard code the directory for the above scenario.

if PROM name is present in the file it should check in

/retail/market/loki/

If ADUS name is present in the file it should check in

/retail/market/salil

cat input.txt | tail -4
HTS40002.W1978.PROM
HTS40003.W1978.PROM
HTS40004.W1978.ADUS
HTS40003.W1978.PROM
HTS40004.W1978.ADUS

Don_Cragun · September 7, 2015, 3:52am

Perhaps something like:

#!/bin/ksh
ADUSf=adus.txt
ADUSp=/retail/market/salil
NONEf=none.txt
PROMf=prom.txt
PROMp=/retail/market/loki
rm -f "$ADUSf" "$NONEf" "$PROMf"
while IFS=. read head week tail
do      file="$head.$week.$tail"
        case "$tail" in
        (ADUS)  path="$ADUSp/$week/$file"
                printf '%s\n' "$file" >> "$ADUSf"
                ;;
        (PROM)  path="$PROMp/$week/$file"
                printf '%s\n' "$file" >> "$PROMf"
                ;;
        (*)     printf '%s\n' "$file" >> "$NONEf"
                continue
                ;;
        esac
        if [ -s "$path" ]
        then    printf '%s is available\n' "$file"
        else    printf '%s is not available\n' "$file"
        fi
done < input.txt

Note that -s tests for existence of a file with size greater than 0. If you just want to know if the file exists, change the:

        if [ -s "$path" ]

to:

        if [ -e "$path" ]

and if you want to know if it exists and is a regular file:

        if [ -f "$path" ]

arun888 · September 7, 2015, 5:32am

Hi Don Sir,

Really Thanks for your input. From your code I have learned lo of new things.

I have noticed the input file and it looks for "ADUS" the convention is different. I am not sure how to change the code.

cat input.txt | tail -4
HTS40002.W1978.PROM
HTS40003.W1978.PROM
HTS40004.W1978.SDLMF.ADUS
HTS40003.W1978.PROM
HTS40004.W1978.SDLMF.ADUS

Don_Cragun · September 7, 2015, 12:53pm

Hi arun888,
It is hard to keep up with your changing requirements.

First every input line has three fields separated by periods. Then there are three fields and sometimes the 3rd field is empty. And now, sometimes there are three fields (for PROM files) and sometimes there are four fields (for ADUS files).

Instead of showing us examples of lines from your input file, please describe in English the format of the filenames in that file:

How many (period separated) fields can there be in your filenames?
Is the 2nd field (the string between the 1st and 2nd periods in the filename) always the week number?
Is the last field always the field that contains ADUS or PROM (or none of the above)?
Can any field other than the last field be empty (i.e., in addition to HTS40007.W1978. could there be a filename like HTS40006.W1978..PROM )?

With a clear description of the input you're trying to process, we can write code to process it. Without a clear description, we'll never know if we have code that can cope with a filename format we haven't seen before.

Please help us help you.

Aia · September 7, 2015, 1:19pm

arun888:

Hi Don Sir,

Really Thanks for your input. From your code I have learned lo of new things.

I have noticed the input file and it looks for "ADUS" the convention is different. I am not sure how to change the code.
cat input.txt | tail -4
HTS40002.W1978.PROM
HTS40003.W1978.PROM
HTS40004.W1978.SDLMF.ADUS
HTS40003.W1978.PROM
HTS40004.W1978.SDLMF.ADUS

Hi arun888
If you do not mind to learn some more
You do not need to use the command cat there; the command tail can do the same by itself

tail -4 input.txt

By the way, there should be only four (4) lines in your output, instead of five (5).

arun888 · September 8, 2015, 2:33am

Hi don,

Please find my exact requirement and answers for your questions.

Instead of showing us examples of lines from your input file, please describe in English the format of the filenames in that file:
User will be provided the file names in the input.txt file and the file name with "ADUS" need to be check whether it is exist in /retail/market/salil directory and the file name with "PROM" need to be checked whether it is present in /retail/market/loki directory.

tail -6 input.txt
INFOPRC.W1078.V018.ADUS
INFOPRC.W1078.V019.ADUS
INFOPRC.W1078.V118.ADJUS
INFOPRC.PJOMGPJ.HTC40004.W1078.PROM
INFOPRC.POMGPJ.HTS40001.W1078.PROM
INFOPRC.PJMGPJ.HTS40002.W1078.PROM

How many (period separated) fields can there be in your filenames?

5 field separtor in PROM file and 4 field separator in ADUS file.
Is the 2nd field (the string between the 1st and 2nd periods in the filename) always the week number?
In PROD and ADUS file it differs.
Is the last field always the field that contains ADUS or PROM (or none of the above)?
yes last file contains ADUS or PROM.
Can any field other than the last field be empty (i.e., in addition to HTS40007.W1978.
Last field might be empty sometimes.
could there be a filename like HTS40006.W1978..PROM )?
no i have wrongly provided.

Don_Cragun · September 8, 2015, 4:15pm

Assuming you still want to look for the file in a subdirectory of the directories you named for the ADUS files and the PROM files corresponding to the week number named in the file you're processing, perhaps something like:

#!/bin/ksh
#set -xv	# Uncomment this line to enable tracing.
ADUSf=adus.txt
ADUSp=/retail/market/salil
INPUTf="${1:-input.txt}"
NONEf=none.txt
PROMf=prom.txt
PROMp=/retail/market/loki
rm -f "$ADUSf" "$NONEf" "$PROMf"
OIFS="$IFS"
IFS=.
while read -r file
do	set -- $file
	if [ $# -eq 4 ] && [ "$4" = "ADUS" ]
	then	week="$2"
		path="$ADUSp/$week/$file"
		printf '%s\n' "$file" >> "$ADUSf"
	elif [ $# -eq 5 ] && [ "$5" = "PROM" ]
	then	week="$4"
		path="$PROMp/$week/$file"
		printf '%s\n' "$file" >> "$PROMf"
	else	printf '%s\n' "$file" >> "$NONEf"
		continue
	fi
	if [ -s "$path" ]
	then	printf '%s is available\n' "$file"
	else	printf '%s is not available\n' "$file"
	fi
done < "$INPUTf"
IFS="$OIFS"

would work. Note that if there isn't any other code you want to add to the end of this script, there is no need to save the old value of $IFS before setting it to . and no need to reset it to its prior value after the while loop completes.

arun888 · September 10, 2015, 8:51am

Hi Dan,

Thanks for your help