Unable to do grep in a script

Hi,

I am trying to grep a filename from a script after taking the file name and other variables as keyboard input .When I run the grep command with the same filename on the prompt, it runs fine, but it is either not giving me the correct output or not running at all from the script using the same grep command.

from the prompt:

$/bin/grep 20170219_HMS_80Byte_PAYbig3_1_4.prc /backup/omnftppasrv/bw3/201702/bw3-syst_INST19-201702[0-9][0-9].lst

pasting snippet of my script code

#building regex for date searching
regex=$yr_mnth\[0-9\]\[0-9\]

#building date searching regex list file
xtr_regex=$xtr_flptrn1$inst_nmb-$regex.$list_file

cmptl_extr_list_file=$xfile_pth/$xtr_regex	#storing complete file name with path into a variable

#echo $cmptl_extr_list_file

#grepping the filename from list file
/bin/grep '$flnm_80BYTE' $cmptl_extr_list_file

experts need your advice on this please

This is quite sparse a specification which doesn't necessarily encourage people to offer you help. At first sight, the single quotes prevent the $flnm_80BYTE from being expanded.

sorry about that, totally missed mentioning I am on AIX and I had tried it first with double quotes and with no luck, I just tried it with single quotes, but no luck again.

Please let me know any other info required and Ill promptly reply

#!/bin/ksh
set -x
#building regex for date searching
regex="${yr_mnth}[0-9][0-9]"
# where's $yr_mnt defined?

#building date searching regex list file
xtr_regex="${xtr_flptrn1}${inst_nmb}-${regex}.${list_file}"
#where are xtr_flptrn1  nst_nmb list_file defined?
# placing {} for possible variable names - validate

 cmptl_extr_list_file="${xfile_pth}/${xtr_regex}"  #storing complete file name with path into a variable

#echo $cmptl_extr_list_file

#grepping the filename from list file
/bin/grep '$flnm_80BYTE' $cmptl_extr_list_file
# as noted don't use single quotes - they will hide the variable expansion 
# /bin/grep "${flnm_80BYTE}" ${cmptl_extr_list_file}"

edited code (edits/questions in red) - for possible variable name spec changes - validate the placement of {}'s
Enabled set -x to aid in debugging

Hi, I am pasting the complete code meanwhile I am editing the script accoding to vgersh99's advise. Hope, I am able to make some sense. Please let me know if you need any other information

#!/bin/bash

#static value assignment to a variable

list_file=lst	   #taking a variable into extraction list file

enc_tar_xtnsn=tar.gz.enc	#taking a variable into extraction encrypted file 

xtr_flptrn1=bw3-syst_INST 	#taking a variable into extraction file path

#main_scp=/usr/local/scripts/omniadm/omnftppasrv/extract.sh  #main extraction script

echo "Enter file processing date [YYYYMMDD]"
read file_x_date       

echo "Inst nu:"
read inst_nmb      

#breaking 80byte_flnm into YYYYMM and DD

yr_mnth=`echo "${file_x_date}"|awk '{print substr($0,1,6)}'`

only_dt=`echo "${file_x_date}"|awk '{print substr($0,7,8)}'`

#variable value assigned dynamically based on static value assignemnt above

xtr_flnm_list="${xtr_flptrn1}""${inst_nmb}"-"{$file_x_date}"\."{$list_file}"    #taking a variable into extraction file name

#---echo $xtr_flnm_list

xfile_pth=/backup/omnftppasrv/bw3/"${yr_mnth}"  # storing extraction file path into a variable

#---echo $xfile_pth

enc_tar_filename="${xtr_flptrn1}""${inst_nmb}"-"${file_x_date}"\."${enc_tar_xtnsn}"

#---echo $enc_tar_filename

#cmptl_extr_list_file=$xfile_pth/$xtr_flnm_list	#storing complete file name with path into a variable

#echo $cmptl_extr_list_file

cmptl_extr_enc_file="${xfile_pth}"/"${enc_tar_filename}"

echo $cmptl_extr_enc_file

echo "filename to extract"
read flnm_80byte    

	if [[ "$flnm_80byte" =~ .*\.prc ]]
	then
		echo "correct file name"
	elif [[ "$flnm_80byte" =~ .*\.inc ]]
	  then
		  strppd_flname=`echo $flnm_80byte|awk '{print substr($0, 1, length($0) - 4)}'`
		  flnm_80BYTE=$strppd_flname\.prc
		  echo $flnm_80BYTE
	  else
		  flnm_80BYTE=$flnm_80byte\.prc
		  echo $flnm_80BYTE
	  
	fi

#building regex for date searching
regex="{$yr_mnth}"\[0-9\]\[0-9\]

#building date searching regex list file
xtr_regex="${xtr_flptrn1}${inst_nmb}-${regex}\.${list_file}"

cmptl_extr_list_file="${xfile_pth}"/"${xtr_regex}"	#storing complete file name with path into a variable

echo $cmptl_extr_list_file

#grepping the filename from list file
/bin/grep "${flnm_80BYTE}" "${cmptl_extr_list_file}"

add set -x at the beginning of the script and see the output of the execution - follow the script's variable expansion and the execution path.

Hi, I have updated the script to ksh and added the set -x. Thanks a lot for that. However still facing errrors, as expected :). I have pasted the output also

your advice please

#!/bin/ksh

set -x

#static value assignment to a variable

list_file=lst	   #taking a variable into extraction list file

enc_tar_xtnsn=tar.gz.enc	#taking a variable into extraction encrypted file 

xtr_flptrn1=bw3-syst_INST 	#taking a variable into extraction file path

#main_scp=/usr/local/scripts/omniadm/omnftppasrv/extract.sh  #main extraction script

echo "Enter file processing date [YYYYMMDD]"
read file_x_date       

echo "Inst nu:"
read inst_nmb      

#breaking 80byte_flnm into YYYYMM and DD

yr_mnth=`echo "${file_x_date}"|awk '{print substr($0,1,6)}'`

only_dt=`echo "${file_x_date}"|awk '{print substr($0,7,8)}'`

#variable value assigned dynamically based on static value assignemnt above

xtr_flnm_list="${xtr_flptrn1}""${inst_nmb}"-"{$file_x_date}"."{$list_file}"    #taking a variable into extraction file name

#---echo $xtr_flnm_list

xfile_pth=/backup/omnftppasrv/bw3/"${yr_mnth}"  # storing extraction file path into a variable

#---echo $xfile_pth

enc_tar_filename="${xtr_flptrn1}""${inst_nmb}"-"${file_x_date}"."${enc_tar_xtnsn}"

#---echo $enc_tar_filename

#cmptl_extr_list_file=$xfile_pth/$xtr_flnm_list	#storing complete file name with path into a variable

#echo $cmptl_extr_list_file

cmptl_extr_enc_file="${xfile_pth}"/"${enc_tar_filename}"

echo $cmptl_extr_enc_file

echo "filename to extract"
read flnm_80byte    

	if [[ "$flnm_80byte" =~ .*\.prc ]]
	then
		echo "correct file name"
	elif [[ "$flnm_80byte" =~ .*\.inc ]]
	  then
		  strppd_flname=`echo $flnm_80byte|awk '{print substr($0, 1, length($0) - 4)}'`
		  flnm_80BYTE=$strppd_flname\.prc
		  echo $flnm_80BYTE
	  else
		  flnm_80BYTE=$flnm_80byte\.prc
		  echo $flnm_80BYTE
	  
	fi

#building regex for date searching
regex="{$yr_mnth}"\[0-9\]\[0-9\]

#building date searching regex list file
xtr_regex="${xtr_flptrn1}${inst_nmb}-${regex}\.${list_file}"

cmptl_extr_list_file="${xfile_pth}"/"${xtr_regex}"	#storing complete file name with path into a variable

echo $cmptl_extr_list_file

#grepping the filename from list file
/bin/grep "${flnm_80BYTE}" "${cmptl_extr_list_file}"
./extract80byte.sh
+ list_file=lst
+ enc_tar_xtnsn=tar.gz.enc
+ xtr_flptrn1=bw3-syst_INST
+ echo Enter file processing date [YYYYMMDD]
Enter file processing date [YYYYMMDD]
+ read file_x_date
20170219
+ echo Inst nu:
Inst nu:
+ read inst_nmb
19
+ + awk {print substr($0,1,6)}
+ echo 20170219
yr_mnth=201702
+ + echo 20170219
+ awk {print substr($0,7,8)}
only_dt=19
+ xtr_flnm_list=bw3-syst_INST19-{20170219}.{lst}
+ xfile_pth=/backup/omnftppasrv/bw3/201702
+ enc_tar_filename=bw3-syst_INST19-20170219.tar.gz.enc
+ cmptl_extr_enc_file=/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170219.tar.gz.enc
+ echo /backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170219.tar.gz.enc
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170219.tar.gz.enc
+ echo filename to extract
filename to extract
+ read flnm_80byte
20170216_HMS_80Byte_PAYbig3_1_6.inc
./extract80byte.sh[52]: syntax error at line 52 : `=~' unexpected

one potential issue that I can see:

xtr_flnm_list="${xtr_flptrn1}""${inst_nmb}"-"{$file_x_date}"."{$list_file}"

should be

xtr_flnm_list="${xtr_flptrn1}${inst_nmb}-${file_x_date}.${list_file}"

misplaced {-s.
You don't need extra double-quotes in blue
Edit the script and continue debugging looking at the output of tracing and making modifications...

thanks a lot for pointing out the mistakes. also I have changed the script to bash again cos it seems the regex pattern matching is not avaiable in ksh ...continuing to edit, will let you know

---------- Post updated at 03:15 PM ---------- Previous update was at 02:47 PM ----------

Hi, OK, iv edited the script a bit --just changed the places of the statements and some formatting, but it kind of gets stuck coming to the grep function :(. Kind of lost as to how to move forward, cos iv been stuck here before with not much clue

#!/bin/bash

set -x

#static value assignment to a variable

list_file=lst	   #taking a variable into extraction list file

enc_tar_xtnsn=tar.gz.enc	#taking a variable into extraction encrypted file 

xtr_flptrn1=bw3-syst_INST 	#taking a variable into extraction file path

#main_scp=/usr/local/scripts/omniadm/omnftppasrv/extract.sh  #main extraction script

echo "Enter file processing date [YYYYMMDD]"
read file_x_date       

echo "Inst nu:"
read inst_nmb      

#breaking 80byte_flnm into YYYYMM and DD

yr_mnth=`echo "${file_x_date}"|awk '{print substr($0,1,6)}'`

only_dt=`echo "${file_x_date}"|awk '{print substr($0,7,8)}'`

#variable value assigned dynamically based on static value assignemnt above

xtr_flnm_list="${xtr_flptrn1}${inst_nmb}-${file_x_date}.${list_file}"    #taking a variable into extraction file name

#---echo $xtr_flnm_list

xfile_pth=/backup/omnftppasrv/bw3/"${yr_mnth}"  # storing extraction file path into a variable

#---echo $xfile_pth

enc_tar_filename="${xtr_flptrn1}${inst_nmb}-${file_x_date}.${enc_tar_xtnsn}"

#---echo $enc_tar_filename

#cmptl_extr_list_file=$xfile_pth/$xtr_flnm_list	#storing complete file name with path into a variable

#echo $cmptl_extr_list_file

cmptl_extr_enc_file="${xfile_pth}/${enc_tar_filename}"

echo $cmptl_extr_enc_file

func_xtract_file()
{
#building regex for date searching
regex="$yr_mnth"\[0-9\]\[0-9\]

#building date searching regex list file
xtr_regex="${xtr_flptrn1}${inst_nmb}-${regex}.${list_file}"

cmptl_extr_list_file="${xfile_pth}/${xtr_regex}"	#storing complete file name with path into a variable

#echo $cmptl_extr_list_file

#grepping the filename from list file
/bin/grep "$flnm_80BYTE" "${cmptl_extr_list_file}"
}

echo "filename to extract"
read flnm_80byte    

	if [[ "$flnm_80byte" =~ .*\.prc ]]
	then
		echo "correct file name"
		#assigning variable for grep pattern searching
		flnm_80BYTE=$flnm_80byte
		echo "calling func_xtract_file"
		func_xtract_file
	elif [[ "$flnm_80byte" =~ .*\.inc ]]
	  then
		  strppd_flname=`echo $flnm_80byte|awk '{print substr($0, 1, length($0) - 4)}'`
		  flnm_80BYTE=$strppd_flname\.prc
		  echo $flnm_80BYTE
		  echo "calling func_xtract_file"
		  func_xtract_file
	  else
		  flnm_80BYTE=$flnm_80byte\.prc
		  echo $flnm_80BYTE
		  echo "calling func_xtract_file"
		  func_xtract_file
	fi
#script ends here

what's output of set -x ?
you need to make sure that all the vars get expanded as expected and each command/function does what you'd expect...

Hi, sorry forgot to paste the output last time around. variables are getting expanded, just that when expansion for the path needs to occur in the grep command, things are not working out. I tried do to ls manually and the path is there.

dsiddiq:P7_LIVE:omnftppa:/home/dsiddiq> ./extract80byte.sh
+ list_file=lst
+ enc_tar_xtnsn=tar.gz.enc
+ xtr_flptrn1=bw3-syst_INST
+ echo 'Enter file processing date [YYYYMMDD]'
Enter file processing date [YYYYMMDD]
+ read file_x_date
20170219
+ echo 'Inst nu:'
Inst nu:
+ read inst_nmb
19
++ echo 20170219
++ awk '{print substr($0,1,6)}'
+ yr_mnth=201702
++ echo 20170219
++ awk '{print substr($0,7,8)}'
+ only_dt=19
+ xtr_flnm_list=bw3-syst_INST19-20170219.lst
+ xfile_pth=/backup/omnftppasrv/bw3/201702
+ enc_tar_filename=bw3-syst_INST19-20170219.tar.gz.enc
+ cmptl_extr_enc_file=/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170219.tar.gz.enc
+ echo /backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170219.tar.gz.enc
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170219.tar.gz.enc
+ echo 'filename to extract'
filename to extract
+ read flnm_80byte
20170219_HMS_80Byte_PAYbig3_1_4.prc
+ [[ 20170219_HMS_80Byte_PAYbig3_1_4.prc =~ .*\.prc ]]
+ echo 'correct file name'
correct file name
+ flnm_80BYTE=20170219_HMS_80Byte_PAYbig3_1_4.prc
+ echo 'calling func_xtract_file'
calling func_xtract_file
+ func_xtract_file
+ regex='201702[0-9][0-9]'
+ xtr_regex='bw3-syst_INST19-201702[0-9][0-9].lst'
+ cmptl_extr_list_file='/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-201702[0-9][0-9].lst'
+ /bin/grep 20170219_HMS_80Byte_PAYbig3_1_4.prc '/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-201702[0-9][0-9].lst'
grep: can't open /backup/omnftppasrv/bw3/201702/bw3-syst_INST19-201702[0-9][0-9].lst
dsiddiq:P7_LIVE:omnftppa:/home/dsiddiq>ls /backup/omnftppasrv/bw3/201702/bw3-syst_INST19-201702[0-9][0-9].lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170201.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170202.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170203.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170204.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170205.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170206.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170207.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170208.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170209.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170210.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170211.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170212.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170213.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170214.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170215.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170216.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170217.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170218.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170219.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170220.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170221.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170222.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170223.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170224.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170225.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170226.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170227.lst
/backup/omnftppasrv/bw3/201702/bw3-syst_INST19-20170228.lst

not to be overly simplistic but are you sure your path to grep is /bin/grep?

what's the output of

which grep

loose the double quotes - your cmptl_extr_list_file contains the file globbing that needs to be expanded by shell :

/bin/grep "$flnm_80BYTE" ${cmptl_extr_list_file}
1 Like

@os2mac, grep is fine, no issues there, but thanks

@vgersh99: thanks a lot, i found it also, but thanks thanks a lot for your patience. I have been banging my head on this for the last 3-4 hours. had it not been for your pointing out, i wouldn't have been able to figure this out. for now at least a small part is fixed, i need to include other functions also, but phew this grep part had really got me. thanks a lot..

/bin/grep $flnm_80BYTE ${cmptl_extr_list_file}

..did the trick for me

1 Like