Error with expr - "expr: syntax error"

Hi All,

I'm writing a shell script in KSH, where I want to store the filename, total record count and actual record count of all the source files. The source files reside in 4 different sub-folders under the same root folder.

Below is code:

 
#!/usr/bin/ksh
common_path_in="/interface_in/rsc"
file_out="/interface_in/rsc/record_count.csv"
tot_rec_count=-1
act_rec_count=-1
echo ${common_path_in}
echo ${file_out}
echo ${tot_rec_count}
echo ${act_rec_count}
set -A folders horizon mysteryshopper rcc reference
echo ${folders[@]}
for i in ${folders[@]}
do
        ls -1 ${common_path_in}/$i/*.csv | xargs -n1 basename | while read filename
        do
                tot_rec_count=`wc -l ${common_path_in}/$i/*.csv | cut -f1 -d' '`
                echo ${tot_rec_count}
                act_rec_count=`expr ${tot_rec_count} - 1`
                echo ${act_rec_count}
                #echo "$filename,${tot_rec_count},${act_rec_count}" >> ${file_out}
        done
done

I'm getting output till the folder names, after that getting "expr: syntax error", for number of source files I've.

Surprisingly among these errors only once I'm getting correct count outputs for the case, where the source folder contains only single data file.

I've done the coding in vi editor and have not used MS editor.

Please suggest.

Regards,
Jagari

I doubt this works:

tot_rec_count=`wc -l ${common_path_in}/$i/*.csv | cut -f1 -d' '`

(cant test now...)

I would do it like this:

tot_rec_count=$(ls -1 $common_path_in/$i/*.csv|wc -l )
let act_rec_count=$tot_rec_count-1

Once your "wc -l" line found more than one file it output more than one line and a "total" line.
Assuming that you want to process file-by-file:

       ls -1 ${common_path_in}/$i/*.csv | while read filename
        do
                tot_rec_count=`wc -l "${filename}" | cut -f1 -d' '`

I wanted to add : The while loop puzzles me...
What is expected?

I dont get your logic: Once your are reading A (yes 1!) filename what is the point to go through all this strange calculation and no storing of $filename (???), I may be (quite sure..) very tired today but I am missing something there...

Hi Methyl,

Thanks, you identified the right point.
The problem was with -

 
tot_rec_count=`wc -l ${common_path_in}/$i/*.csv | cut -f1 -d' '`

It worked correctly with $filename as you suggested.
Then I've modified that portion as below -

 
ls -1 ${common_path_in}/$i/*.csv | xargs -n1 basename | while read filename
        do
            echo "${filename}"
            tot_rec_count=`wc -l "${common_path_in}/$i/${filename}" | cut -f1 -d' '`

It's now working properly.

But I've few doubts, would be great if you could answer -

1. The usage of curly braces {} and " " in displaying variable value - when we use which one?
2. use of back quote in value assignment to a variable and with expr command. For eg, the way I'm assigning tot_rec_count.

Thanks,
Jagari

1. The usage of curly braces {} and " " in displaying variable value - when we use which one?
   echo "${filename}"
   I have got into the habit of always putting quotes round string variables and strings because it prevents so many silly errors when a value contains spaces.
   I have also got into the habit of using curly braces even when they are not strictly necessary because it makes the variable name totally unambiguous.
   It also prevents errors like this:

# Didn't recognise variable
fred="abc"
echo $fred42
sh: fred42: Parameter not set.

# Spaces disappeared
fred="abc"
echo ${fred}42     displaced
abc42 displaced

# All working now. Spaces not lost
fred="abc"
echo "${fred}42     displaced"
abc42     displaced

2. use of back quote in value assignment to a variable and with expr command. For eg, the way I'm assigning tot_rec_count.

In ksh and Posix Shell whatever is between backticks is to be executed.
This more modern syntax is preferred. $((arithmetic)).

act_rec_count=$((${tot_rec_count} - 1))

Also, I wouldn't bother with an array.

This works the same and imho is easier to read and easier to type:

for i in "horizon" "mysteryshopper" "rcc" "reference"

Thanks it's really very helpful.

Regards,
Jagari

---------- Post updated at 10:25 AM ---------- Previous update was at 10:23 AM ----------

The while loop reads each filename and then counts the number of records in each of them. As the files have a header record, the count is decreased by 1 to get the actual number of records.