Bash script issues

Shell Programming and Scripting
1

Hi.

The below part of my bash script kicks out the following error message:

[user@test01-stage-btv cmd]$ ./extract_eod_report_stats_new.sh 2010-04-23
./extract_eod_report_stats_new.sh: line 204: syntax error near unexpected token `('
./extract_eod_report_stats_new.sh: line 204: `TRANSACTIONS_RECEIVED_TOP=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep -v 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{if(x<$5) {x=$5;y=$3}} END {printf y}'`'

I have debugged the failing row (and the surrounding rows) for quite some time without being able to pinpoint the problem.

I would be very grateful for help.

Best regards,
Peter Lovell

...
TRANSACTIONS_RECEIVED=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{x+=$5} END {printf x}'`
TRANSACTIONS_PUBLISHED=`grep 'Transactions' $OUTPUT_FILE_PERF_TXD | grep 'AllFirms' | grep 'TXD1' | sed s/'"'/''/g | awk -F "," '{print $5}'`
TRANSACTIONS_PRICE_ERRORS=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{x+=$6} END {printf x}'`
TRANSACTIONS_QTY_ERRORS=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{x+=$8} END {printf x}'`
TRANSACTIONS_REJECTED=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{x+=$10} END {printf x}'`
TRANSACTIONS_CANCELLATIONS=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{z=$11+$12; x+=z} END {printf x}'`
TRANSACTIONS_AMENDMENTS=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{z=$13+$15;x+=z} END {printf x}'`

TRANSACTIONS_RECEIVED_TOP=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep -v 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{if(x<$5) {x=$5;y=$3}} END {printf y}'`

TRANSACTIONS_RECEIVED_TOP=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep -v 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," '{if(x<$5) x=$5} END {printf x}'`
TRANSACTIONS_RECEIVED_LOW=`grep 'Transactions' $OUTPUT_FILE_TRANSACTIONS | grep -v 'AllFirms' | grep "TX[24]" | sed s/'"'/''/g | awk -F "," 'BEGIN {x=999999999} {if(x>$5) {x=$5;y=$3}} END {printf y}'`
...
2

what are some of the lines above 204? maybe lines 194-204 would help as well.

3

I have attached the whole script.

The script might look slightly different from my earlier posting since I have edited names of variables.
The same error message still occur though.

4

IMHO the error is here:

I can reproduce the error message by including this code in a simple "case" statement. Just remove the excess brackets etc. which were probably intended to create a sub-shell for error messages?

None of the code in Post #1 appears in the script attached to Post #3 . There is no mention of TX[24] whether or not the variable names have changed. The error messages cannot be "the same".

Please post script and matching error messages from running that script.

For debugging consider running as "bash -x" or even commenting out lines from the bottom up until the error disappears.

Btw: The huge sed pipeline (30+ lines long) towards the end of the script will need attention. I can't see any line continuation characters "\" . Hopefully is it not one continuous wrapped line?

Btw: The main script structure reads the same files umpteen times. If these files are of any size it could be slow.

5

I get the error message even if I update the case clause. I also tried to comment out the whole case clause and the error message still appear:

[user@test01-stage-btv cmd]$ ./extract_eod_report_stats_new.sh 2010-04-23
./extract_eod_report_stats_new.sh: line 204: syntax error near unexpected token `('
./extract_eod_report_stats_new.sh: line 204: `TRADES_RECEIVED_TOP_CONTRIBUTOR_NAME=`grep 'Trades' $OUTPUT_FILE_TRADES | grep -v 'AllFirms' | grep "ME[24]" | sed s/\"/''/g | awk -F "," '{if(x<$5) {x=$5;y=$3}} END {printf y}'`'

Anymore ideas?

6

(Post rewritten). The sed statements to remove double quotes could be replaced with "tr" to make the script easier to read. This is not the main issue.

tr -d '"'

---------- Post updated at 18:46 ---------- Previous update was at 17:49 ----------

At last!

This line is decidedly suspect and turns half the script into a "sed" statement.

7

The script works again.
Many many thanks Methyl!!!
I also appreciate the other comments about my script, I will update it accordingly.
Cheers.