Wow I'm so zoned out I don't even know if I posted this question up already (I couldn't find it in my book marks or in "yesterday's" post).
My question is, I'm writing a korn script that does something like the following, but I don't yet completely understand the syntax. I need to check that the first field equals a string and that the second string equals a space. This isn't working -- is it right or wrong?
FIELD1=`cut -c2-5 $FILENAME`
FIELD5=`cut -c68 $FILENAME`
if [ $FIELD1 -eq 'ABC' && $FIELD5 -eq ' ' ]; then
$count++
fi
Ah yes, of course. Let me type out the larger picture so you can see how it's working.
The ls.txt is a cut listing of filenames (one filename per line).
As I'm opening each file, which happen to be all fixed-length csv files, I'm cutting the first field and the fifth field.
The following would be an example of two records (each line in each fixed-length csv file is a record with aproximately 32 fields). I have isolated the two fields for you.
Example fields (the forum deletes extra spaces so it would be easier I think if I describe it instead: FIELD1 is in columns 2-4 (a string of 3 letters), and the the second column aka FIELD5 can either be filled with a one digit number (1-9) or a single space (" ").
while read FILENAME; do
FIELD1=`cut -c2-5 $FILENAME`
FIELD5=`cut -c68 $FILENAME`
if [ $FIELD1 -eq "PBR" && $FIELD5 -eq " " ]; then
$count++
fi
done < ls.txt
I'm kind of thinking that maybe it's not working because... if you try to use the "cut" command and it cuts a single space, maybe the shell doesn't assign anything to the variable?
Question: I just started learning shell scripting a few days ago and I was under the assumption that you didn't need to typeset anything.
One of my previous counters looked like this (in a while statement, not an if statement:
var1=`expr $var1 + $var2`
..and it worked.
But when I tried the expr command in the if statement, it didn't increment at all. Would you know why this is? Could it be because once you enter an if statement, you must use different rules?
this NOT using ksh/bash builtin capabilities for doing simple integer math.
nope, that's not it.
are you saying that the 'expr ' increment did not work, but the 'count=$((count+1))' did?
there must be something else different in your 'while' loop.
is your 'while' loopbeing piped with data?
I think it's because I'm doing something terribly wrong with my nested while statement.. Sorry I didn't mention it before, I didn't think it was the fault.
Here's the big picture.
while read FILENAME; do #get filename in file_names.txt
while read RECORD; do #read/open each filename
. . . (This is where I'm putting the if statement.)
done < $FILENAME
done < file_names.txt
I have a while statement inside of another while statement because the first while reads the filenames I need from file_names.txt. I then need to use the 2nd while statement to begin reading each filename.
edit: Hmm.... You know what, let me work on this for another hour and come back and tell you if it's still broken.
hmm.. I stripped all the code, essentially leaving just:
while read RECORD; do
echo "RECORD->[${RECORD}]"
done < ${PIRFE_CSV_PATH}PEURO03496.1118769614122
There are other, similar files, I tried to put them into the read loop as well.
For some reason, the shell script hangs when it gets to the while loop for these files. I think the problem may be with the files themselves, not the code.. The files can't be too large, only about 100-300 lines, each with 32 "fields"..
All the variables are working as they should. System just hangs when it gets to the while. I'm still trying to figure out why it hangs... I'll keep this post updated. It's just a typical CSV file with each line being one record.
I don't think it's permissions, all users have at least read permissions:
-rwxr--r--
I'm certain it's a file, and not a directory since I can view it in my text editor as a regular csv file.
In "ls -al" there's no "->" link to somewhere else.
From the same directory, I can run commmands like:
cat PEURO03497.1118773120982
and
more PEURO03497.1118773120982
and it opens fine.
This is a long shot but let me try renaming the file.
run this and see where it gets stuck.... change the name/path of 'file_names.txt' as needed.
#!/bin/ksh
FIELD1='PBR'
FIELD2=' '
#typeset -i count
while read FILENAME; do
while read RECORD; do
if [ "$FIELD1" = 'PBR' -a "$FIELD2" = ' ' ]; then
count=$(($count + 1))
echo "file->[$FILENAME] count->[${count}]"
fi
done < $FILENAME
done < file_names.txt
I enabled "set -vx" so we can see what's going on as it's running... here's the output. (note: i simplified the ls command output..)
%ls -la
total 560
-rw-r--r-- PEURO03497.1118773120982
-rw-r--r-- PUSA05496.1118776734108
-rw-r--r-- file_names.txt
-rwxr-xr-x test.sh
% test.sh
FIELD1='PBR'
+ FIELD1=PBR
FIELD2=' '
+ FIELD2=
#typeset -i count
while read FILENAME; do
while read RECORD; do
if [ "$FIELD1" = 'PBR' -a "$FIELD2" = ' ' ]; then
count=$(($count + 1))
echo "file->[$FILENAME] count->[${count}]"
fi
done < $FILENAME
done < file_names.txt+ 0< file_names.txt
+ read FILENAME
+
This is where it gets stuck/hangs. Then I have to use kill command to terminate.
The first filename in file_names.txt is "PEURO03497.1118773120982"
I only put two file names into that txt file.. and I copied both files into the same dir as the script. I also tried switching the order in the txt file, same result. The shell let's me do more and cat on them just fine.
I think we've isolated everything except the file contents... perhaps the actual csv data is conflicting in some way.
The files are all CSV. Each line is a record.
Each record, or line, contains 32 fixed-length fields separated by commas.
Fields can be left blank (filled with spaces).
Every record follows the same format.
Since the post doesn't allow multiple spaces, I'll try to stick it into HTML tags to see if it will preserve the spacing.
That's it.. I vi'ed it to make sure there weren't any hidden control characters.
They are the filenames of the files that need to be opened and fed through the while loop.
edit:: You know what....... it's probably not the code... something must be wrong or different about these particular files...
If and when I find out, I will return here and tell you what was wrong.