Extraction of data from multiple text files, and creation of a chart

Hello dear friends,

My problem as explained below seems really basic.
Fact is that I'm totally new to programming, and have only a week to produce a script ( CShell or Perl ? ) to perform this action.

While searching on the forums, I found a command that could help me, but I don't know how to use it properly :s.

But first of all . . . the context.
I have 7300 text files, each containing the same type of information.
Below is shown how the text files are written :

3 PARAMETERS ON 05JAN01.
1 AZU1 STA X -0.247297943318988E+07 +- 0.241394876950605E-02
2 AZU1 STA Y -0.467133800760033E+07 +- 0.374164031593362E-02
3 AZU1 STA Z 0.355810775092403E+07 +- 0.257172727253398E-02
2 1 0.571958818322482E+00
3 1 -0.659378461245129E+00
3 2 -0.847505892381651E+00
AZU1 ANTENNA LC 0.1636 0.0000 0.0000 !up north east (m)

In order to analyze all the data, I need to extract the bold values, and put each one in a different column of an Excel file (or equivalent) or in a text file ( but then with the three values on a line, and maybe try to create a csv file ).

To finish, I have a file like this in a different folder . . . . and there's 3 or 4 levels of difference in the arborescence. As :

desktop/data_2005/05JAN01/out/solution/05JAN01AZU1_fix.txt
desktop/data_2005/05JAN02/out/solution/05JAN02AZU1_fix.txt
and it goes on for the whole year . . .

The idea would be to use the 'grep' command I guess :
I tried with this "egrep "1|2|3" < 05JAN01AZU1_fix.stacov.txt | awk '{ print $5 }'>> output.txt" but it doesn't work properly. It did output cleanly the 3 values that I want when I changed the "1|2|3" to "10|20|30" and did the changed the values 1,2,3 to 10,20,30before the three values I wanted to extract.

So, basically . . . I have a text file for each day of the year, containing three values that I want to put in three separate columns of an excel file, or in the same line of the csv file.

I guess that I have to find the way to extract correctly my values, but also that I have to write a kind of loop, to go through all the folders(days) of the year.

CShell or Perl ?

Does anyone has one or more ideas to help me go further ?
If you do, I'll be pleased if you drop it there :wink:
Thanks . . .

P.S :I don't know who wrote the rules of the forum . . . the point n�15 "No smoking in the forums" made me smile . . . :smiley:

Don't use cshell - sounds like you are in university... Do you have bash or ksh or even Bourne? There are very valid reasons for avoiding csh.

Here is a non- Cshell answer

find /desktop/data_2005  -name '*AZU1_fix.txt' |
while read filename
do
     awk '{ printf("%s ", $5)} END {printf("\n")} ' $filename
done | tr -s ' ' ',' > data_2005.csv

Thank you very much Jim, it does work perfectly :wink:
I still have to search to understand fully how it works . . .:slight_smile: I mean the syntax and everything . . . I would like to output the dates e.g. "05JAN01" in the first column aswell . . .

I do notice a difference between the values of the txt files and the values in the csv . . .
For instance when the first value is " -0.247297943318988E+07" (see the exemple in my first post ), it becomes "-2472979.90616501" (decimal part, it seems). I don't know yet if I actually need that precision . . . I'm checking this.

Any ideas ?

Your code will help me A LOT . . . Thank you again !
Ackheron.