How do I strip and add tabbed fields to a long text file?

saint65 · February 20, 2007, 11:35am

I was wondering if there was a way to modify a tab delimited text file, up to 185,000 lines long by adding a repeated field to a block of 20 data , then repeat the process until the file is processed.

The current file looks like:

/////////////////////////////////////////////////////////////////////////
//File Name:  
//Test Date and Time: 
//
//Test Information
//
Customer ID:
Company ID: 
Project ID: T6010
Test ID: OFAT7
Test Type:Test
Sheet : DP600-GI 1.2
Material: D2 -5mm
t: MO
Tester: 
Test Date: 06-12-20
//--------------------------------------------------------------------------
// DataLength
// TotalStroke GroupStroke
// PullSpeed(m/s)
//
//    Time(s)  Tin(�)     TDie(�)    Tout(�)  Pull_Force(N)
//--------------------------------------------------------------------------
20
1  1
0.124
      0.00      19.53      16.11      19.78     273.68
      0.05      16.85      16.11      19.53    4707.32
      0.10      19.04      16.36      19.29    8046.24
      0.15      19.04      16.60      19.29    8319.92
      0.20      19.53      16.36      19.53    9359.91
      0.25      18.31      16.60      19.53    9907.28
      0.30      19.04      16.11      19.53   11275.68
      0.35      18.80      16.36      19.29   12315.67
      0.40      19.04      16.60      19.29   13081.98
      0.45      19.04      16.85      19.29   13793.55
      0.50      19.04      16.36      19.29    9578.86
      0.55      18.80      16.60      19.29      54.74
      0.60      19.04      16.85      19.29     109.47
      0.65      19.29      16.60      19.29     164.21
      0.70      18.80      16.85      19.29     164.21
      0.75      18.80      16.85      19.78      54.74
      0.80      18.80      17.33      19.29     164.21
      0.85      18.80      17.33      19.04     109.47
      0.90      19.04      17.33      19.29     164.21
      0.95      19.04      17.58      19.29     109.47
2  2
0.124
      1.95      20.02      19.78      20.26     164.21
      2.00      19.78      20.26      20.26   13903.03
      2.05      19.78      19.78      20.51   14231.45
      2.10      19.78      19.78      20.26   13848.29
      2.15      19.78      20.26      20.02   13684.08
      2.20      20.02      20.02      20.26   13629.35
      2.25      20.02      20.26      20.02   13684.08
      2.30      19.78      20.26      20.26   13629.35
      2.35      19.78      20.51      20.26   13848.29
      2.40      20.02      20.51      20.26   13629.35
      2.45      19.78      20.75      20.02   13574.61
      2.50      19.78      20.51      20.02     383.15
      2.55      19.78      20.75      20.51     218.95
      2.60      19.78      21.00      20.26     109.47
      2.65      19.53      21.00      20.26     328.42
      2.70      20.02      21.00      20.26     218.95
      2.75      19.78      21.48      20.02     218.95
      2.80      19.78      21.73      20.51     218.95
      2.85      19.78      21.97      20.26     218.95
      2.90      19.78      21.73      20.26     273.68
3  3
0.124
      3.88      19.53      23.19      19.78     164.21
      3.92      19.29      23.19      19.78   14286.18
      3.98      19.29      23.44      20.02   14012.50
      4.03      19.29      22.95      20.02   13848.29
      4.08      19.29      23.19      19.78   13684.08
      4.13      19.53      23.44      20.02   13848.29
      4.17      19.53      23.19      19.78   13684.08
      4.22      19.78      23.68      19.78   13684.08
      4.28      19.78      23.68      19.78   13684.08
      4.33      19.53      23.68      20.02   13684.08
      4.38      19.53      23.93      20.02   13519.87
      4.42      19.29      23.93      20.02     109.47
      4.47      19.29      23.93      19.78     218.95
      4.53      19.53      23.93      19.78     164.21
      4.58      19.53      24.17      19.53     164.21
      4.63      19.53      24.17      19.53     164.21
      4.67      19.53      24.66      19.78     164.21
      4.72      19.53      24.41      20.02     218.95
      4.78      19.53      24.17      20.26     164.21
      4.83      19.29      24.90      19.78     164.21
4  4
0.124
      5.80      18.55      24.90      19.04      54.74

I wish to reorganize the file to remove the top 27 lines and format like the following, remember that there are up to 9000 sample times 20 points of data:

//1  1
//0.124
1  1  0.124      0.00      19.53      16.11      19.78     273.68
1  1  0.124      0.05      16.85      16.11      19.53    4707.32
1  1  0.124      0.10      19.04      16.36      19.29    8046.24
1  1  0.124      0.15      19.04      16.60      19.29    8319.92
1  1  0.124      0.20      19.53      16.36      19.53    9359.91
1  1  0.124      0.25      18.31      16.60      19.53    9907.28
1  1  0.124      0.30      19.04      16.11      19.53   11275.68
1  1  0.124      0.35      18.80      16.36      19.29   12315.67
1  1  0.124      0.40      19.04      16.60      19.29   13081.98
1  1  0.124      0.45      19.04      16.85      19.29   13793.55
1  1  0.124      0.50      19.04      16.36      19.29    9578.86
1  1  0.124      0.55      18.80      16.60      19.29      54.74
1  1  0.124      0.60      19.04      16.85      19.29     109.47
1  1  0.124      0.65      19.29      16.60      19.29     164.21
1  1  0.124      0.70      18.80      16.85      19.29     164.21
1  1  0.124      0.75      18.80      16.85      19.78      54.74
1  1  0.124      0.80      18.80      17.33      19.29     164.21
1  1  0.124      0.85      18.80      17.33      19.04     109.47
1  1  0.124      0.90      19.04      17.33      19.29     164.21
1  1  0.124      0.95      19.04      17.58      19.29     109.47
//2  2
//0.124
2  2  0.124      1.95      20.02      19.78      20.26     164.21
2  2  0.124      2.00      19.78      20.26      20.26   13903.03
2  2  0.124      2.05      19.78      19.78      20.51   14231.45
2  2  0.124      2.10      19.78      19.78      20.26   13848.29
2  2  0.124      2.15      19.78      20.26      20.02   13684.08
2  2  0.124      2.20      20.02      20.02      20.26   13629.35
2  2  0.124      2.25      20.02      20.26      20.02   13684.08
2  2  0.124      2.30      19.78      20.26      20.26   13629.35
2  2  0.124      2.35      19.78      20.51      20.26   13848.29
2  2  0.124      2.40      20.02      20.51      20.26   13629.35
2  2  0.124      2.45      19.78      20.75      20.02   13574.61
2  2  0.124      2.50      19.78      20.51      20.02     383.15
2  2  0.124      2.55      19.78      20.75      20.51     218.95
2  2  0.124      2.60      19.78      21.00      20.26     109.47
2  2  0.124      2.65      19.53      21.00      20.26     328.42
2  2  0.124      2.70      20.02      21.00      20.26     218.95
2  2  0.124      2.75      19.78      21.48      20.02     218.95
2  2  0.124      2.80      19.78      21.73      20.51     218.95
2  2  0.124      2.85      19.78      21.97      20.26     218.95
2  2  0.124      2.90      19.78      21.73      20.26     273.68
//3  3
//0.124
3  3  0.124      3.88      19.53      23.19      19.78     164.21
3  3  0.124      3.92      19.29      23.19      19.78   14286.18
3  3  0.124      3.98      19.29      23.44      20.02   14012.50
3  3  0.124      4.03      19.29      22.95      20.02   13848.29
3  3  0.124      4.08      19.29      23.19      19.78   13684.08
3  3  0.124      4.13      19.53      23.44      20.02   13848.29
3  3  0.124      4.17      19.53      23.19      19.78   13684.08
3  3  0.124      4.22      19.78      23.68      19.78   13684.08
3  3  0.124      4.28      19.78      23.68      19.78   13684.08
3  3  0.124      4.33      19.53      23.68      20.02   13684.08
3  3  0.124      4.38      19.53      23.93      20.02   13519.87
3  3  0.124      4.42      19.29      23.93      20.02     109.47
3  3  0.124      4.47      19.29      23.93      19.78     218.95
3  3  0.124      4.53      19.53      23.93      19.78     164.21
3  3  0.124      4.58      19.53      24.17      19.53     164.21
3  3  0.124      4.63      19.53      24.17      19.53     164.21
3  3  0.124      4.67      19.53      24.66      19.78     164.21
3  3  0.124      4.72      19.53      24.41      20.02     218.95
3  3  0.124      4.78      19.53      24.17      20.26     164.21
3  3  0.124      4.83      19.29      24.90      19.78     164.21
//4  4
//0.124
4  4  0.124      5.80      18.55      24.90      19.04      54.74

Any suggestions of how I could approach this would be greatly appreciated. I just do not know where to start.

anbu23 · February 20, 2007, 11:56am

awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file

saint65 · February 20, 2007, 4:37pm

That works fantastically. Thanks so much, I tried

awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file

which dumped to screen, I added

awk '
NR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file > xyz

to put it into a file.

If i was to add the contents of a second data file that has to be added in sequence, how would i change the numbering?

The first digit (field) is the total count 1-9000 (file 1) the second digit (second field) is group count (just used but not totalled), I need to make 1-9000 (first field) of second data file go from 9001-18000 and so on.

I will be adding contents of file, file1, file2, ... file_n ---> xyz

anbu23 · February 21, 2007, 1:18am

awk '
FNR > 24 { 
if ( NF == 2 ) 
{ 
	a=$1;b=$2; print "//"$0 ; 
	getline c; print "//"c ; 
	getline;
}
print a "\t" b "\t" c "\t" $0 } ' file1 file2 file3  > xyz

If the format of the files are same, then you can use the above code