Commands to reorganize a text file

Shell Programming and Scripting
1

Hi!

I am trying to create a script to reorder the contents of a text file. Below is the text file initially, followed by how I would like it reordered:

File initially:
---
Initial lines with text and/or numbers
Initial lines with text and/or numbers
Initial lines with text and/or numbers
Initial lines with text and/or numbers
Initial lines with text and/or numbers

x1 y1 z1 num1
1 E1,1
2 E1,2
3 E1,3
4 E1,4
5 E1,5
6 E1,6
...
{note to forum: there is always one linespace in between}
x2 y2 z2 num2
1 E2,1
2 E2,2
3 E2,3
4 E2,4
5 E2,5
6 E2,6
...

(The above sections are repeated hundreds of times.)

File reordered:
---
x y z 1 2 ...
x1 y1 z1 E1,1 E1,2 ...
x2 y2 z2 E2,1 E2,2 ...
...

I hope this makes sense. Essentially I need to be able to generate an Excel plot of x, y and z versus the energies. Other notes:

  1. x1, y1, z1, E1,1 etc. are all numbers (decimals and floating point).
  2. The '1', '2', '3', ... numbers above are exactly what appears.

Any help that can be provided would be great! I have attached a copy of the real initial file.

Thanks!
gwr

2 
awk 'A&&NF!=2{A=0} A{printf A" ";L=1;A=x} /^$/&&L{print;L=0} L{printf " "$2","$1} NF==4{A=$1" "$2" "$3" "$4}' eigenval
3

Thanks Chubler_XL!

When I execute that though as follows, I get the attached output file which I can't plot.

#!/bin/bash

var1=$(awk 'A&&NF!=2 {A=0} A{ printf A" "; L=1; A=x} /^$/&&L { print; L=0 } L{ printf " "$2","$1} NF==4 {A=$1" "$2" "$3" "$4}' EIGENVAL)

echo ${var1} >> out.txt

exit

Am I doing something wrong?

Thanks again,
gwr

4

Yes, just redirect output directly to your out.txt:

#!/bin/bash
awk 'A&&NF!=2{A=0} A{printf A" ";L=1;A=x} /^$/&&L{print;L=0} L{printf " "$2","$1} NF==4{A=$1" "$2" "$3" "$4}' eigenval > out.txt
5

Thanks Chubler_XL! Works a treat.

gwr

6

Hi Chubler_XL,

One further question. Attached is the Excel file I get when I import the resulting text file, delimiting it by space and comma. Notice the columns with all 1's, 2's, 3's, etc. Is there anyway to get rid of these columns? It's a real pain for plotting. Preferably, I would like these numbers to appear once above their respective energy columns as headings.

Thanks again!

gwr

7

Yep this should do it - just using comma as delimiter now:

awk '
NR==FNR{if(Z&&NF!=2)Z=0;if(Z)C=C>$1?C:$1;if(NF==4)Z=1;next}
C{printf "X,Y,Z,GWR,";while(--C)printf ++i",";print ++i}
A&&NF!=2 {A=0} A{ printf A; L=1; A=x} /^$/&&L{print;L=0}
L{printf ","$2} NF==4{A=$1","$2","$3","$4}' eigenval eigenval > out.txt
8

Thanks Chubler_XL, that's perfect.

Cheers,
gwr

9

Chubler_XL,

Can you help me? The commands you created were working great for the first file attached, until I encountered the second file attached, which doesn't work. The files should be the same, and I can't see a difference. But in the output from the awk command, the second file is not split into rows (two commas ,, appear where there should be a line break).

Attached files:

  1. Original input file, for which awk command works perfectly
  2. Original output file from awk command with input file from '1' above
  3. New input file, for which awk doesn't work
  4. New output file showing the rows not being divided

I need the command to work for both input file types, ideally. Your help would be greatly appreciated!!!

Thanks again,
gwr

10

Files attached now, sorry!

gwr

11

In file #2 the blank lines between datasets has some space characters in it, the following update should fix:

awk '
NR==FNR{if(Z&&NF!=2)Z=0;if(Z)C=C>$1?C:$1;if(NF==4)Z=1;next}
C{printf ",,,,";while(--C)printf ++i",";print ++i}
A&&NF!=2 {A=0} A{ printf A; L=1; A=x} /^[ \t]*$/&&L { print; L=0 } L{ printf ","$2} NF==4 { A=$1","$2","$3","$4}' EIGENVAL_2.txt EIGENVAL_2.txt
12

Thanks Chubler_XL! I will give that a try.

gwr