Splitting textfile based on pattern and name new file after pattern

Shell Programming and Scripting
1

Hi there, I am pretty new to those things, so I couldn't figure out how to solve this, and if it is actually that easy. just found that awk could help:(.

so i have a textfile with strings and numbers (originally copy pasted from word, therefore some empty cells) in the following structure:

SC 3 5 6
<empty> 5 6 7 
<empty> 7 6 2
<empty><empty><empty>
SP 7 2 3
<empty> 9 6 7 
<empty> 7 5 2
<empty><empty><empty>
...

There are 6 such strings in total that should each define the first line of a new textfile, and the textfile should have that name.
so for example for SC, I would like a textfile SC.txt, with content:

3 5 6
5 6 7
7 6 2

the 6 textfiles shouldn't contain weird empty spaces anymore, since another program will read them and i want to make sure this won't cause trouble.
(ah, the number of lines is always 6, except for the last string, if that helps).

I am ideally looking for an easy solution in bash or python which i can understand. my files are not big...

thank you for any suggestions

2

Here is an awk solution:

awk '{if ($1 !="<empty>") fn=$1; print $2,$3,$4 > fn".txt"}' file.txt

cat SC.txt                                                           
3 5 6
5 6 7
7 6 2

cat SP.txt                                                           
7 2 3
9 6 7
7 5 2
3

mjf's solution is close, but it doesn't deal with those <empty><empty><empty> lines. Please tell us

  • how fields are separated, esp. that <empty><empty><empty> ones
  • what does <empty> mean
    You might want to post a (short) binary listing of a sample file (use od or hexdump).
4

My awk solution writes blank lines to the file <empty><empty><empty>.txt. Once luja defines how fields are separated then we can make the proper adjustments.

5

sorry, this was confusing. the <empty> means there is a space. so the solution does not work. there is no empty in the file. the file looks like this:

SC    32.245    24.153    1
    213.179    24.154    1
    368.069    24.154    1
    464.704    24.153    1
    742.256    24.154    1
    871.102    24.154    1
            
SP    122.712    24.153    1
    245.391    24.153    1
    309.814    24.154    1
    587.383    24.153    1
    710.061    24.154    1
    838.891    24.153    1

uh, hexdump, ok i did that (in the terminal just typed that followed by the file name, yes?!) it gave me something really not similar to the file but ok, if that helps:

0000000 53 43 09 30 2e 30 33 33 09 32 34 2e 31 35 33 09
0000010 31 0d 0a 09 32 37 33 2e 36 32 31 09 32 34 2e 31
0000020 35 33 09 31 0d 0a 09 34 30 32 2e 34 36 36 09 32
0000030 34 2e 31 35 33 09 31 0d 0a 09 34 39 30 2e 39 32
0000040 36 09 32 34 2e 31 35 34 09 31 0d 0a 09 36 36 37
0000050 2e 38 38 31 09 32 34 2e 31 35 33 09 31 0d 0a 09
0000060 38 32 30 2e 37 38 30 09 32 34 2e 31 35 33 09 31
0000070 0d 0a 09 09 09 0d 0a 53 50 09 31 32 30 2e 37 32
0000080 32 09 32 34 2e 31 35 33 09 31 0d 0a 09 32 34 31

....

---------- Post updated at 06:31 PM ---------- Previous update was at 06:21 PM ----------

i just realised that actually the empty fields do not matter. when i used awk '{print $1}' i got the first column whether there was an empty space at the beginning of the line or not. so i basically just got the first entry in each line. so i guess that makes the solution easier. it should just write the numbers, so colums 2-4 if there is the string in front and then continue with colums 1-3 until the next string. does that make sense?

6

luja, try the below as this seems to fit your request:

awk 'NF>0 {if (NF==4) {fn=$1; print $2,$3,$4 > fn".txt"} else {print $0 >> fn".txt"}}' file.txt
7

You might need to add a close() when done with each file, to avoid reaching the open file descriptor limit.

Regards,
Alister

8

thank you, yes first got an error, then added the brackets:

awk 'NF>0 {if (NF==4) {fn=$1; print $2,$3,$4 > (fn".txt")} else {print $0 >> (fn".txt")}}' test_fsl.txt

so that worked.
the resulting textfiles still show an empty space before lies 2 to 6 (not in line 1 because there was the string). i don't understand enough about the way those textfiles are encoded, i.e. what they actually contain or not. but maybe i should remove those in the resulting files to make sure. so i got rid of these with:

awk '{$1=$1}1' SP.txt > SP2.txt

. maybe that could be combined with the first awk command? but else i just put it in sequentially, actually i could do that first i guess...

9

Given the file sticks to the structure posted above, try

awk '$1 {if (FN) close (FN); FN=$1".txt"} NF==4{print $2,$3,$4 >FN}' FS="\t" OFS="\t" file

.
Anyhow, that is not a *nix text file; you should get rid of the <CR> (^M, \r,carriage return) char first...

10

Thank you! This works as well!