Split the text file into two

Shell Programming and Scripting
1

OS : RHEL 7.3

I have a file like below. I want to move (cut and paste) the first 7 lines of file1 to another file (file2).
How can I do this ? In my real life scenario, I will be moving first 12 millions lines of file1 to file2

$ cat file1.txt
      7369|SMITH     |CLERK    |      7902|17-DEC-80|       800|          |        20
      7499|ALLEN     |SALESMAN |      7698|20-FEB-81|      1600|       300|        30
      7521|WARD      |SALESMAN |      7698|22-FEB-81|      1250|       500|        30
      7566|JONES     |MANAGER  |      7839|02-APR-81|      2975|          |        20
      7654|MARTIN    |SALESMAN |      7698|28-SEP-81|      1250|      1400|        30
      7698|BLAKE     |MANAGER  |      7839|01-MAY-81|      2850|          |        30
      7788|SCOTT     |ANALYST  |      7566|19-APR-87|      3000|          |        20
      7839|KING      |PRESIDENT|          |17-NOV-81|      5000|          |        10
      7844|TURNER    |SALESMAN |      7698|08-SEP-81|      1500|         0|        30
      7876|ADAMS     |CLERK    |      7788|23-MAY-87|      1100|          |        20
      7900|JAMES     |CLERK    |      7698|03-DEC-81|       950|          |        30
      7902|FORD      |ANALYST  |      7566|03-DEC-81|      3000|          |        20
      8839|KATE      |ANALYST  |      7566|17-FEB-79|      6000|          |        40
2

Did you consider the split command?

3

Hello kraljic,

Could you please try following and let me know if this helps you.

awk -v count=1 -v line=7 'NR>line &&!a{close("file"count);count++;a++} {print > "file"count}' FiLe1.txt

You have to change variable named line's value to as per your lines which you want in file1, also this logic will only create 2 files named file1(where it will put those many lines from starting of file1.txt which you will give in line variable) and file2 will have remaining lines, kindly do give a try to it and let me know if this helps you.

Thanks,
R. Singh

1 Like
4

If you want to deploy awk , try

awk -vLN=7 '{print > ("file" 2-(NR>LN))}' file
2 Likes
5

Hello kraljic,

I have a few to questions pose in response first:-

  • What have you tried so far?
  • What output/errors do you get?
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)

Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.

Did split help, it's probably the right tool for this, unless you want to do other processing at the same time. With split you can break up your file based on the number of bytes, lines, etc. and define output filenames and suffix length to make it suitable. It will not easily take the literal name file1 and generate a file2, but you can force it to be something sensible and then rename the output as needed.

If you generate 5 files of output and want to be separating out just the content of the first, it is straightforward to rename that one to the desired name and either delete the remainder or cat them back together (in the right order) to overwrite the original file, thereby removing the lines you have just extracted. You would then need to tidy the temporary files up too to save space.

We're all here to learn and getting the relevant information will help us all.

Have a go with split as my learned member RudiC suggests and let us know if it helps or you get stuck. I'm sure we can get a working solution that you can support and/or reuse.

kind regards,
Robin

1 Like
6

Well, that's a neater solution than the temporary split files I was thinking of. I will save that in my own hints & tips file, thank you :b: :wink:

Many thanks,
Robin