Work with huge Zipped files

Shell Programming and Scripting
1

Hello dear members,

I have one general and one specific question which I will be very grateful if you could help me with them. Let's start with my general question:

  1. I am working on cluster computer shared with other people and I need to manipulate a big zipped text file of 13 GB. There is no possibility of unzipping it, if I could do that, I would have any problem but my problem is that working with zipped files using Awk or Linux commands become very difficult. I would be very grateful if you could give me some directions in this regard.

  2. The second question is that being in that 13 GB zipped, I want to select one part of the file to make an example file to write my code to be able to later apply them to the whole file. When I display the file on shell by the command:

zcat zippedfile | less

and I copy a section just by clicking and I paste it in a text editor, the spacing, number of fields, everything gets destroyed. How could I select a part of the file by command line without unzipping it?

Thank you

2

Hello Homa,

  • How do you know which section you are after?
  • How many lines is the total file and how many lines do you want to extract?
  • What operating system and version are you using?

Thanks, in advance,
Robin