Tar gzip compression rate

UNIX for Advanced & Expert Users
1

How good is the compression rate of gzip when you use tar with the gzip option? I am pretty amazed that a 1 GB file was reduced to 1019K. This is what I did.

tar -cvf tar_test.tar.gz -T /list_of_files
ls -hl
-rw-r-----. 1 owner group    19 Jul 23 16:00 list_of_files
-rw-r-----. 1 owner group 1019K Sep 19 02:17 tar_test.tar.gz
1 Like
2

Hi,

The compression efficiency really depends on what is being compressed, in general high compression ratios of 90% plus seem to be available on large text files. But most compression utilities tend to give only a few percent when compressing binaries or populated database tables.

I dont think I've seen the 99.9% that you've achieved here, but I have seen 90% regularly on text files and I'm sure that on one occasion a few years ago an Oracle Database that I was migrating reduced by a huge amount just can't rmember by how much.

From the example above, I'm not sure how you compressed the file, are you sure that everything is there?

Regards

Gull04

1 Like
3

Looks strange to me...
You did not use the GNU tar gzip option -z at all.
So content is not compressed.

Can we see output of

file tar_test.tar.gz

Does tar -tvf tar_test.tar.gz display the files you wanted to tar on standard output ?

Are you sure you used right from file location ?
The /list_of_files is under root(/), is that your working folder ?

Compression ratio depends on what are you compressing.

Regards
Peasant.

4

I checked the GNU version of tar and the only way to auto compress using file extension magic is to use the "a" flag:

You can also let GNU tar select the compression program based on the
suffix of the archive file name. This is done using �--auto-compress' (�-a')
command line option. For example, the following invocation will use bzip2
for compression:

$ tar caf archive.tar.bz2 .

GNU tar manual in PDF attached as reference.

1 Like
5

Hi Neo,

Thanks for the feedback, I was about to go and look for an update to tar as mine didn't seem to work that way.

Regards

Gull04

6

Welcome.

I wanted to test further but I am in the middle of making a new "how to use code tags and format posts video" for unix.com, and have been having some issues with rendering today; so my entire day has been wasted rendering video. But the good news is that "eventually" we will have a new "how to use code tags" video :slight_smile:

Sorry, if my post(s) on this GNU tar issue are not fully tested, as I have a lot of balls up in the air right now, video, code changes, mobile CSS for the site and more..... but I did finally take a closer look at the GNU tar PDF while video was rendering.

Cheers

1 Like
7

Sparse file?

ls -las t*
  8 -rw-rw-r-- 1 user group 1024000002 Sep 19 13:50 test
972 -rw-rw-r-- 1 user group     993894 Sep 19 13:51 t.gz

And, gzip invoked by tar takes account of "The obsolescent environment variable GZIP" for e.g. the -# --fast --best compression method.

8

Totally depends on what you are compressing. If it's plain text you would be able to compress by the size you have mentioned. Try video it would be another story altogether.

9
Moderator comments were removed during original forum migration.
10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.