Using libarchive to generate gzip files

I am working through this example: Examples · libarchive/libarchive Wiki · GitHub

Have installed libarchive with: sudo apt install libarchive-dev
Header files to add:

#include <stdio.h>
#include <fcntl.h>
#include <archive.h>
#include <archive_entry.h>

Using gcc 11.4.0 to compile as follows: gcc archivecreator.c -larchive -o archivecreator
Then run as follows: ./archivecreator ../images, where the latter is a bunch of png files, and the program terminates normally.
However, I am not sure what this program actually outputs: no files have changed right after execution, debuggers do not show any abnormal behavior, and as a corollary no gzip file has actually been produced.
Note that the struct archive itself is defined in libarchive/archive_private.h

Can someone explain how I should use or amend the example to create a gzip file of a directory?

@technossomy,

Have you tried tracing the execution ?

I'll try this when near a computer....

@technossomy , having read the code .....

this creates a gzipped tar ball - .tgz

To create the archive you want the command line to include the name of the archive and a list of files to be archived - a directory name is not sufficient

archivecreator yourArchiveName files- to-archive

see session output below

ls -al images/*.png
-rw-rw-r-- 1 munke munke 9183 Dec 21 00:07 images/api-figure1.png
-rw-rw-r-- 1 munke munke 8983 Dec 21 00:07 images/api-figure2.png
-rw-rw-r-- 1 munke munke 8860 Dec 21 00:07 images/api-figure3.png
-rw-rw-r-- 1 munke munke 6091 Dec 21 00:07 images/array-elements.png
-rw-rw-r-- 1 munke munke 6078 Dec 21 00:07 images/general-program.png
-rw-rw-r-- 1 munke munke 7856 Dec 21 00:07 images/process-flow.png

#
# create archive
#

./archiver test/pngarchive.tgz images/*.png

#
# list contents
#

tar -tzvf test/pngarchive.tgz 
-rw-r--r-- 0/0            9183 1970-01-01 01:00 images/api-figure1.png
-rw-r--r-- 0/0            8983 1970-01-01 01:00 images/api-figure2.png
-rw-r--r-- 0/0            8860 1970-01-01 01:00 images/api-figure3.png
-rw-r--r-- 0/0            6091 1970-01-01 01:00 images/array-elements.png
-rw-r--r-- 0/0            6078 1970-01-01 01:00 images/general-program.png
-rw-r--r-- 0/0            7856 1970-01-01 01:00 images/process-flow.png

hope this helps ....

1 Like

That's great and have replicated your suggestion, so it works now.
Also learned about strace and will try that more often in the future.

My implicit point was that the folks at libarchive have a penchant for withholding information, and apparently the essentials that need to be added to the code are "things that everybody knows".

@technossomy , glad you got it working, it is a super basic implementation afaics and can easily be vastly improved with a bit of effort.

As for the project team , i can't comment as I've never used this before, however, presume you could improve the effort by contributing to it .... :nerd_face:

rgds

Does it produce a .tgz file?
Then you could as well define a function

archiver(){ tar czf "$@"; }

and run it with the files

archiver test/pngarchive.tgz images/*.png

or with the directory (and verbose)

archiver test/pngarchive.tgz --verbose images/

Thank you, but this looks like shell-commands. The business case was to use a standard library that I can integrate in other C code.

@technossomy , fyi , you should look into using zlib a free, well known and respected compression.

Forzlib I have done the following, where zpipe.c seems to be the main example:

$ wget https://zlib.net/zpipe.c
$ wget https://github.com/lz4/lz4/raw/dev/lib/lz4.h
$ gcc zpipe.c -g -Wall -o zpipe
/usr/bin/ld: /tmp/ccml3hsd.o: in function `def':
/home/madrid/Documents/zpipe.c:37: undefined reference to `deflateInit_'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:45: undefined reference to `deflateEnd'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:56: undefined reference to `deflate'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:60: undefined reference to `deflateEnd'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:71: undefined reference to `deflateEnd'
/usr/bin/ld: /tmp/ccml3hsd.o: in function `inf':
/home/madrid/Documents/zpipe.c:94: undefined reference to `inflateInit_'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:102: undefined reference to `inflateEnd'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:113: undefined reference to `inflate'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:120: undefined reference to `inflateEnd'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:125: undefined reference to `inflateEnd'
/usr/bin/ld: /home/madrid/Documents/zpipe.c:134: undefined reference to `inflateEnd'
collect2: error: ld returned 1 exit status

Also tried with lz4:

$ wget https://github.com/lz4/lz4/archive/refs/heads/dev.zip
$ unzip -q dev.zip
$ cd lz4-dev/
$ make
$ sudo make install
$ cd examples/
$ gcc blockStreaming_lineByLine.c -g -Wall -o blockStreaming_lineByLine
/usr/bin/ld: /tmp/ccXKZ7I8.o: in function `test_compress':
/home/madrid/Documents/lz4-dev/examples/blockStreaming_lineByLine.c:43: undefined reference to `LZ4_createStream'
/usr/bin/ld: /home/madrid/Documents/lz4-dev/examples/blockStreaming_lineByLine.c:67: undefined reference to `LZ4_compress_fast_continue'
/usr/bin/ld: /home/madrid/Documents/lz4-dev/examples/blockStreaming_lineByLine.c:82: undefined reference to `LZ4_freeStream'
/usr/bin/ld: /tmp/ccXKZ7I8.o: in function `test_decompress':
/home/madrid/Documents/lz4-dev/examples/blockStreaming_lineByLine.c:92: undefined reference to `LZ4_createStreamDecode'
/usr/bin/ld: /home/madrid/Documents/lz4-dev/examples/blockStreaming_lineByLine.c:107: undefined reference to `LZ4_decompress_safe_continue'
/usr/bin/ld: /home/madrid/Documents/lz4-dev/examples/blockStreaming_lineByLine.c:120: undefined reference to `LZ4_freeStreamDecode'
collect2: error: ld returned 1 exit status

Though the error message do not provide enough information to continue, the similarity between them seems to suggest a more fundamental issue, ie something that ought to be done as a prerequisite. What is the best course of action, given that also here the documentation is relatively sparse with troubleshooting information?

Ensure you have liblz4 installed (and linked).

Type  Library

bz2   libbzip2 (bzip2)
gz    zlib (gzip)
lz4   liblz4
xz    liblzma

So if the commands as stated here have been executed:

what is then meant by installation? And by linking?

Hi @technossomy,

you've forgotten to link against the lz4 lib, in which the compiled functions like LZ4_createStream(...) are located. Of course this lib must be installed on the system, either as a package (e.g. liblz4-1 on Ubuntu/Debian) or compiled by yourself.

$ gcc source.c -g -Wall -l lz4 -o outfile

The dev libs resp. header files are almost always available as a separate package (e.g. liblz4-dev on Ubuntu/Debian), so you won't need to fetch them from the development site. This also guarantees that the lib and the dev lib are in the same version resp. compatible.
See e.g. Linker (computing) - Wikipedia.

2 Likes