Remove Unwanted Libraries - optimizing

We have a huge makefile composing of inclusion of libraries, objects and system libraries to generate a binary.

How do we find out that which of the libraries we can remove in the most efficient way? Doing hit and trial method is a waste of time and can during the linking with some post linking script can we find that it is fine to remove this / these library from makefile

Compile your binary.

ldd mybinaryfile

will list the libraries linked into the executable.

But ldd prints the shared libraries required by each program or shared library specified on the command line. We have many static libraries also mentioned in our makefile

Hello ,
Make a backup copy of the Makefile.
do one thing. remove all the libraries except the shared ones that 'ldd' mentions as reported in the previous post.
so now you are left with only the shared library in the makefile. Compile the source and you would be shown the missing dependancies . Those dependancis are likely to be the required static libraries. and you can know which ones do you need.
After that you can take care of the original Makefile and trim it for useless static libraries.
Hope this helps.
Regards.

This is what I want to avoid - hit and trial method, the problem will get worse when inclusion of one object / static library file I have another dependency. Further this will take lots of workload and days to drive it to completion

Hello,
but this is once and for all. you would have to do only one step to know everything. This is not kind of hit and trial I feel. Its the only absolute solution if you dont want hit and trial.
Otherwise you would have to check the documentation or consult the actual developers to know the exact libraries and their dependancy.
Regards.

I am curious. What are you trying to achieve and why? Does your makefile force unneeded static libs to be included in the executable?

It seems that earlier developers to save effort on there end included all libraries within our project workspace which would be around 500. Now we are coding a separate test module which includes some objects referenced in the makefile. We created a new makefile for same and inclusion of some libraries for generating binary is giving hundreds of 'undefined reference' symbol error.

Are you using GNU Make? It has some tools builtin to find your dependencies, build static dependency files, and copious debug output. You can use these to achieve your goals.

Though we are not using GNU make, but right now we are in dead need to know that amongst the 500 libraries if I using some of them then what are the others which I can remove from my new makefile.

Please provide some reference for same.

Here is a genetic way of doing it

Use the nm utility to ouput the symbol table for your application. A 'T' in the second column indicates a function that is either defined in your source code or comes from a static library. A 'U' in the second column indicates a function that is included via a shared library. You then need to massage that list (listA) to remove all the functions that are defined within your source code.

Next write a script to nm each of your 500 libraries and construct a sorted list (listB) of library + function names.

Finally write a script which uses listB to update listA by adding a column containing the name of the library where that function is defined.

At this stage you should have a list of all the external functions used by your application together with the name of library which defined the function.

It is then a trivial exercise to figure out which libraries are not needed by the application.

Thanks a lot. We do need one automated script that would do this all.

I don't think that trial/error would be so tough. In pseudo perl code:

my @all_libs  = split (/\s+/, `ls $LIBPATH/*.a`);
my $good_libs = "";

# get number of unresolved messages w/o any static lib:
my $n_symbols = `make target | grep 'unresolved' | wc -l`;
chomp ($n_symbols);

foreach my $lib ( @all_libs )
{
  # try to add that static lib, and see if the number of unresolved syms decreases

  $lib =~ s/^lib//;
  $lib =~ s/\.a$//;
  
  my $check = `/bin/env LDFLAGS=-l$lib make target | grep 'unresolved' | wc -l`;
  chomp ($check);

  if ( $check < $n_symbols )
  {
     # good lib, well done!
     $good_libs .= " -l$lib");
  }    
}

# we want to link against all good libs
print "$good_libs\n";

You may getting symbols resolved twice - but that is a situation you should have stumbled over previously, so I guess thats unlikely. In any case, that will be simplier then the more formal, and elegant :), way via nm, which was proposed earlier.

The above won't work if some of your static libs depend on other static libs - in that case, you need to add those 'good' libs you found to the test linking step.