AIX linker oddities...what is going on?

I've tried to figure out what the linker is smoking in AIX to no avail...so I'm trying to find a little information to see why it's being inconsistent.

I have the following code in a shared library, it doesn't do anything useful, it's just there to exercise functions in a few system libraries:

#include <curses.h>
#include <math.h>
#include <pthread.h>

static pthread_mutex_t g_mx = PTHREAD_MUTEX_INITIALIZER;

static void *testT(void *arg)
{
	pthread_mutex_lock(&g_mx);
	printf("Hello from thread...\n");
	pthread_mutex_unlock(&g_mx);
}

void shlibfn()
{
	double	r;
	pthread_t	tid;
	
	pthread_mutex_lock(&g_mx);

	pthread_create(&tid, NULL, testT, NULL);
	
	r = sin(0.0);
	
	initscr();
	endwin();

	pthread_mutex_unlock(&g_mx);
}

It is built as follows:

cc shlibsrc.c -G -o libsh.so

Now, I have a very simple main application:

void main()
{
	extern void shlibfn();
	
	shlibfn();
}

Built as follows:

cc main.c -lpthread -lcurses -lm -L. -brtl -lsh -o main

I expect that ldd would say that main needs pthread, curses, and the math library in addition to sh at this point, but it doesn't. What ldd says is:

main needs:
	/usr/lib/libpthread.a(shr_xpg5.o)
	./libsh.so
	/usr/lib/libc.a(shr.o)
	/usr/lib/librtl.a(shr.o)
	/usr/lib/libpthreads.a(shr_comm.o)
	/unix
	/usr/lib/libcrypt.a(shr.o)

Ok...fine, but when I run it, even though I asked it to link main to curses, it did not. Therefore, I get a run time error from rtld that the symbols (initsrc and endwin) aren't found and libsh.o needs them.

Interesting...so, for grins I modify main to use something from curses:

#include <curses.h>

void main()
{
	extern void shlibfn();
	
	shlibfn();
	endwin();
}

Build it the same way, and now ldd picks up the curses library requirement on main. It also runs now. That's pretty...interesting...but I suppose the linker is being smarter than me...or is it.

Let's go back to the first main, without the call to endwin. We'll link him a few ways, here's the results:

Build 1:

>cc mainsrc.c -L. -brtl -lsh -o main
>ldd main
main needs:
         ./libsh.so
         /usr/lib/libc.a(shr.o)
         /usr/lib/librtl.a(shr.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)

Build 2:

>cc mainsrc.c -lpthread -L. -brtl -lsh -o main
>ldd main
main needs:
         /usr/lib/libpthread.a(shr_xpg5.o)
         ./libsh.so
         /usr/lib/libc.a(shr.o)
         /usr/lib/librtl.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)
>./main
exec(): 0509-036 Cannot load program ./main because of the following errors:
rtld: 0712-001 Symbol sin was referenced
      from module ./libsh.so(), but a runtime definition
      of the symbol was not found.
rtld: 0712-001 Symbol initscr was referenced
      from module ./libsh.so(), but a runtime definition
      of the symbol was not found.
rtld: 0712-001 Symbol endwin was referenced
      from module ./libsh.so(), but a runtime definition
      of the symbol was not found.

Interesting, I change nothing in the main source, I don't use anything from pthreads in main, but the linker does put a dependency on pthreads when I ask it to during the second build. As expected it doesn't run.

Now we'll go back to the second version of main where it uses something from curses and build him a few ways:

Build 1:

>cc mainsrc.c -lpthread -L. -brtl -lsh -o main
ld: 0711-317 ERROR: Undefined symbol: .endwin
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.

Expected, I'm trying to use endwin...let's link him; build 2:

>cc mainsrc.c -lpthread -lcurses -L. -brtl -lsh -o main
>ldd main
main needs:
         /usr/lib/libpthread.a(shr_xpg5.o)
         /usr/lib/libcurses.a(shr42.o)
         ./libsh.so
         /usr/lib/libc.a(shr.o)
         /usr/lib/librtl.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)
>./main
exec(): 0509-036 Cannot load program ./main because of the following errors:
rtld: 0712-001 Symbol sin was referenced
      from module ./libsh.so(), but a runtime definition
      of the symbol was not found.

Ok...getting closer, since we use endwin, it decides it is needed and, as such, pulls it in. But, I still didn't link math, so there's no sin and it won't run. That's all well and good, but I don't use anything from math in mainsrc.c, so if you follow the same logic that was needed to get curses to link, you'd think I'd need to use something...but, build 3:

>cc mainsrc.c -lpthread -lm -lcurses -L. -brtl -lsh -o main
>ldd main
main needs:
         /usr/lib/libpthread.a(shr_xpg5.o)
         /usr/lib/libcurses.a(shr42.o)
         ./libsh.so
         /usr/lib/libc.a(shr.o)
         /usr/lib/librtl.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)
>./main
Hello from thread...

And it works. The funny thing is, there is NO CHANGE to the output of ldd between the two versions. One works, one doesn't....same ldd output, and I didn't need to put anything in main to pick up the math library.

So...this isn't consistent, if fact it's downright mind boggling. I'm wondering if anyone can shed some light on why this is happening.

I also want to note that I can fix all this stupidity by changing the way I link the library. If I link the library to the pthreads, curses, and math library as follows:

cc shlibsrc.c -G -o libsh.so -lpthread -lm -lcurses

Then I can build mainsrc.c as follows:

>cc mainsrc.c -L. -brtl -lsh -o main

And it works fine and the ldd output is almost identical (just with the order switched) to the working version above. I realize that this is probably the "best" way to link this stuff together so that the library states its dependencies, but I can't always have control over how the library was built (we don't build them all) and so I'm running into these oddities.

Sorry for the long post...but this AIX linker is driving me nuts!

P.S. None of this stupidity happens in Linux. In Linux, the minute I try to build main linking to sh without any of the required libraries (pthread, math, ncurses) it complains and won't build.

$ cc mainsrc.c -L. -lsh
./libsh.so: undefined reference to `endwin'
./libsh.so: undefined reference to `initscr'
./libsh.so: undefined reference to `pthread_create'

I link it the way I expect to work in AIX:

$ cc mainsrc.c -L. -lpthread -lm -lncurses -lsh
$ export LD_LIBRARY_PATH=.
$ ./a.out
Hello from thread...

And it works without a hitch. Seems like Linux's linker has its act a little better together, or it's just AIX's stupid "shared archive" vs. "shared library" crap that's messing with me. I don't know...but that's why I'm posting here...maybe someone familiar with AIX will.

Thanks all!

Have you tried compiling in verbose ? I think in your later compiles, that libm is linking against the static library, not the shared libm, and thats why you dont see it on the ldd.

I'd try using "nm -P" and "dump -H" to try and track down what functions are missing from where.

However, I would expect your:

cc shlibsrc.c -G -o libsh.so -lpthread -lm -lcurses

to be the correct one...I dont think when you link the mainsrc.o against libsh.so it checks that all the symbols are defined. Therefore I would expect you to have to specifically link shared objects against the libraries they need.

(Sorry If Ive missed stuff in the post - its pretty long, and I obviously work with computers so my attention span is short ;-))

Is there any way to force the compiler to behave like the Linux compiler? There are a few libraries we've had built for us that don't link all their dependencies correctly and, as such, we need to pull strings to get the linker to build the final executable "right" and in working order.... Sometimes we have to go searching around for the library that contains the symbol and link it to the main executable; sometimes that works (like pthreads took) and sometimes it doesn't (like curses didn't). I'd expect when linking mainsrc.c that if I tell the linker to link a library that it will regardless of whether or not code in that library is needed.

So, while you may very well be correct about libm, that doesn't explain pthreads vs. curses. I don't have to use any functions from pthreads to force the linker to give main a dependency to that, while I do have to use a function from curses to get that dependency. What gives there?

Ive always found the AIX linker to be an oddity - different from most of the others...
I believe GNU get around it using "libtool" - If you use that on all your build platforms, it works out what and how to link what you need (and will do stuff like setting the rpath in the shared object - something you may not have thought about)

You know the saying: "Sometimes a programmer has a problem, and he thinks to himself 'I know, I'll solve it with a regular expression!'. Now he has two problems." I think the same of autotools.

I totally agree....but then, who doesnt use regular expressions ?
Cant live with them, cant live without them...

Never used autoconf or automake - have heard nightmares about them, but Ive used libtool and I find for certain crossplatform issues it works well...

Speaking of paths...it seems the linker places the path of the shared library in the executable itself. The documentation says to pass -bnoipath to, supposedly, prevent this. However, using -bnoipath (or -Wl,-bnoipath from cc) doesn't seem to work. The path is still present and shown by ldd. This isn't directly an issue because it does appear the runtime linker uses the LD_LIBRARY_PATH first before consulting the paths in the executable, but I'd prefer it not to have the option. This way, if a developer forgets to properly configure the LD_LIBRARY_PATH they don't pick up a version of the library by default, rather they get an error that the library isn't found.

As for the original problem...I still can't make heads or tails of it. We're just going to have to keep fighting the linker. Ugh...

Yep...thats what my comment about "rpath" was about...If I remember you need -blibpath as well...

---------- Post updated at 09:11 PM ---------- Previous update was at 09:08 PM ----------

Use "dump -Hv" to check the settings once you have compiled - that will show where it searches for the lib. Also make sure you have no LIBPATH set when you compile, as that can make a different to where the linker finds the libs

1 Like

We'll see how that works.... I hate this linker, lol. Out of all the environments I've used, I think AIX is probably the worst. HP-UX probably was my favorite, and I originally hated Linux and thought it was really quirky, but AIX has made it look better, HA!

I had to break a ton of habits moving to linux. In retrospect a lot of them were pretty bad habits. :o But I've never really had the opportunity to program on "big iron" like HP-UX or AIX, though I used a Sun system in college... what's the key differences you see as a programmer?

My main job for the last 10 years or so is porting different low level security apps to UNIX platforms. AIX is almost always the "difficult" one. Everything is always implemented differently. I code around the native security databases (/etc/passwd, shadow, /etc/security, TCB, etc) and AIX is always the strange one. I do have to say though that HP-UX has its quirks, although it generally conforms. Linux is easy, cos you can find the code to everything, so you can code round stuff or at least find out how it works. And there are always lots of people that have tried stuff or implemented stuff and its all relatively up-to-date. I have to admit though that Solaris is my favourite...things just seem to work, and if they dont a quick O/S patch usually fixes it. That is until Solaris 10 where, although I love some of the additions, I hate the horrible SMF service stuff, and process contracts that can make life a pain.

Anyhew, on the linker front, that is why I suggested libtool. You *can* make build systems that build generically and you dont need to know this low level stuff.

But thats all my 2c worth...other people have different views on libtool.

I'll check out libtool...but we've already fought to get some freeware tools installed and it was hard enough to win those battles...I'm not sure I want to fight another yet, lol.

As for Corona's Q: as far as developing goes, I've just felt that HP-UX has been the easiest to work with. Things just seem to compile and run well and as expected.

Jumping over to Linux I've been hit with times when an application just randomly seems to die and the core file is useless. We never had that in HP-UX, if it core'd we could use the core and figure out what happened. Plus, much of the code in Linux is the same as it was in HP-UX and the code ran more stable in HP-UX. Could it be our code...sure...but many of the times we have tracked down issues it was because Linux was just less tolerant or, worse, buggy itself and we had to code around its issues.

AIX is somewhere in between. It does seem pretty stable compared to Linux. However, sometimes intuition and AIX just don't go hand in hand. It is as if IBM went out of their way to do something just different enough to mess with you, lol -- like this weird linker crap, lol. While it is more stable...nothing else in it is better than Linux, IMO. They just run away with things like a Microsoft...trying to make it better than other people's stuff but getting in their own way when doing so.