relative path library (linux)

While you can set LD_LIBRARY_PATH to let the dynamic linker know where to look, there are better options. You can put your shared library in one of the standard places, see /etc/ld.so.conf (on Linux) and /usr/bin/crle (on Solaris) for the list of these places

You can pass -R <path> to the linker when building your binary, which will add <path> to the list of directories scanned for your shared library. Here's an example. First, showing the problem:

libtest.h:

void hello_world(void);

libtest.c:

#include <stdio.h>
void hello_world(void) {
  printf("Hello world, I'm a library!\n");
}

hello.c:

#include "libtest.h"
int main(int argc, char **argv) {
  hello_world();
}

Makefile (tabs must be used):

all: hello
hello: libtest.so.0
%.o: %.c
        $(CC) $(CFLAGS) -fPIC -c -o $@ $<
libtest.so.0.0.1: libtest.o
        $(CC) -shared -Wl,-soname,libtest.so.0 -o libtest.so.0.0.1 libtest.o
libtest.so.0: libtest.so.0.0.1
        ln -s $< $@
clean:
        rm -f hello libtest.o hello.o libtest.so.0.0.1 libtest.so.0

Let's run it:

$ make
cc  -fPIC -c -o libtest.o libtest.c
cc -shared -Wl,-soname,libtest.so.0 -o libtest.so.0.0.1 libtest.o
ln -s libtest.so.0.0.1 libtest.so.0
cc     hello.c libtest.so.0   -o hello
$ ./hello
./hello: error while loading shared libraries: libtest.so.0: cannot open shared object file: No such file or directory

How to fix it? Add -R <path> to the linker flags (here, by setting LDFLAGS).

$ make clean
(...)
$ make LDFLAGS="-Wl,-R -Wl,/home/maciej/src/tmp"
(...)
cc   -Wl,-R -Wl,/home/maciej/src/tmp  hello.c libtest.so.0   -o hello
$ ./hello
Hello world, I'm a library!

Looking at the binary, you can see that it needs libtest.so.0:

$ objdump -p hello | grep NEEDED
  NEEDED               libtest.so.0
  NEEDED               libc.so.6

The binary will look for its libraries, apart from the standard places, in the specified directory:

$ objdump -p hello | grep RPATH
  RPATH                /home/maciej/src/tmp

If you want the binary to look in the current directory, you can set the RPATH to $ORIGIN. This is a bit tricky, because you need to make sure that the dollar sign is not interpreted by make. Here's one way to do it:

$ make CFLAGS="-fPIC" LDFLAGS="-Wl,-rpath '-Wl,\$\$ORIGIN'"
$ objdump -p hello | grep RPATH
  RPATH                $ORIGIN
$ ./hello
Hello world, I'm a library!