Mastering Linux Library Paths: Solving pkg-config and LD_LIBRARY_PATH Issues

Background

When compiling software from source on Linux, two frequent library‑search problems appear: missing dependencies detected by pkg-config during the configure stage, and runtime loader errors such as “error while loading shared libraries: libxxx.so.y: cannot open shared object file: No such file or directory”. Understanding how pkg-config locates .pc files and how the dynamic loader resolves shared objects is required to fix both issues.

Problem 1 – Missing Dependencies During Source Installation

Typical source‑build workflow:

./configure
make
make install

The configure script invokes pkg-config to check that required libraries are present. pkg-config reads metadata files with a .pc extension. By default these files are installed under ${prefix}/lib/pkgconfig and ${prefix}/share/pkgconfig. If --prefix is omitted, prefix defaults to /usr/local, so the default locations become:

/usr/local/lib/pkgconfig

/usr/local/share/pkgconfig

Example librtmp.pc file:

prefix=/usr/local
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
incdir=${prefix}/include

Name: librtmp
Description: RTMP implementation
Version: v2.3
Requires: libssl, libcrypto
URL: http://rtmpdump.mplayerhq.hu
Libs: -L${libdir} -lrtmp -lz
Cflags: -I${incdir}

Running pkg-config --cflags librtmp and pkg-config --libs librtmp yields the compiler and linker flags that would otherwise have to be hard‑coded in a Makefile:

# pkg-config --cflags librtmp
-I/usr/local/include

# pkg-config --libs librtmp
-L/usr/local/lib -lrtmp -lz -lssl -lcrypto

If pkg-config cannot find a .pc file, the cause is usually one of two:

The search directories do not contain the file.

The upstream package does not provide a .pc file.

Search paths are built from:

${libdir}/pkgconfig

${datadir}/pkgconfig

When a package manager installs a library (e.g., yum or apt), libdir and datadir resolve to /usr/lib and /usr/share. When building from source without a custom --prefix, they resolve to /usr/local/lib and /usr/local/share.

Solutions

Install the dependency with --prefix=/usr so its .pc files land in the system directories that are always searched.

Export PKG_CONFIG_PATH to prepend the directory that contains your custom .pc files, for example:

export PKG_CONFIG_PATH=/opt/mylibs/lib/pkgconfig:$PKG_CONFIG_PATH

Adding the export line to ~/.bash_profile or /etc/profile makes the setting persistent.

Problem 2 – Runtime “libxxx.so.y not found” Errors

When an executable starts, the dynamic loader ( /lib/ld-linux.so.*) searches for shared objects in the following order:

Directories listed in the LD_LIBRARY_PATH environment variable (colon‑separated).

The cache file /etc/ld.so.cache, which is generated by ldconfig.

Standard library directories such as /lib, /usr/lib, /lib64, /usr/lib64.

To inspect which libraries an executable needs, use:

ldd /path/to/executable

If the output contains lines like libfoo.so.1 => not found, the program will abort.

Typical remedies

Temporarily set LD_LIBRARY_PATH in the current shell before launching the program, e.g.:

export LD_LIBRARY_PATH=/opt/mylibs/lib:$LD_LIBRARY_PATH
./myprogram

Install the missing library system‑wide (requires root) and run ldconfig to rebuild the cache.

Persistently adding LD_LIBRARY_PATH to global profile files is discouraged in production because it can interfere with other applications.

Key Takeaways

Both compile‑time and run‑time library‑search failures stem from the loader being unable to locate the required metadata ( .pc) or shared‑object files. By:

Understanding that pkg-config searches ${libdir}/pkgconfig and ${datadir}/pkgconfig and configuring PKG_CONFIG_PATH when necessary, and

Knowing the loader’s search order and using LD_LIBRARY_PATH or ldconfig appropriately,

developers can reliably build and run Linux applications without “cannot open shared object file” errors.