Create new library

This page has been automatically translated using the Google Translate API services. We are working on improving texts. Thank you for your understanding and patience.

The only thing that you absolutely have to know, is the location of the library. Albert Einstein

The use of libraries will allow us to share common code between several projects. An example is the NAppGUI SDK, which has been organized into several static or dynamic link libraries. In Use of libraries we already saw a first introduction, which we will expand on in this chapter.

1. Static libraries

To escape the simplistic introduction of the previous chapter, we are going to use two applications included in the NAppGUI examples: Die (Figure 1) and Dice (Figure 2). In both you must be able to draw the silhouette of a dice.

It is not very complicated to intuit that we could reuse the parametric drawing routine in both projects. One way to do this would be to copy said routine from Die to Dice, but this is not the most advisable since we would have two versions of the same code to maintain. Another option, the most sensible, is to move the drawing function to a library and link it in both applications. This is very easy to do thanks to find_package() and NAppProject.cmake. Download the complete example from this link. Unzip it and inspect its files. The structure of the project is very similar to what was seen in the previous chapter, starting with the main CMakeLists.txt:

1
2
3
4
5
6
7

cmake_minimum_required(VERSION 3.0)
project(NAppDice)
find_package(nappgui REQUIRED)
include("${NAPPGUI_ROOT_PATH}/prj/NAppProject.cmake")
nap_project_library(casino casino)
nap_project_desktop_app(Die die)
nap_project_desktop_app(Dice dice)

• Line 1: Set the minimum version of CMake.
• Line 2: Project name.
• Line 3: Locate the NAppGUI-SDK installation.
• Line 4: Includes the NAppProject.cmake module.
• Line 5: Look for a target library in the casino directory.
• Line 6: Look for an application target in the die directory.
• Line 7: Look for a target application in the directory says.

In /die/CMakeLists.txt and /dice/CMakeLists.txt we see the link with casino:

nap_desktop_app(Die "casino" NRC_EMBEDDED)
target_link_libraries(Die ${NAPPGUI_LIBRARIES})

nap_desktop_app(Dice "casino" NRC_NONE)
target_link_libraries(Dice ${NAPPGUI_LIBRARIES})

The only thing that, so far, we have not seen are the constants NRC_EMBEDDED and NRC_NONE. In Resource processing we will see them in detail. Don't worry about them for now. You can build and compile the project in the usual way:

cmake -S . -B build -DCMAKE_INSTALL_PREFIX=C:/nappgui
cmake --build build --config Debug

Both Die and Dice have added a dependency on casino (Figure 3) via the dependList parameter of the nap_desktop_app() command. This way CMake knows that it must link, in addition to NAppGUI-SDK (NAPPGUI_LIBRARIES), the casino library, which is where common code from both projects is found (Figure 4).

What does it really mean that Die and Dice have a dependency on casino? That from now on none of them can be compiled if there is an error in the casino code, since it is a fundamental module for both. Within the build project (Visual Studio, Xcode, Makefile, Ninja, etc) several things have happened:

• Both applications know where casino is located, so they can do #include "casino.h" without worrying about its location.
• The binary code of the casino functions will be included in each executable in the linking process. CMake has already taken care of linking the library with the executables.
• Any changes made to casino will force the applications to be recompiled due to the previous point. Again, the build project will know how to do it in the most efficient way possible. We just have to run cmake --build ./build again to update all the binaries.

2. Dynamic libraries

Dynamic libraries are essentially the same as static libraries. The only thing that changes is the way they link to the (Figure 5) executable. In the static link, the library code is added to the executable itself, so the size of the latter will grow. In dynamic linking the library code is distributed in its own file (.dll, .so, .dylib) and is loaded just before the executable program.

To create the dynamic version of casino, open /casino/CMakeLists.txt and change the buildShared parameter of nap_library() from NO to YES. You will also need to link casino with NAppGUI-SDK, something that does not need to be done in the static version.

nap_library(casino "" YES NRC_NONE)
target_include_directories(casino PUBLIC "${NAPPGUI_INCLUDE_PATH}")
target_link_libraries(casino ${NAPPGUI_LIBRARIES})

After re-generating and re-compiling the solution, you will notice that a new casino.dll appears in /build/Debug/bin. This dll will be shared by Die.exe and Dice.exe, something that did not happen when compiling the static version.

12/18/23  04:38 PM    <DIR>          .
12/18/23  03:59 PM    <DIR>          ..
12/18/23  04:38 PM            53,248 casino.dll
12/18/23  04:38 PM            92,672 Dice.exe
12/18/23  04:38 PM           102,400 Die.exe

2.1. Advantages of DLLs

As we have been able to intuit in the previous example, using DLLs we will reduce the size of the executables, grouping the common binary code (Figure 6), (Figure 7). This is precisely what operating systems do. For example, Die.exe will ultimately need to access Windows API functions. If all applications were to statically link Windows binaries, their size would grow inordinately and a lot of space within the file system would be wasted.

Another great advantage of DLLs is memory savings at runtime. For example, if we load Die.exe, casino.dll will be loaded at the same time. But if we then load Dice.exe, both will share the existing copy of casino.dll in memory. However, with static linking, there would be two copies of casino.lib in RAM: One built into Die.exe and one from Dice.exe.

2.2. Disadvantages of DLLs

The main drawback of using DLLs is the incompatibility that can arise between the different versions of a library. Suppose we release a first version of the three products:

casino.dll        102,127 (v1)
Die.exe            84,100 (v1)
Dice.exe           73,430 (v1)

A few months later, we released a new version of the Dice.exe application that involves changes to casino.dll. In that case, the layout of our suite would look like this:

casino.dll        106,386  (v2)*
Die.exe            84,100  (v1)?
Dice.exe           78,491  (v2)*

If we have not been very careful, it is very likely that Die.exe no longer works because it is not compatible with the new version of the DLL. This problem is causing many developers head and has been dubbed DLL Hell. Since in this example we work on a "controlled" environment we could solve it without too much trouble, creating a new version of all the applications running under casino.dll(v2).

casino.dll        106,386  (v2)
Die.exe            84,258  (v2)
Dice.exe           78,491  (v2)

This will not always be possible. Now suppose that our company develops only casino.dll and third parties work on the final products. Now each product will have its production and distribution cycles (uncontrolled environment) so, to avoid problems, each company will include a copy of the specific version of the DLL with which their product works. This could lead to the following scenario:

/Apps/Die
casino.dll        114,295  (v5)
Die.exe            86.100  (v8)

/Apps/Dice
casino.dll        106,386  (v2)
Dice.exe           72,105  (v1)

Seeing this, we intuit that the benefits of using DLLs are not so good anymore, especially with regard to space optimization and load times. The fact is that it can get even worse. Typically, libraries are written to be as generic as possible and to serve many applications. On many occasions, a given application uses only a few functions from each library it links to. By using static libraries, the size of the (Figure 8) executable can be considerably reduced, since the linker knows exactly what specific functions the application uses and adds the code that is strictly necessary. However, using DLLs, we must distribute the entire library for very few functions that the (Figure 9) executable uses. In this case, you are wasting space and unnecessarily increasing application load times.

2.3. Check links with DLLs

When an executable is launched, for example Die.exe, all dynamic libraries linked to it are loaded into memory (if they don't already exist). If there are any problems while loading, the executable will fail to start and the operating system will display some kind of error.

Links on Windows

Windows will display a (Figure 10) error message when it cannot load a DLL associated with an executable.

If we want to see which DLLs are linked to an executable, we will use the dumpbin command.

dumpbin /dependents Die.exe

Dump of file Die.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    casino.dll
    KERNEL32.dll
    USER32.dll
    GDI32.dll
    SHELL32.dll
    COMDLG32.dll
    gdiplus.dll
    SHLWAPI.dll
    COMCTL32.dll
    UxTheme.dll
    WS2_32.dll

We see, at the beginning, the dependency with casino.dll. The rest are Windows libraries related to the kernel and the user interface. In the case that we make a static link of casino:

dumpbin /dependents Die.exe

Dump of file Die.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    KERNEL32.dll
    USER32.dll
    GDI32.dll
    SHELL32.dll
    COMDLG32.dll
    gdiplus.dll
    SHLWAPI.dll
    COMCTL32.dll
    UxTheme.dll
    WS2_32.dll

casino.dll no longer appears, having been statically linked inside Die.exe.

Links in Linux

In Linux something similar happens, we will get an error if it is not possible to load a dynamic library (*.so).

:~/$ ./Die
./Die: error while loading shared libraries: libcasino.so: cannot open shared object file: No such file or directory

To check which libraries are linked to an executable we use the ldd command.

~/$ ldd ./Die
linux-vdso.so.1 (0x00007fff58036000)
libcasino.so => libcasino.so (0x00007f6848bf4000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6848bba000)
libgtk-3.so.0 => /lib/x86_64-linux-gnu/libgtk-3.so.0 (0x00007f6848409000)
libgdk-3.so.0 => /lib/x86_64-linux-gnu/libgdk-3.so.0 (0x00007f6848304000)
libpangocairo-1.0.so.0 => /lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007f68482f2000)
libpango-1.0.so.0 => /lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007f68482a3000)
libcairo.so.2 => /lib/x86_64-linux-gnu/libcairo.so.2 (0x00007f684817e000)
libgdk_pixbuf-2.0.so.0 => /lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007f6848156000)
libgio-2.0.so.0 => /lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007f6847f75000)
libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f6847f15000)
libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f6847dec000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6847c9d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6847aa9000)
...

Where we see that Die depends on libcasino.so. The rest are dependencies of the Linux kernel, the C standard library, and GTK.

Links on macOS: We use the otool command.

% otool -L ./Die.app/Contents/MacOS/Die
@rpath/libcasino.dylib
/System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa
/System/Library/Frameworks/UniformTypeIdentifiers.framework/Versions/A/UniformTypeIdentifiers
/usr/lib/libc++.1.dylib
/usr/lib/libSystem.B.dylib
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
/usr/lib/libobjc.A.dylib

2.4. Loading DLLs at runtime

Until now, the importation of DLL symbols is resolved at compile time, or rather at link time. This means that:

• Executables can directly access global variables and functions defined in the DLL. Returning to the code of Dice.exe, we have:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

#include "ddraw.h" ... static void i_OnRedraw(App *app, Event *e) { const EvDraw *params = event_params(e, EvDraw); color_t green = color_rgb(102, 153, 26); real32_t w = params->width / 3; real32_t h = params->height / 2; real32_t p = kDEF_PADDING; real32_t c = kDEF_CORNER; real32_t r = kDEF_RADIUS; draw_clear(params->ctx, green); die_draw(params->ctx, 0.f, 0.f, w, h, p, c, r, app->face[0]); die_draw(params->ctx, w, 0.f, w, h, p, c, r, app->face[1]); die_draw(params->ctx, 2 * w, 0.f, w, h, p, c, r, app->face[2]); die_draw(params->ctx, 0.f, h, w, h, p, c, r, app->face[3]); die_draw(params->ctx, w, h, w, h, p, c, r, app->face[4]); die_draw(params->ctx, 2 * w, h, w, h, p, c, r, app->face[5]); }

• Made a #include "ddraw.h", public header of casino.
• die_draw(), kDEF_PADDING, kDEF_CORNER, kDEF_RADIUS have been used.
• The dynamic library casino.dll will be loaded automatically just before Dice.exe.
• Using a static or dynamic version of casino does not imply changes to the Dice code. We would only have to change the /casino/CMakeLists.txt and recompile the solution.

# Static library nap_library(casino "" NO NRC_NONE) target_include_directories(casino PUBLIC "${NAPPGUI_INCLUDE_PATH}") # Dynamic library nap_library(casino "" YES NRC_NONE) target_include_directories(casino PUBLIC "${NAPPGUI_INCLUDE_PATH}") target_link_libraries(casino ${NAPPGUI_LIBRARIES})

However, there is the possibility that the programmer is in charge of loading, unloading and accessing the symbols of the DLLs at any time. This is known as run-time binding or symbol-less binding. At demo/dice2 we have a new version of Dice:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

typedef void(*FPtr_ddraw)(DCtx*, const real32_t, const real32_t, const real32_t, const real32_t, const real32_t, const real32_t, const real32_t, const uint32_t);

static void i_OnRedraw(App *app, Event *e)
{
    const EvDraw *params = event_params(e, EvDraw);
    DLib *casino = dlib_open(NULL, "casino");
    FPtr_ddraw func_draw = dlib_proc(casino, "die_draw", FPtr_ddraw);
    color_t green = color_rgb(102, 153, 26);
    real32_t w = params->width / 3;
    real32_t h = params->height / 2;
    real32_t p = *dlib_var(casino, "kDEF_PADDING", real32_t);
    real32_t c = *dlib_var(casino, "kDEF_CORNER", real32_t);
    real32_t r = *dlib_var(casino, "kDEF_RADIUS", real32_t);
    draw_clear(params->ctx, green);
    func_draw(params->ctx, 0.f, 0.f, w, h, p, c, r, app->face[0]);
    func_draw(params->ctx, w, 0.f, w, h, p, c, r, app->face[1]);
    func_draw(params->ctx, 2 * w, 0.f, w, h, p, c, r, app->face[2]);
    func_draw(params->ctx, 0.f, h, w, h, p, c, r, app->face[3]);
    func_draw(params->ctx, w, h, w, h, p, c, r, app->face[4]);
    func_draw(params->ctx, 2 * w, h, w, h, p, c, r, app->face[5]);
    dlib_close(&casino);
}

• Line 6 loads the casino_d library.
• Line 7 accesses the die_draw function defined in casino_d.
• Lines 11-13 access public variables of casino_d.
• Lines 15-20 use die_draw via the func_draw pointer.
• Line 21 unloads the casino_d library from memory.

As we can see, this loading at runtime does imply changes to the source code, but it also brings with it certain advantages that we can take advantage of.

• The library is loaded when we need it, not at the start of the program. This is why it is very important that casino_d does not appear as a dependency of Dice2.

nap_desktop_app(Dice2 "" NRC_NONE)

• We can have different versions of casino and choose which one to use at runtime. This is the working mechanism of the plug-ins used by many applications. For example, the program Rhinoceros 3D enriches its functionality thanks to new commands implemented by third parties and added at any time through a system of plugins (.DLLs) (Figure 11).

2.5. Location of DLLs

When the operating system must load a dynamic library, it follows a certain search order. On Windows systems it searches in this order:

• The same directory as the executable.
• The current working directory.
• El directorio %SystemRoot%\System32.
• The %SystemRoot% directory.
• The directories specified in the PATH environment variable.

On the other hand, on Linux and macOS:

• The directories specified in the environment variable LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (macOS).
• The directories specified in the rpath executable.
• The system directories /lib, /usr/lib, etc.

Here we have a big difference between Windows and Unix, since in the latter it is possible to add dependency search directories inside the executable. This variable is known as RPATH and is not available on Windows. To query the value of the RPATH:

// In Linux
~/$ readelf -d ./Die | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [${ORIGIN}]

 // In macOS
otool -l ./Die.app/Contents/MacOS/Die
...
Load command 25
          cmd LC_RPATH
      cmdsize 40
         path @executable_path/../../.. (offset 12)
...

Executables generated by NAppGUI's CMakeLists.txt automatically set the RPATH to find dynamic dependencies in the same directory as executables on Linux or bundles on macOS.

3. Symbols and visibility

In the linking process after the compilation of the library, those elements that can generate machine code or occupy space in the final binary are called symbol. These are methods, functions, and global variables. Symbols are not considered:

• Type definitions such as enum, struct, or union. They help the programmer to organize the code and the compiler to validate it, but they do not generate any binary code. They do not exist from the point of view of the linker.
• Local variables. These are automatically created and destroyed in the Stack Segment during program execution. They do not exist at link time.

On the other hand, all functions and global variables declared as static inside a *.c module will be considered private symbols not visible in link time and where the compiler is free to perform optimizations. With this in mind, the code within NAppGUI is organized as follows:

• *.c: Implementation file. Definition of symbols (functions and global variables).
• *.h: Public header file. Declaration of global functions and variables (extern), available to the user of the library.
• *.hxx: Declaration of public types: struct, union and enum.
• *.inl: Declaration of functions and private variables. Only modules internal to the library will have access to these symbols.
• *.ixx: Declaration of private types. Those shared between the modules of the library, but not with the outside.

If a function is only needed inside a *.c module, it is not included in a *.inl. It will be marked as static within the *.c itself. This way it will not be visible to the linker and will allow the compiler to perform optimizations.

In the same way, types that are only used within a specific module will be declared at the beginning of the *.c and not in the *.ixx.

In favor of code maintainability and scalability, type and function declarations will be kept as private as possible.

3.1. Export in DLLs

When we generate a dynamic link library, in addition to including the public symbols in one or more *.h headers, we must explicitly mark them as exportable. The export macro is declared in the *.def file of each library. For example in casino.def, the macro _casino_api is defined.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

/* casino library import/export */

#if defined(NAPPGUI_SHARED)
    #if defined(NAPPGUI_BUILD_CASINO_LIB)
        #define NAPPGUI_CASINO_EXPORT_DLL
    #else
        #define NAPPGUI_CASINO_IMPORT_DLL
    #endif
#endif

#if defined(__GNUC__)
    #if defined(NAPPGUI_CASINO_EXPORT_DLL)
        #define _casino_api __attribute__((visibility("default")))
    #else
        #define _casino_api
    #endif
#elif defined(_MSC_VER)
    #if defined(NAPPGUI_CASINO_IMPORT_DLL)
        #define _casino_api __declspec(dllimport)
    #elif defined(NAPPGUI_CASINO_EXPORT_DLL)
        #define _casino_api __declspec(dllexport)
    #else
        #define _casino_api
    #endif
#else
    #error Unknown compiler
#endif

This macro must precede all public functions and variables declared in the *.h of the library. Projects based nap_desktop_app() will define the macros NAPPGUI_XXXXX_EXPORT_DLL when the DLL is compiled and NAPPGUI_XXXXX_IMPORT_DLL when the DLL is used in other targets. This way, the export and import of symbols will be done correctly on all platforms.

3.2. Checking in DLLs

We can see, from a dynamic library binary, which public symbols it exports. On Windows we will use dumpbin /exports dllname, on Linux nm -D soname and on macOS nm -gU dylibname.

C:\>dumpbin /exports core.dll
2    1 00001000 array_all
3    2 00001010 array_bsearch
4    3 00001090 array_bsearch_ptr
5    4 00001120 array_clear
6    5 000011C0 array_clear_ptr
7    6 00001260 array_copy
8    7 00001340 array_copy_ptr
9    8 00001420 array_create
10    9 00001430 array_delete
11    A 00001530 array_delete_ptr
12    B 00001640 array_destopt
13    C 00001650 array_destopt_ptr
14    D 00001660 array_destroy
15    E 000016F0 array_destroy_ptr
16    F 00001790 array_esize
17   10 000017A0 array_find_ptr
18   11 000017D0 array_get
...

$ nm -D ./libcore.so
0000000000011f85 T array_all
000000000001305c T array_bsearch
000000000001316d T array_bsearch_ptr
0000000000011832 T array_clear
00000000000118a1 T array_clear_ptr
0000000000011009 T array_copy
000000000001115d T array_copy_ptr
0000000000010fdd T array_create
0000000000012649 T array_delete
000000000001276b T array_delete_ptr
0000000000011668 T array_destopt
0000000000011746 T array_destopt_ptr
00000000000115c3 T array_destroy
00000000000116ad T array_destroy_ptr
0000000000011b87 T array_esize
0000000000012dd3 T array_find_ptr
0000000000011e8c T array_get

% nm -gU ./libcore.dylib
00000000000029f0 T _array_all
0000000000003c90 T _array_bsearch
0000000000003d60 T _array_bsearch_ptr
00000000000024c0 T _array_clear
00000000000025d0 T _array_clear_ptr
0000000000001c20 T _array_copy
0000000000001dd0 T _array_copy_ptr
0000000000001b50 T _array_create
00000000000030f0 T _array_delete
0000000000003350 T _array_delete_ptr
00000000000022f0 T _array_destopt
0000000000002470 T _array_destopt_ptr
0000000000002120 T _array_destroy
0000000000002340 T _array_destroy_ptr
00000000000028b0 T _array_esize
0000000000003980 T _array_find_ptr
00000000000028f0 T _array_get