Workflows

When you run nbuild:

nbuild -n network.json -w workflow.json -s secrets.json

The second parameter workflow.json contains the workflow configuration.

1. Code packaging

The first task that will be carried out in an nbuild flow will be the selection, pre-processing and packaging of the source code (Figure 1). It is possible that a certain flow only needs to process a few targets to generate the final product, ignoring the rest of the repository. In this context we call target a collection of source files that generate a library or an executable. In large projects or organizations, a repository can contain an infinite number of interdependent targets, which serve as a basis for building different flows with which to manage different production lines. On the other hand, during this pre-processing, static analysis tools (cpp-check) or code formatting tools (clang-format) can be executed. This will give cohesion to all files, regardless of who or how they were edited. The source files will also be automatically edited, including headers with the project's legal information. We remember that for nbuild, the source code is just another artifact, and can be part of the final product. This phase will leave all sources ready to be published on GitHub or any other distribution platform.

2. workflow.json

In (Listing 1) we have the example of a workflow in Json format. The global object will contain the configuration parameters and targets, in turn, the array of libraries/executables necessary to complete the flow.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

{
"global" : {
    "project" :  "NAppGUI",
    "description" : "Cross-platform C SDK",
    "start_year" : 2015,
    "author" : "Francisco Garcia Collado",
    "license" : [
        "MIT Licence",
        "https://nappgui.com/en/legal/license.html" ],

    "flowid" : "nappgui_dev",

    "repo_url" : "svn://192.168.1.2/svn/NAPPGUI",
    "repo_branch" : "trunk",
    "repo_user" : $SECRET{repo_user},
    "repo_pass" : $SECRET{repo_pass},

    "doc_repo_url" : "svn://192.168.1.2/svn/NAPPGUI/nbuild/doc",
    "doc_repo_user" : $SECRET{repo_user},
    "doc_repo_pass" : $SECRET{repo_pass},
    "doc_url" : "https://nappgui.com",
    "doc_deflang" : "en",

    "hosting_url" : $SECRET{hosting_url},
    "hosting_user" : $SECRET{hosting_user},
    "hosting_pass" : $SECRET{hosting_pass},
    "hosting_cert" : false,
    "hosting_docpath" : "./web/docs",
    "hosting_buildpath" : "./web/builds"
    },

"targets" : [
    {"name" : "src/sewer", "legal" : true, "format" : true, "analyzer" : true },
    {"name" : "src/gui", "legal" : true, "format" : true, "analyzer" : true },
    ...
    {"name" : "demo/dice", "legal" : false, "format" : true, "group" : "NAPPGUI_DEMO" },
    ...
    ],

"paths" : [
    { "name" : "prj" },
    { "name" : "src/nappgui.h" },
    { "name" : "Changelog.md" },
    { "name" : "README.md" },
    { "name" : "LICENSE" }
    ],

"makefile" : "CMakeLists.txt",
"version" : "prj/version.txt",
"build" : "prj/build.txt",
"clang_format" : ".clang-format"
}

3. Global

The fields to include in the globals object are:

Basic data (mandatory):

• project: Project name.
• description: Brief description of the project (40-80 characters).
• start_year: Project start year.
• author: Author.
• license: Array of texts with information about the license.
• flowid: Flow identifier. nbuild can run multiple instances in parallel, as long as they work in different workflows. It is also used to create the main storage directories on the drive node drive::path::flowid.

Repository data (required):

• repo_url: Url of the repository. At the moment only Subversion is supported.
• repo_branch: Branch of the repository where we will download the code.
• repo_user: User of the repository.
• repo_pass: Repository password.

Project documentation (optional): If the project has documentation in ndoc format and we want to generate it, we must provide the following information:

• doc_repo_url: Url of the repository that contains the documentation. At the moment only Subversion is supported.
• doc_repo_user: User of the documentation repository.
• doc_repo_pass: Documentation repository password.
• doc_url: Url of the site where the documentation is published. If omitted, a public site is considered not to exist and the documentation is distributed packaged.
• doc_deflang: Default language in public documentation portal. ndoc supports documents in several languages.

Web hosting data (optional): Both the project documentation generated by ndoc and the reports on the results of each session are stored in the drive node. (see Drive artifact management). If we want to publish it on a Web server in order to have it more accessible, we must provide the hosting data where said pages and resources will be hosted.

• hosting_url: Hosting access url.
• hosting_user: Hosting user.
• hosting_pass: Hosting password.
• hosting_cert: true if the hosting uses the RSA certificate of the master node.
• hosting_docpath: Path within the hosting where the project documentation is hosted.
• hosting_buildpath: Path within the hosting where the nbuild reports are hosted.

4. Targets

Once the general configuration of the project has been established, we are going to define the targets that we are going to include within the flow. We remember that a target is a compilation project that generates a binary artifact (a library or an executable). In workflow.json, in the target array we must provide these fields for each entry:

• name: Path of the target's main folder, starting from the root of the repository.
• legal: true if we want to add legal or authorship information to each target source file.
• format: true if we want to apply code formatting (clang-format) to each target source file.
• analyzer: true if we want to run the static analyzer (cpp-check) on each target source file.
• group: Group to which the target belongs (optional). See CMakeLists.txt.

If we have selected true in the format field, this information will be added as the header of each target source file, generated from the project data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

/*
 * NAppGUI Cross-platform C SDK
 * 2015-2024 Francisco Garcia Collado
 * MIT Licence
 * https://nappgui.com/en/legal/license.html
 *
 * File: button.h
 * https://nappgui.com/en/gui/button.html
 *
 */

For .h, .hpp or .hxx headers, if we have configured a public url for the documentation (doc_url), a link will be included in the source file to the documentation page.

5. Other files

The paths entry of workflow.json indicates files or directories from the original repository that must be packaged together with the source code (configuration, resources, etc.). Generally, they are files necessary for the project to compile, without being source code. A direct, raw copy will be made from the repository to the generated package.

• name: Path of the file or directory, starting from the root of the repository to copy in the packaging.

6. CMakeLists.txt

The makefile entry of workflow.json indicates the path within the repository to the CMakeLists.txt file needed to compile the flow. This file will be installed in the main packaging folder. We must ensure that it meets these two requirements:

• Installation: You must include the install() commands necessary to perform a correct installation of the project when executing the cmake --install command. As we will see later, nbuild installs the project once compiled on any host node, before launching any execution or testing task.
• Support target list: nbuild will create a file CMakeTargets.cmake (Listing 2) with those entries included in the targets array from workflow.json. Therefore, the CMakeLists.txt must reference this file, avoiding calling add_subdirectory() directly (Listing 3). If we have indicated a group when defining the target, its generation will be conditional on a CMake option (for example, NAPPGUI_DEMO).

1 2 3 4 5 6 7 8 9

<c>CMakeTargets.cmake</c>. # CMakeTargets.cmake automatic generated by nbuild set(ALL_TARGETS "") set(ALL_TARGETS ${ALL_TARGETS};src/sewer;src/osbs;...;src/gui) if (NAPPGUI_DEMO) set(ALL_TARGETS ${ALL_TARGETS};demo/dice;...) endif()

1 2 3 4 5 6 7

... # Generate targets include(${NAPPGUI_ROOT_PATH}/CMakeTargets.cmake) foreach (target ${ALL_TARGETS}) add_subdirectory(${target}) endforeach() ...

7. Additional data

Finally, these entries are optional within the workflow.json but it is recommended to include them.

• version: File within the repository that indicates the version of the flow. You just have to include a line with the format major.minor.path (1.5.6).
• build: Path within the final packaging that will contain the version of the repository with which the flow has been generated.
• clang_format: Path within the repository to the .clang-format file with which to format the source files. Different streams can use different formats for the same source code.

8. Drive artifact management

This section details the file and directory structure in the drive node.