nbuild

nbuild is the continuous integration (CI/CD) system used daily to build and test NAppGUI-SDK. It has been developed in ANSI-C90, using the NAppGUI-SDK itself. In principle it is not a general purpose utility, rather it has been created ex profeso to cover the specific needs of this project. However, with the necessary evolution and extensions, it can support any C/C++ project that is based on CMake.

nbuild -n network.json -w workflow.json -s secrets.json

• network.json: Node network configuration. See Red de compiladores.
• workflow.json: List of tasks to be performed. See Workflows.
• secrets.json: Contains sensitive values, such as passwords, that cannot be hosted in conventional repositories. See Secrets.

1. Motivation and features

The development of nbuild began in 2017 as a small utility to launch remote build commands via ssh. Over time it has evolved, taking on characteristics of conventional CI/CD systems such as Jenkins, TeamCity or GitHub Actions. There are several reasons why NAppGUI does not use these CI/CD systems but they can be summarized in three: Deployment complexity, avoiding scripting and guaranteeing lifetime operation on legacy platforms. All this, together with the ease and power that the C+ssh tandem provides to implement any current or future need, has been enough motivation to improve this tool over the years. The main features of nbuild can be summarized as:

• Compiling, testing and packaging C/C++ projects based on CMake.
• Oriented to Open-Source projects, where the source code can be part of the final product.
• Minimal configuration on the nodes (runners) where it is only necessary to install SSH, CMake and the compilers (GCC, Xcode, MSVC or Clang).
• Does not require cloud support (AWS, Artifactory, GitHub, etc). You also don't need an Internet connection. It operates on a private local area network.
• Does not use Ansible or any other remote access tool. SSH only.
• No need to coding scripts. Just indicate which targets we want to compile, within workflow.json.
• There are no control panels or web interface. Runs as a command line utility.
• Each project can define its own network of nodes.
• There is no need to pay to compile.
• Works with "legacy" nodes such as Ubuntu 12, Windows XP or Mac OS X Snow Leopard.
• Multi-stage. A relatively large workflow can be divided into several stages of different priority, obtaining partial results as soon as possible.
• Multithread. Control different nodes in parallel.
• Scheduler. Match jobs with available nodes.
• Redundancy. It is possible to configure several nodes with the same characteristics (OS, compiler version, etc.), in order to accelerate processes that require the same platform.
• Non-blocking. At no time does running nbuild interfere with or block new changes to the repository.
• nboot. Ability to turn runners on/off on demand, whenever they are needed.
• ndoc. Generation of documentation associated with the project.
• nreport. Generate web reports with the result of each execution.

2. Flows in build

nbuild executes Workflows described in workflow.json files, which include the project description, the list of targets and the target platforms. This flow will restart every time changes are detected in the repository, even if there are tasks left to be done from previous flows. That is, the goal of nbuild is not to complete all tasks in a flow, but rather to complete the highest priority tasks as soon as possible with the most recent version of the repository. In summary, a workflow is made up of (Figure 1):

• Code packaging. nbuild does not work directly with the repository source code. Previously, a selection, pre-processing and packaging of those files required by the flow will be carried out, storing the results in the drive node. This source package would be ready to be distributed, either through GitHub or directly in a .zip.
• Documentation. If the project has documentation, and it contains changes, a new version of the documentation will be created and stored in the drive node.
• Compile/test. The source package will be compiled and tested on different nodes in the network, in order of priority.
• Reports. Each run of nbuild will be accompanied by a web report. Every time the state of the repository changes, for example, adding a new commit, a new report will open. The same report will be updated as work is completed within the same commit.
• Hosting. Both project documentation and reports can be uploaded to a public server, making them easier to consume.

3. Run nbuild

To launch the CI/CD system you must execute the nbuild command manually or through a unix-cron administrator. There is no event, hook or trigger that automatically activates the system based on changes in the repository. This CI/CD has been designed to complete the highest priority tasks as quickly as possible, so successive runs of nbuild will cooperate in resolving the workflow defined in workflow.json (Figure 2).

In stages where the code changes frequently, nbuild will focus on those tasks with the highest priority. With this we aim to detect as soon as possible any errors introduced in the latest commits. On the contrary, in stages of low activity (weekends), the system will gradually complete low priority tasks, until covering the entire spectrum of compilations and tests existing in workflow.json.

Executing nbuild will only perform a few tasks, those with the highest priority within the flow. The project administrator is in charge of scheduling how frequently the CI/CD should be executed.

Only one nbuild instance can be running at a time. If one run is launched while another is running, the second will terminate without doing anything.

4. Licencia

At the moment, nbuild is a closed source tool for proprietary use.