Red de compiladores

When running nbuild:

nbuild -n network.json -w workflow.json -s secrets.json

The first parameter network.json indicates the configuration of the network of nodes responsible for generating and testing the project. In this network we highlight three types of (Figure 1) nodes:

• Master: It is the node where the nbuild program runs. Coordinates the rest of the nodes and executes the workflows. You do not need a machine with large capacities, since it will only perform coordination tasks. It does not use local disk storage and does not need to be included in network.json.
• Drive: It is the storage node. It will contain the collection of packaged source files, binaries, reports and execution logs. All of this will be organized in folders representing the states (or versions) of the repository. It is recommended that this node has a high-capacity SSD device.
• Host: Build node (or runner). It will have the compilers and CMake pre-installed. The master node will activate these machines depending on the jobs it must perform, indicated in workflow.json. It is not necessary for these nodes to have large storage capacity, since they do not store history or "garbage" between different compilations. They only handle temporary directories with partial results.

1. Content of network.json

network.json includes the configuration of the drive machine as well as each and every build host (Listing 1). As we already indicated, the master node does not have to be included, since it is where the nbuild process itself runs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

{
"drive" : {
    "name" : "RPi 4 Raspbian 10 Buster",
    "path" : "/media/pi/builds",
    "login" : {
    "ip" : "192.168.1.5",
    "user" : $SECRET{drive_user},
    "pass" : $SECRET{drive_pass},
    "platform" : 3 }
},

"hosts" : [
{
    "name" : "Ubuntu 24.04 VBox",
    "workpath" : "/home/fran",
    "login" : {
        "ip" : "192.168.1.32",
        "user" : $SECRET{host_user},
        "pass" : $SECRET{host_pass},
        "platform" : 3 },
    "type" : "vbox",
    "vbox_uuid" : "288d134e-23e3-4546-acc2-ee04056f4c7b",
    "vbox_host" : "i7-13700-WIN11",
    "generators" : ["Unix Makefiles", "Ninja"],
    "tags" : ["ubuntu24", "x64"]
},
...
] }

• name: Name of the machine. It must be unique.
• login::ip: IP address.
• login::user: User with which to connect to the remote machine.
• login::pass: Password.
• login::platform: Machine operating system: 1 Windows, 2 macOS, 3 Linux.
• path: Only on drive. Base directory for storing artifacts.

The following parameters only exist for host machines:

• workpath: Temporary directory for build and test tasks.
• type:
• metal for physical machines.
• vbox for VirtualBox machines.
• macos for macOS machines. These machines support multiple boot disks with different operating systems and configurations.
• vbox_uuid: Virtual machine code (only for vbox).
• vbox_host: Name of the machine that hosts the virtual one (only for vbox). It must be of type metal.
• macos_host: Name of the macOS machine (only for macos).
• macos_volume: Name of the boot volume connected to the machine macos_host (only for macos).
• generators: List of CMake generators supported on the node. "Unix Makefiles", "Ninja", "Xcode", "Visual Studio 17 2022", etc.
• tags: List of tags with the node characteristics. They will be used to know if this machine is suitable or not when executing a process.
• x86, x64, arm, arm64: Supported architectures.
• ubuntu24, ubuntu22, ubuntu20, ubuntu18, ubuntu16, ubuntu14, ubuntu12: Linux version.
• sequoia, sonoma, ventura, monterey, bigsur, catalina, mojave, highsierra, sierra, elcapitan, yosemite, mavericks, mountainlion, lion, snowleopard: macOS/MacOSX version.

Virtualized macOS machines are not supported at this time.

2. SSH protocol

nbuild uses the SSH (Secure Shell) protocol to coordinate and communicate the different machines on the network. The master node will issue ssh commands to the build hosts, and they will communicate with drive to send the binaries. Before starting to work with nbuild we must configure the network, which is nothing more than installing the RSA public keys on the hosts and drive:

• Each host needs to know the identity of master (its id_rsa.pub), to let it execute remote commands.
• Drive needs to know the identity of master and each host to authorize it to store files.

2.1. Generate and install RSA keys

To prevent ssh from constantly asking for passwords (a fact that is very bad for automation), we are going to install the master certificate on the (Figure 2) hosts.

• The first thing will be to create the certificate. From master, open a terminal and type ssh-keygen. This will generate two rsa certificates in .ssh, one private, which will remain in master, and another public, which will go to the host.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

fran~>ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/fran/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/fran/.ssh/id_rsa. Your public key has been saved in /home/fran/.ssh/id_rsa.pub. The key fingerprint is: SHA256:Up6KjbnEV4Hgfo75YM393QdQsK3Z0aTNBz0DoirrW+c fran@192.168.1.24 The key's randomart image is: +---[RSA 2048]----+ | . ..oo..| | . . . . .o.X.| | . . o. ..+ B| | . o.o .+ ..| | ..o.S o.. | | . %o= . | | @.B... . | | o.=. o. . . .| | .oo E. . .. | +----[SHA256]-----+

• Copy the file id_rsa.pub to the /USER/.ssh directory on each host.

1 2	cd .ssh scp id_rsa.pub fran@192.168.1.27:/home/fran/.ssh

• Open a terminal on the remote host and type:

1 2 3 4 5

cd .ssh type id_rsa.pub >> authorized_keys (Host Windows) ... cd .ssh cat id_rsa.pub >> authorized_keys (Host macOS/Linux)

• If everything has gone well, we can now execute an ssh command from master to host without being asked for a password. We do the test from master:

1 2 3 4 5 6 7 8 9 10

fran~>ssh fran@192.168.1.27 ls Desktop Documents Downloads Library Movies Music Pictures Public ...

2.2. Enable SSH

Both the host and drive nodes must have an SSH server installed and activated, otherwise they will not be able to accept remote commands.

SSH in Windows 11/10

Windows 11 includes support for SSH, but it is not active by default. You must activate it in Settings->Apps->Manage Optional Features->Add a Feature->OpenSSH Client/Server. To ensure which service is running run services.msc and start OpenSSH SSH Server if it is not already running.

SSH in macOS

SSH is also included in Apple operating systems. To activate it:

• Open System Preferences from the Apple menu and click the Sharing panel.
• Activate the Remote Login option.
• This will instantly start several services, including SFTP and SSH.

For security reasons, SSH does not access all bash shell environment variables on macOS. This can cause a remote command to fail, but one typed from a terminal on the machine itself to work. For example, this occurs when trying to run cmake or svn remotely. To correct it:

• Add the necessary routes to the PATH variable in /.ssh/environment.

1	PATH=$PATH:/bin:/usr/bin:/usr/local:/usr/local/bin:/Applications/CMake.app/Contents/bin

• We edit the file /private/etc/ssh/sshd_config or /etc/ssh/sshd_config, enabling the variable PermitUserEnvironment PATH,LANG.
• We stop and relaunch the service by checking Remote Login of Sharing.

SSH in Linux

By default, SSH is not included in Ubuntu. For this distribution and others based on Debian, the service is installed and activated with this command.

1

sudo apt-get install openssh-server -y