Overview

The Goupile server is developed in C++. The compiled binary directly includes the database engine (SQLite), an HTTP server, as well as the HTML/CSS/JavaScript code sent to web browsers.

The web client is developed in HTML, CSS, and JavaScript. Page updates via JavaScript code make extensive use of lit-html templates. All JavaScript and CSS code is bundled and minified using esbuild. Static files (images, etc.) and files generated by esbuild are embedded in the Goupile server binary at compile time.

Databases

Several SQLite databases are created and used for each domain. First, there is a master database that contains the list of projects, users, and permissions. Then, each project uses one or more databases (one for the project + one per site in case of a multi-site project). For example, a domain with two projects, one of which is multi-site, could use the following files:

goupile.db # Main database
instances/projet1.db
instances/projet2.db
instances/projet2@lille.db
instances/projet2@paris.db

Service Isolation

Each domain is managed by a dedicated service (for example, launched by systemd), which can self-containerize on Linux (using POSIX capabilities, namespaces, and seccomp filters) in a namespace with virtually no access other than to the SQLite files.

This process still retains the ability to fork, and a child process is created to handle each project within the domain. This ensures isolation of each project in its own dedicated process, responsible for handling HTTP requests (static files and API) for that project.

Compilation Options

In addition to containerization, several Clang compilation options are used to mitigate server vulnerabilities in case of exploits. When compiling Goupile (as described later), this is called Paranoid mode.

Several measures are aimed at preventing stack corruption or control-flow hijacking attacks:

• Stack Smashing Protection (and Stack Clash Protection) to limit stack corruption attacks
• Safe Stack to reduce ROP-style attacks
• PIE compilation for ASLR support (complementing the previous measure)
• CFI (Control Flow Integrity): coarse-grained forward-edge protection
• Linker options: relro, noexecstack, separate-code

In addition, during development we use various sanitizers (ASan, TSan, and UBSan) to detect memory access errors, multi-threading issues, and undefined behavior in C/C++.

Data Format

Project form scripts are stored and versioned within the SQLite databases.

Data entered in a project is stored in the corresponding SQLite database (for multi-site studies, each site has a separate database). Two SQLite tables are used for data:

• rec_threads table: contains one row per record
• rec_entries table: contains one row per record page, with summary information
• rec_fragments table: contains all historical modifications of a record page (audit trail)

The primary key of a record follows the ULID format. This allows record IDs to be generated on the client side (with an infinitesimal risk of collision), simplifying offline mode implementation while avoiding the performance issues of UUIDv4 indexing.

Data Validation

Data validity checks against constraints occur on both the client side (systematically) and the server side only when requested by the project administrator with Batch permission.

To perform data validation on the server in batch mode, Goupiel forks a zygote process that runs along it. When data validations needs to happen, a request with the necessary data is sent to the zygote process, which forks itself and runs the scripts in a highly constrained namespace, without access to the main file system, and restricted permissiond and system calls (seccomp).

These checks rely on the JavaScript code of each page, which can define conditions and errors based on the entered data. Errors are logged in the metadata alongside each record data.

Run Goupile (debug)

To work on Goupile, use debug builds. Start by cloning the main repository, and build the build tool (felix) with the bootstrap script.

git clone https://codeberg.org/Koromix/rygel.git
./bootstrap.sh

After this is done, you can use felix to build and directly run the goupile binary. You must start by creating a new domain, and then run it, with the following commands:

cd rygel
mkdir tmp

./felix --run goupile init tmp/domain # Create the domain
./felix --run goupile -C tmp/domain   # Run the domain

You can then open Goupile with your browser, by default it runs on port 8889, so the address is http://localhost:8889/. Access the admin panel at http://localhost:8889/admin/.

More information about compilation is available in the self-hosting page.