GitLab#

Getting Help and Administrators#

If you need help with the GitLab, you can ask

Namespaces#

Each user has a personal namespace in the CTAO gitlab. Additionally, there are groups which also may contain subgroups.

Contrary to how you might intuitively think about groups in an organization, GitLab subgroups always contain all members of their parent groups, you might only add additional people compared to the parent group or raise the permission level of users compared to the default permissions.

Organizing Projects#

CI/CD Systems#

To build, test and run code, CTAO has access to two CI systems: GitLab CI/CD and Jenkins.

GitLab CI/CD#

GitLab CI/CD is directly integrated with GitLab, however to actually run CI/CD jobs, a project needs access to CI runners.

DESY provides CI runners based on the Kubernetes executor, meaning it executes CI jobs on Linux in Docker containers.

Two runners are provided and are available to all projects:

  • Default runner, tagged DESY-IT and kubernetes. This runner will be used if no tags are specified in the .gitlab-ci.yaml. This runner has a fixed resource limit of 2 CPUs and 4 GB of memory.

  • A runner with more computing resources, tagged hi-load. The default resources are 4 CPUs and 8 GB of memory. This can be increased by setting the KUBERNETES_CPU_LIMIT and KUBERNETES_MEMORY_LIMIT variables to 10 CPUs and 20 GB of memory respectively.

Note that jobs still see all physical cores and need to “behave themselves”. If they use more than the allocated resources, the job will be killed. See Limiting resources used by CI Jobs for some guidance on how to make sure that CI pipelines stay within these limits.

For security reasons, CI jobs using this runner cannot run arbitrary docker commands. Docker images can be build and uploaded into a registry using Kaniko. Example configuration can be found in the python-template-repository.

By default, CI pipelines run on push events to the repository. It is recommended to build on pushes to the main branch, merge request builds, tag creation and manual triggering of a pipeline. A configuration example for this can be found in both the cpp-template-repository and the python-template-repository.

The CI/CD of projects should:

  • Perform static checks of the code, including for the coding style

  • Build the project, including documentation

  • Run the unit tests

  • Upload coverage report and trigger the analysis of the quality gate (Sonarqube) server.

  • Building docker containers

Additional steps, e.g. for release builds could be:

  • Publish the documentation to gitlab pages

  • Publishing build artifacts, e.g. uploading python packages to PyPI and tagged containers to a registry.

Refer to the template repositories for example configurations.

Limiting resources used by CI Jobs#

Aside from command-line switches like make -j<N>, there are several environment variables that can be set to limit the number of parallel threads a specific software can use. It is recommended to set these to the CPU limit or slightly above if the tasks are not expected to be CPU bound. E.g. a rule of thumb for compilation is to ask for parallel jobs = 1.5 x CPUs.

Limiting memory is harder to steer, but e.g. for compilation also usually scales with the number of parallel tasks.

Below is a table of relevant environment variables for common software.

Variable

Explanation

CMAKE_BUILD_PARALLEL_LEVEL

Number of threads to use when building a project using cmake --build

OMP_NUM_THREADS

Number of threads to use for openmp, this affects e.g. numpy

NUMBA_NUM_THREADS

Number of threads for numba parallel computing

MKL_NUM_THREADS

Number of threads of Intel MKL parallel computing

Jenkins#

TODO, contact ACADA