Change Log#

aivkit 5.1.3 (2026-06-01)#

API Changes#

Bug Fixes#

  • Clear all subcharts before every deployment in upgrade test. Fixes a problem which occurs when a subchart can not be rendered without overriding some values. [!448]

New Features#

Maintenance#

Refactoring and Optimization#

aivkit v5.1.2 (2026-05-26)#

Maintenance#

  • Update document references for DPPS release reports. [!447]

aivkit v5.1.1 (2026-05-21)#

Bug Fixes#

  • The ci-changelog step now works by using towncrier check, which ensures the step properly uses the towncrier configuration. [!446]

aivkit v5.1.0 (2026-05-08)#

Bug Fixes#

  • Remove aiv-config.yml from ci-functions.yml, disabling defaults it set for variables (we expect that no subsystems relied on this). [!429]

  • aiv install now also works in a repository without .gitlab-ci.yml [!435]

New Features#

  • Allow to apply project CRDs before an upgrade by adding two variables:

    • APPLY_PRE_UPGRADE_CRDS: if set to true, finds all yaml files in crds path and applies them just the for an upgrade.

    • APPLY_PRE_UPGRADE_URL: if set to a non-empty value, runs kubectl apply on the content of this variable them just the for an upgrade. [!431]

  • Call helm-docs to create a chart README.md in the ci-publish.yaml step that packages helm charts. [!436]

aivkit v5.0.2 (2026-04-15)#

Bug Fixes#

  • Fix aiv docker install issue related to pinned python version. [!425]

New Features#

  • Update default DPPS release to 0.7.0. [!420]

  • When testing an upgrade, fetch commits for cherry-pick from origin, in case they are not on the current branch. [!425]

  • Allow to apply pre-upgrade patches with patch files, not cherry-picking commits. [!427]

aivkit v5.0.0 (2026-04-02)#

API Changes#

  • Remove support for multiple SonarQube servers. This requires to replace the old SONAR_API_TOKEN_*=<host>,<token> secret with a single SONAR_API_TOKEN=<token> secret in the CI/CD settings of the repository. [!409]

Bug Fixes#

  • Hadolint is now always run and uses the correct way to find the dockerfile. [!386]

  • helm-dev command now takes into account CHART_VALUES (–values-file) override, as it should, allowing to specify custom values file. [!388]

  • Fix a missing space when concatenating extra helm values in aiv-deploy helm-dev. [!394]

  • Treat correctly charts with symlinks in them (like dpps-iam). [!395]

  • Fix pre-commit hook not always running and thus missing some required updates to .gitlab-ci.yml. [!401]

New Features#

  • The aivkit now always installs a haproxy ingress controller into the kind cluster.

    The DNS rules inside the kind cluster are adjusted so that all domains *.test and *.test.local resolve as CNAMEs to the ingress controller. This allows reaching the ingress controller internally to test defined ingress rules. [!290]

  • Also update sonar-project.properties when running aiv install. [!385]

  • Use custom docker image for test job to avoid repeatedly installing tools at job run time. [!389]

  • Increase the default node pod limit in kind clusters to 250 from the default 110. [!391]

  • Adapt pypi workflow to use trusted publishing, which is now available for publishing from the CTAO GitLab to the CTAO PyPI organization. [!399]

  • Switch default tag to desy-testcluster-ctao-computing-aiv, choosing by default common AIV cluster. All repositories using the toolkit will now switch by default to new and much faster test cluster for kubernetes testing. If any custom overrides of runner tags were made, they will not be affected. [!412]

Maintenance#

  • Add section in the documentation about using private images from harbor in CI pipelines. [!390]

  • Update default DPPS release version for report to 0.6. [!414]

Refactoring and Optimization#

aivkit v4.0.0 (2026-01-29)#

API Changes#

  • As part of the move to the common toolkit for DPPS and SUSS, the environment variable in CI DPPS_AIV_TOOLKIT_DIR has been changed to AIV_TOOLKIT_DIR and the default location is now $CI_PROJECT_DIR/aiv-toolkit. [!353]

  • Make the computing system configurable. This change removes all hard-coded instances of “DPPS” from the code and introduces the new configuration variable AIV_SYSTEM, which should be set to DPPS or SUSS for the corresponding projects.

    Variables that started with DPPS_ should be replaced by removing the DPPS_ prefix. [!357]

New Features#

  • Add a reusable job testing upgrade. [!210]

  • Allow to read release plan from remote URL, reducing the number of submodules. [!340]

  • Add a CLI command to build dev image. [!340]

  • Publish the toolkit chart to harbor. This allows to install it to subsystems without accessing the git repository. [!340]

  • Use pixi task to build documentation. [!342]

  • Work without aiv-config.yml: needed when installing kit. [!351]

  • Add aiv install command, installing or updating aiv toolkit into a repository. [!351]

  • The toolkit now supports getting jama information via the new jama-gateway, which allows read-only access to Jama’s rest API using general credentials. To use it, configure the CI secrets JAMA_GATEWAY_USER and JAMA_GATEWAY_PASSWORD, ask the AIV team to get the necessary credentials. [!356]

  • In the container build job, allow to override the default Dockerfile with AIV_DOCKERFILE variable. [!367]

  • Install LFS in pypi publish job. This allows to compute version correctly in repositories using LFS. [!368]

  • The CI pipeline now automatically detects the previous release tag and uses it by default for the k8s upgrade tests. The detected tag can still be overridden by setting UPGRADE_BASE_REF. [!371]

Maintenance#

  • Refactor config loading for aiv-deploy, allowing to load defaults from env variables and config.

    Adding additional mechanism for loading config motivated removing almost all default config values within Makefile. Makefile now serves only as a mechanism to establish dependencies between aiv-deploy CLI tasks. [!340]

  • Remove all references and links to old sonar, which is now decommissioned. [!343]

aivkit v3.3.0 (2025-12-15)#

API Changes#

  • Verify images used in the deployment to make sure they are pulled from harbor (with some exceptions). [!293]

Bug Fixes#

  • The error when project is not found in sonar is now more comprehensive. [!332]

  • Fix CWL docker image hint patching for Workflows. Patching is now performed only for command line tools. [!383]

New Features#

  • Allow to randomize release names, making sure that they are not explicitly assumed in the code, simplifying eventual integration and deployment. [!329]

  • Add overview table of sonarqube quality gates. [!318]

  • Automatically detect platform, allowing to support dev environment in Mac. [!326]

  • Remove all test pods before upgrade. This allows to directly upgrade the deployment with failed tests. [!321]

  • If job fails during first attempts but eventually succeeds, there failed pods are now removed, allowing to complete with clean successful deployment. Failed pods are reported as a warning. [!332]

  • Retry pulling images to be resilient against temporary failures. [!322]

  • Add an example GUI test with playwright. [!323]

  • Add optional autocomplete for AIV-deploy. [!327]

  • Switch dev test to python, preserving make interface. [!327]

  • cert-generator grid now stores the hash-based alias symlinks for the ca certificate and the crl in the corresponding kubernetes secret. [!331]

  • Add an option to the ci-docs step to install dependencies using pixi. Set PYTHON_INSTALL_METHOD=pixi in your variables to use pixi instead of pip. [!339]

aivkit v3.2.0 (2025-10-07)#

Bug Fixes#

  • Fix sonarqube quality gate reporting raising a KeyError “actualValue” in some cases. [!314]

New Features#

  • Update default DPPS release to v0.4.0 [!313]

aivkit v3.1.0 (2025-09-25)#

API Changes#

  • Remove variable SONAR_HOST_URL from ci-sonar.yml. Projects should set the host url in sonar-project.properties if they do not already do so. [!248]

  • Introduce restriction on required python version above 3.12. [!287]

  • List of UCs for given DPPS release can now be read from Jama. [!289]

Bug Fixes#

  • Fix a bug that if during report generation it is not possible to fetch sonar results, the resulting latex does not compile. Instead, it should compile and show an error. [!288]

New Features#

  • Allow to specify CHART_VALUES environment variable specifying an alternative Helm chart values file. This feature can be used to test different deployment scenarios, for example lighter more focused ones. [!256]

  • Add proxy registry definitions for quay.io and ghcr.io. [!281]

  • Each DPPS AIV Toolkit version now contains an aligned version of DPPS release used for determining UCs and Requirements to verify. This version is used by default if no version is specified in subsystem’s aiv-config.yaml. [!289]

  • “Revised” UC status is now set if a UC occurred in previous releases, not read directly from table. [!289]

  • All requests with request library are now cached by default. This accelerates a lot repeated requests to Jama and GitLab. One should keep in mind that if remote state (e.g. gitlab job state) changes, local cache might need to be manually invalidated. [!289]

  • Pipelines are now automatically cancelled when new commits are pushed to the same merge request. [!298]

  • The toolkit now packages a chart before running helm upgrade, which enables overriding the chart’s app version. This should make overriding the dev image tag unnecessary in pipelines. [!299]

DPPS AIV Toolkit v3.0.0 (2025-07-29)#

API Changes#

  • By default, the toolkit will no longer modify the user’ global kubeconfig in $HOME/.kube/config but instead write the config to kubeconfig-<cluster-name>.yaml in the current directory.

    To interact with the cluster, directly set:

    $ export KUBECONFIG=$(pwd)/kubeconfig-<cluster-name>.yaml [`!254 <https://gitlab.cta-observatory.org/cta-computing/common/aiv-toolkit/-/merge_requests/254>`__]

  • Remove hard-coded options for sonar-scanner in the ci-sonar.yml step. These options conflict with the new sonarqube server and are also better set in the per-project configuration either via CI variables or in the sonar-project.properties.

    The following options are removed, and might need to be defined now in the project specific settings:

    • sonar.python.coverage.reportPaths='*/coverage.xml'

    • sonar.language=python [!257]

  • The toolkit now puts all the files it generates (downloaded binaries, logs, etc.) into a .toolkit directory instead of the local directory to avoid cluttering the base directory of the repository and make ignoring the files easier.

    Add .toolkit to your .gitignore. [!259]

Bug Fixes#

  • Fix git-info step using an image not providing git and failing silently. [!250]

  • Use GITLAB_TOKEN for changelog step in case it is defined. CI_JOB_TOKEN is missing permissions to query MRs. [!265]

  • Links to MRs do not redirect anymore: fix to correct current repository. [!277]

New Features#

  • Parse configured towncrier snippet names from the towncrier configuration in ci-changelog.yml instead of using a hardcoded list of types. [!241]

  • Add DEV_DOCKER_EXTRA_ARGS args variable in command to build docker image, allowing subsystems to pass additional options to the docker build: DEV_DOCKER_EXTRA_ARGS=--build-arg DATAPIPE_VERSION=v0.2.1. [!253]

  • Allow to select custom helm repository list by setting HELM_REPO_CONFIG variable for the make. [!260]

  • Add make target fetch-cluster-certs to download cluster certificates in .toolkit directory. These certificates can be used to access cluster services through proxy pass or ingress. This is an advanced developer feature and it is not by default advised to developers who do not realise its implications. [!262]

  • Generate kind cluster configuration dynamically, based on new configuration variables (ENABLE_REGISTRY_MIRRORS, ENABLE_INGRESS, MOUNT_REPO).

    Refactor helm upgrade functionality, relying more on python and less on Makefile. To install controller before your application chart, add this rule to your application Makefile: install-chart: ingress-controller. [!266]

  • Simplify name of default kubeconfig, is now always just ./.toolkit/kubeconfig.yaml. This means you can export KUBECONFIG=.toolkit/kubeconfig.yaml and switch between directories of projects without having to change KUBECONFIG. [!269]

  • Support projects on multiple sonarqube servers.

    This also adds support for private projects or servers globally enforcing authentication by adding the option to get tokens from environment variables.

    Since the token is only valid for a specific server, the expected definition is:

    SONAR_API_TOKEN_1=sonar1.example.com,squ_asdsadsadsa
SONAR_API_TOKEN_2=sonar2.example.com,squ_kjlkjkljkkj

    The tokens need to be “User” tokens, not “Project Analysis” tokens. [!270]

  • Make the generated user certificates configurable. By default, generate 3 users. [!273]

Maintenance#

  • Update fluent-bit to 0.49.1 and get docker image from harbor. [!244]

  • Added a note on toolkit configuration source. [!260]

  • Improve quickstart and configuration docs. [!274]

Refactoring and Optimization#

  • De-deduplicate images to be pulled by aiv-deploy kind-pull-images. [!245]

DPPS AIV Toolkit v2.1.0 (2025-06-24)#

API Changes#

  • Use dpps-aiv-toolkit python module as part of the deployment process. It means that this module now has to be installed for local testing. [!222]

  • Toolkit operations now rely on the toolkit python package. [!225]

Bug Fixes#

  • Fix k8s-integration-tests hanging in case of test failure until pipeline timeout is reached. [!227]

New Features#

  • Detect, pull, and load to kind cluster all chart images: more controlled and faster startup. [!222]

  • Derive name of local kind cluster from the directory in which it is started. This allows to run several local clusters at the same time, with different directories. [!222]

  • Test job artifacts now include structured statistics about image pulls. [!229]

  • Allow to override default kind config with KIND_CONFIG variable in aiv-config.yaml or from environment. [!235]

  • If the unit test table is very long, summarize it instead of listing. [!236]

Maintenance#

  • Update kind to 0.29.0 . Security and performance upgrade. [!221]

DPPS AIV Toolkit v2.0.0 (2025-05-28)#

API Changes#

  • Add new job helm-lint for linting helm charts. This job is enabled by default for repositories containing a helm chart (if $CHART_LOCATION/Chart.yaml exists).

    This check might fail in case helm lint finds issues in the helm chart. In this case the pipeline will not pass until the issues are fixed or the job is disabled. [!193]

  • Add kubeconform: a linter enforcing validity of k8s manifests.

    This can be a breaking change if the charts are not compliant! [!203]

  • Disable any support for harbor pull secrets. All our images are public. [!205]

Bug Fixes#

New Features#

  • Add new CI job check-changelog that checks if a merge request contains a towncrier changelog snippet. The check is skipped if the project does not have the docs/changes directory or the MR is labelled with no-changelog-needed. [!191]

  • Add kube-linter. It’s not totally stable, so not enforced. Also added kube-score. It’s useful but too strict currently, not enforced. [!203]

  • Freeze helm version in lint jobs [!211]

Maintenance#

Refactoring and Optimization#

DPPS AIV Toolkit v1.0.1 (2025-05-12)#

API Changes#

  • According to latest dev version of ICD Pipeline - WMS, docker hint for pipeline image is added to all CWL tools. Also, CWL is formatted with cwl-format to be more homogeneous. [!150]

  • At this time, AIV_TOOLKIT_DIR variable should NOT be set to “dpps-aiv-toolkit”. It should be set to full directory path, or it can be dropped completely. [!181]

  • test job is not anymore the source of the artifacts.

    All jobs provided as dependencies for the collect-test-artifacts job will be merged into final test artifacts used in the test report.

    The test jobs in the dependencies should provide artifacts as DIR/report.xml where DIR is any directory name. DIR directory names should be different in different test jobs.

New Features#

  • Read artifacts from other repositories, producing simple combined report. List “deployment UCs”, demonstrated by pipeline passes. [!44]

  • Allow to use in the report custom full application names. E.g. full name of “BDMS” is “DPPS BDMS”. [!71]

  • Collect logs with fluetbit, allowing to: * steam logs to stdout as they come. * store all logs also for jobs which do not exist at the end of the execution. * inspect logs in the app to verify observability. [!119]

  • Allow to configure log collection and streaming. [!125]

  • Include UCs from previous releases, and add a column indicating if the UC is added, revised, or simply retested. [!134]

  • Support linting and publishing of CWL files. [!142]

  • Add a note about report build: the toolkit version and build time.

Refactoring and Optimization#

  • Refactor autoreport argument generation to enable running all generators at once.