Change Log#

aivkit 3.0.1.dev24+g6d70ba1d1 (2025-09-02)#

API Changes#

  • Remove variable SONAR_HOST_URL from ci-sonar.yml. Projects should set the host url in sonar-project.properties if they do not already do so. [!248]

Bug Fixes#

New Features#

  • Add proxy registry definitions for quay.io and ghcr.io. [!281]

Maintenance#

Refactoring and Optimization#

DPPS AIV Toolkit v3.0.0 (2025-07-29)#

API Changes#

  • By default, the toolkit will no longer modify the user’ global kubeconfig in $HOME/.kube/config but instead write the config to kubeconfig-<cluster-name>.yaml in the current directory.

    To interact with the cluster, directly set:

    $ export KUBECONFIG=$(pwd)/kubeconfig-<cluster-name>.yaml [`!254 <https://gitlab.cta-observatory.org/cta-computing/dpps/aiv/dpps-aiv-toolkit/-/merge_requests/254>`__]

  • Remove hard-coded options for sonar-scanner in the ci-sonar.yml step. These options conflict with the new sonarqube server and are also better set in the per-project configuration either via CI variables or in the sonar-project.properties.

    The following options are removed, and might need to be defined now in the project specific settings:

    • sonar.python.coverage.reportPaths='*/coverage.xml'

    • sonar.language=python [!257]

  • The toolkit now puts all the files it generates (downloaded binaries, logs, etc.) into a .toolkit directory instead of the local directory to avoid cluttering the base directory of the repository and make ignoring the files easier.

    Add .toolkit to your .gitignore. [!259]

Bug Fixes#

  • Fix git-info step using an image not providing git and failing silently. [!250]

  • Use GITLAB_TOKEN for changelog step in case it is defined. CI_JOB_TOKEN is missing permissions to query MRs. [!265]

  • Links to MRs do not redirect anymore: fix to correct current repository. [!277]

New Features#

  • Parse configured towncrier snippet names from the towncrier configuration in ci-changelog.yml instead of using a hardcoded list of types. [!241]

  • Add DEV_DOCKER_EXTRA_ARGS args variable in command to build docker image, allowing subsystems to pass additional options to the docker build: DEV_DOCKER_EXTRA_ARGS=--build-arg DATAPIPE_VERSION=v0.2.1. [!253]

  • Allow to select custom helm repository list by setting HELM_REPO_CONFIG variable for the make. [!260]

  • Add make target fetch-cluster-certs to download cluster certificates in .toolkit directory. These certificates can be used to access cluster services through proxy pass or ingress. This is an advanced developer feature and it is not by default advised to developers who do not realise its implications. [!262]

  • Generate kind cluster configuration dynamically, based on new configuration variables (ENABLE_REGISTRY_MIRRORS, ENABLE_INGRESS, MOUNT_REPO).

    Refactor helm upgrade functionality, relying more on python and less on Makefile. To install controller before your application chart, add this rule to your application Makefile: install-chart: ingress-controller. [!266]

  • Simplify name of default kubeconfig, is now always just ./.toolkit/kubeconfig.yaml. This means you can export KUBECONFIG=.toolkit/kubeconfig.yaml and switch between directories of projects without having to change KUBECONFIG. [!269]

  • Support projects on multiple sonarqube servers.

    This also adds support for private projects or servers globally enforcing authentication by adding the option to get tokens from environment variables.

    Since the token is only valid for a specific server, the expected definition is:

    SONAR_API_TOKEN_1=sonar1.example.com,squ_asdsadsadsa
SONAR_API_TOKEN_2=sonar2.example.com,squ_kjlkjkljkkj

    The tokens need to be “User” tokens, not “Project Analysis” tokens. [!270]

  • Make the generated user certificates configurable. By default, generate 3 users. [!273]

Maintenance#

  • Update fluent-bit to 0.49.1 and get docker image from harbor. [!244]

  • Added a note on toolkit configuration source. [!260]

  • Improve quickstart and configuration docs. [!274]

Refactoring and Optimization#

  • De-deduplicate images to be pulled by aiv-deploy kind-pull-images. [!245]

DPPS AIV Toolkit v2.1.0 (2025-06-24)#

API Changes#

  • Use dpps-aiv-toolkit python module as part of the deployment process. It means that this module now has to be installed for local testing. [!222]

  • Toolkit operations now rely on the toolkit python package. [!225]

Bug Fixes#

  • Fix k8s-integration-tests hanging in case of test failure until pipeline timeout is reached. [!227]

New Features#

  • Detect, pull, and load to kind cluster all chart images: more controlled and faster startup. [!222]

  • Derive name of local kind cluster from the directory in which it is started. This allows to run several local clusters at the same time, with different directories. [!222]

  • Test job artifacts now include structured statistics about image pulls. [!229]

  • Allow to override default kind config with KIND_CONFIG variable in aiv-config.yaml or from environment. [!235]

  • If the unit test table is very long, summarize it instead of listing. [!236]

Maintenance#

  • Update kind to 0.29.0 . Security and performance upgrade. [!221]

DPPS AIV Toolkit v2.0.0 (2025-05-28)#

API Changes#

  • Add new job helm-lint for linting helm charts. This job is enabled by default for repositories containing a helm chart (if $CHART_LOCATION/Chart.yaml exists).

    This check might fail in case helm lint finds issues in the helm chart. In this case the pipeline will not pass until the issues are fixed or the job is disabled. [!193]

  • Add kubeconform: a linter enforcing validity of k8s manifests.

    This can be a breaking change if the charts are not compliant! [!203]

  • Disable any support for harbor pull secrets. All our images are public. [!205]

Bug Fixes#

New Features#

  • Add new CI job check-changelog that checks if a merge request contains a towncrier changelog snippet. The check is skipped if the project does not have the docs/changes directory or the MR is labelled with no-changelog-needed. [!191]

  • Add kube-linter. It’s not totally stable, so not enforced. Also added kube-score. It’s useful but too strict currently, not enforced. [!203]

  • Freeze helm version in lint jobs [!211]

Maintenance#

Refactoring and Optimization#

DPPS AIV Toolkit v1.0.1 (2025-05-12)#

API Changes#

  • According to latest dev version of ICD Pipeline - WMS, docker hint for pipeline image is added to all CWL tools. Also, CWL is formatted with cwl-format to be more homogeneous. [!150]

  • At this time, DPPS_AIV_TOOLKIT_DIR variable should NOT be set to “dpps-aiv-toolkit”. It should be set to full directory path, or it can be dropped completely. [!181]

  • test job is not anymore the source of the artifacts.

    All jobs provided as dependencies for the collect-test-artifacts job will be merged into final test artifacts used in the test report.

    The test jobs in the dependencies should provide artifacts as DIR/report.xml where DIR is any directory name. DIR directory names should be different in different test jobs.

New Features#

  • Read artifacts from other repositories, producing simple combined report. List “deployment UCs”, demonstrated by pipeline passes. [!44]

  • Allow to use in the report custom full application names. E.g. full name of “BDMS” is “DPPS BDMS”. [!71]

  • Collect logs with fluetbit, allowing to: * steam logs to stdout as they come. * store all logs also for jobs which do not exist at the end of the execution. * inspect logs in the app to verify observability. [!119]

  • Allow to configure log collection and streaming. [!125]

  • Include UCs from previous releases, and add a column indicating if the UC is added, revised, or simply retested. [!134]

  • Support linting and publishing of CWL files. [!142]

  • Add a note about report build: the toolkit version and build time.

Refactoring and Optimization#

  • Refactor autoreport argument generation to enable running all generators at once.