Building a multi-pipeline CI setup for running automated tests in GitLab

By Akshay Maldhure

Multi-pipeline CI (continuous integration) setup refers to the process of combining two or more different CI pipelines to create a single workflow.

This article explains how to set this process up for combining the development workflow (e.g. unit test → build → deploy) and the testing workflow (e.g. running functional tests from an independent and separate project in CI) into a single pipeline in GitLab.

This is specifically useful in cases where the tests reside outside the application repository. Such a setup could also be enabled for more than one application within a given team by creating reusable CI templates.

Rationale

The process of combining the development and testing workflows lets us run the automated tests (that are typically built and maintained by the QAs/SDETs in the teams) in conjunction with every change made by the developer(s) to any of the applications rather than running those in a standalone/separate pipeline (e.g. a scheduled pipeline).

The idea of running tests only on application side changes could be more efficient and effective since it:

1. Makes it possible to segregate, track, debug, and fix the test failures on a per-application side change basis.
2. Helps ensure more optimal utilization of the CI resources.

Pipeline sample

As seen in the video below, the stages pre_deploy test build deploy_integration deploy_production and post_deploy belong to the development workflow (application project); whereas the stage functional_test belongs to the testing workflow (test project).

In this example, it can be seen that we have combined two different pipelines; a pipeline from the application project and a pipeline from the testing project so that the result looks like a single pipeline below.

Implementation

This setup is a combination of the following pieces coming together.

CI config in the test project (downstream)

This config defines what the downstream pipeline (i.e. the functional testing pipeline) should do using the scriptkeyword. We’ve defined a bunch of rules that define the applicability of the test job based on the pipeline source, project code, and job name coming from the upstream pipeline (via CI templates).

CI templates

This project defines a bunch of reusable test job templates that could be used in the CI configs of all applications in your team. By having a standard naming convention for the integration deployment job for every application pipeline, it becomes possible to make the test job depend upon the integration deployment job using the needs keyword. We could also use some more keywords like allow_failure (with the value false to disallow merging the MRs on functional test failures) and resource_group (with the value same as that in the integration deployment job to ensure that a pipeline does not overwrite the integration deployment or run the tests while the integration deployment or tests from some other pipeline are running).

CI config in application/service (upstream)

This project simply imports the jobs defined in ci-templates using the include keyword. No extra information is passed here since the downstream already defines all of that.

This is how the downstream pipeline defined in the test project with the job definitions from CI templates gets connected to the upstream pipeline defined in the application/service to form a single multi-pipeline setup.

Happy testing, folks!

To check out more stories from our vault, click here.