Implementing Dynamic Multi‑Project GitLab CI/CD Pipelines with Artifacts and Dependencies

GitLab is a powerful Git‑based platform for building complex CI/CD pipelines, and this guide shows how to use dynamic pipelines, multi‑project triggers, and artifact handling to deploy multiple environments in a scalable way.

How to Set Up Dynamic GitLab Pipelines

Dynamic pipelines generate jobs programmatically based on input parameters, allowing you to create any number of environments (e.g., 1, 3, or more) by repeatedly triggering a create‑env job.

In the first stage ( templating ) a custom Python script generates a YAML file ( environments.yml ) that lists the requested environments (dev, prod, staging). The second stage ( deployment ) consumes this file to deploy each environment.

# bootstrap‑env/.gitlab-ci.yml
variables:
  ENVIRONMENTS:
    description: "User input: comma‑separated list of environments"
    value: "dev,prod,staging"

stages:
  - templating
  - deployment

generate‑templates:
  stage: templating
  image: python:3.10
  before_script:
    - pip install -r requirements.txt
  script:
    - python generate_templates.py --env $ENVIRONMENTS
  artifacts:
    paths:
      - environments.yml

deploy‑envs:
  stage: deployment
  trigger:
    include:
      - artifact: environments.yml
    job: generate‑templates
    strategy: depend

How to Set Up Multi‑Project Downstream Pipelines

Each step of the pipeline can live in a dedicated GitLab project. To trigger a pipeline in another project you can use the trigger keyword or call the GitLab API with curl . The example shows using both methods to exceed the default trigger depth of 2.

# bootstrap‑env/generated-template.yml
deploy‑staging:
  environment: staging
  variables:
    GITLAB_PROJECT_ID: 123456789  # the project ID of 'create‑env'
    GITLAB_REF: main
  script:
    - >
      curl --request POST \
           --form "token=$CI_JOB_TOKEN" \
           --form "ref=$GITLAB_REF" \
           --form "variables[ENVIRONMENT]=$CI_ENVIRONMENT_NAME" \
           "https://gitlab.com/api/v4/projects/$GITLAB_PROJECT_ID/trigger/pipeline"

The trigger keyword accepts a project name or a trigger:project syntax (available on GitLab Premium). Using strategy: depend makes the parent pipeline’s status depend on its child pipelines.

How to Pass Artifacts Between GitLab Jobs

Artifacts are used to share information between jobs. By default, all artifacts from previous stages are passed forward, which can cause storage and security concerns. You can limit this with the dependencies field or, for cross‑project pipelines, the needs keyword.

# add‑resources/.gitlab-ci.yml
variables:
  ARTIFACT_RESOURCE3: "resource3-outputs.zip"
  ARTIFACT_RESOURCE2: "resource2-outputs.zip"
  ARTIFACT_RESOURCE1: "resource1-outputs.zip"

stages:
  - resource1
  - resource2
  - resource3

add‑resource1:
  stage: resource1
  trigger:
    include:
      - local: "resource1.yml"
    strategy: depend

add‑resource2:
  stage: resource2
  trigger:
    include:
      - local: "resource2.yml"
    strategy: depend

add‑resource3:
  stage: resource3
  trigger:
    include:
      - local: "resource3.yml"
    strategy: depend

When using needs , the job no longer automatically downloads all previous artifacts; it only fetches those explicitly required, allowing finer‑grained control over execution order and artifact consumption.

Conclusion

The article demonstrates a complete solution for dynamic, multi‑project GitLab CI pipelines, covering environment generation, downstream triggers, artifact passing, and dependency management, enabling teams to build flexible and maintainable CI/CD workflows.