Advanced GitLab CI Practices: Parallel Needs, Artifacts, Includes, Extends, and Triggered Pipelines

The article is part of a "GitLab CI Practice" tutorial and introduces advanced CI/CD techniques for orchestrating jobs, managing artifacts, and reusing configuration across pipelines.

Parallel needs allow jobs to run without strict stage order, enabling concurrent execution. Example:

stages:
  - build
  - test
  - deploy

module-a-build:
  stage: build
  script:
    - echo "hello3a"
    - sleep 10

module-b-build:
  stage: build
  script:
    - echo "hello3b"
    - sleep 10

module-a-test:
  stage: test
  script:
    - echo "hello3a"
    - sleep 10
  needs: ["module-a-build"]

module-b-test:
  stage: test
  script:
    - echo "hello3b"
    - sleep 10
  needs: ["module-b-build"]

When needs references jobs that are not instantiated due to only/except rules, pipeline creation fails with a YAML error. The maximum number of jobs that can be referenced by needs is limited by the ci_dag_limit_needs feature flag (default 10, can be increased to 50):

Feature::disable(:ci_dag_limit_needs)   # 50
Feature::enable(:ci_dag_limit_needs)  #10

Artifact download control can be set per needs entry using artifacts: true or artifacts: false. Example:

module-a-test:
  stage: test
  script:
    - echo "hello3a"
    - sleep 10
  needs:
    - job: "module-a-build"
      artifacts: true

Artifacts from jobs in the same project can also be fetched from other pipelines using the project keyword:

build_job:
  stage: build
  script:
    - ls -lhR
  needs:
    - project: group/same-project-name
      job: build-1
      ref: other-ref
      artifacts: true

Include mechanisms let you import external YAML files:

Local : reference a file in the same repository.

stages:
  - deploy

deployjob:
  stage: deploy
  script:
    - echo 'deploy'

include:
  local: 'ci/localci.yml'

stages:
  - build
  - test
  - deploy

buildjob:
  stage: build
  script: ls

testjob:
  stage: test
  script: ls

File : import from another project.

include:
  - project: demo/demo-java-service
    ref: master
    file: '.gitlab-ci.yml'

Template : use official templates.

include:
  - template: Auto-DevOps.gitlab-ci.yml

Remote : fetch via HTTP/HTTPS URL.

include:
  - remote: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml'

Extends allows a job to inherit configuration from a hidden job definition. Example:

stages:
  - test
variables:
  RSPEC: 'test'

.tests:
  script: echo "mvn test"
  stage: test
  only:
    refs:
      - branches

testjob:
  extends: .tests
  script: echo "mvn clean test"
  only:
    variables:
      - $RSPEC

After merging, the job looks like:

testjob:
  stage: test
  script: mvn clean test
  only:
    variables:
      - $RSPEC
    refs:
      - branches

Combining extends with include enables reuse of both local templates and shared jobs:

#ci/localci.yml (commented out stages)

deployjob:
  stage: deploy
  script:
    - echo 'deploy'
  only:
    - dev

.template:
  stage: build
  script:
    - echo "build"
  only:
    - master

include:
  local: 'ci/localci.yml'

stages:
  - test
  - build 
  - deploy

variables:
  RSPEC: 'test'

.tests:
  script: echo "mvn test"
  stage: test
  only:
    refs:
      - branches

testjob:
  extends: .tests
  script: echo "mvn clean test"
  only:
    variables:
      - $RSPEC

newbuildjob:
  script:
    - echo "123"
  extends: .template

Triggered pipelines create downstream pipelines when a trigger job runs. They support multi‑project pipelines and parent‑child pipelines. Example of a multi‑project trigger:

staging:
  variables:
    ENVIRONMENT: staging
  stage: deploy
  trigger:
    project: demo/demo-java-service
    branch: master
    strategy: depend

In a parent pipeline, a child pipeline can be included and triggered:

stages:
  - build

child-a-build:
  stage: build
  script:
    - echo "hello3a"
    - sleep 10

staging2:
  variables:
    ENVIRONMENT: staging
  stage: deploy
  trigger:
    include: ci/child01.yml  
    strategy: depend

These configurations allow fine‑grained control over job execution order, artifact sharing, reusable CI definitions, and complex pipeline orchestration across multiple repositories.