Pipelines

A pipeline is the build, run by a machine, on every change, with the results where everyone can see them. That sentence contains the whole value: not that the tests run — you could run them — but that they run the same way every time, on a clean machine, before a change can be merged, and that "it passes on my laptop" stops being an argument. This lesson is the stages a backend pipeline has, the three things that make it fast, and the two tools you will meet.

#Stages, and the order that fails fastest

lint → unit tests → build the artifact → integration tests → publish → deploy

The order is by cost and by how much a failure tells you. Lint and compile fail in seconds and catch most typos; unit tests in a minute; the build produces the jar that every later stage uses; integration tests — Testcontainers, the contract suite — take minutes and Docker; publishing puts the artifact somewhere; deployment is its own lesson. A stage runs only if the one before passed, and a stage that fails should say what failed in its own output, not in a log somebody has to scroll.

This repository's npm run quality is that pipeline on one machine, and its report is the shape a pipeline's summary should have: each stage named, PASS or FAIL or NOT CONFIGURED, never a green that was not earned. The GitHub workflow runs the same guards on two Node versions and the Java tests on a third job, in parallel, because they do not depend on each other.

#GitHub Actions: a workflow file

name: ci
on:
  pull_request:
  push: { branches: [main] }
jobs:
  api:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-java@v6
        with: { distribution: temurin, java-version: "25", cache: maven }
      - run: ./mvnw -B -ntp verify
        working-directory: backend/api
      - uses: actions/upload-artifact@v4
        with: { name: api-jar, path: backend/api/target/*.jar, retention-days: 7 }
  contract:
    needs: api
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16-alpine
        env: { POSTGRES_PASSWORD: contract }
        options: --health-cmd pg_isready --health-interval 5s
    steps:
      - uses: actions/checkout@v7
      - uses: actions/download-artifact@v4
        with: { name: api-jar, path: backend/api/target }
      - run: npm run contract:test

The parts to know: on is the trigger (pull requests, pushes to main, a schedule, a manual workflow_dispatch); a job is a fresh virtual machine, so nothing carries between jobs except what you upload as an artifact or cache; needs orders jobs and lets independent ones run in parallel; services starts containers beside the job — a Postgres for the contract suite — the way Testcontainers does inside a test; and a matrix (strategy: { matrix: { node: [20, 22] } }) runs the same job per value, which is how this repository builds on two Node versions with one job definition. Pin actions to a major version and let Dependabot move them; a @main reference is a supply chain you do not control.

#Caching dependencies: the difference between two minutes and twelve

A fresh machine has no ~/.m2 and no node_modules. Without a cache, every run downloads every dependency — for a Spring Boot service, hundreds of megabytes — before it compiles a line. The setup actions cache by lockfile: setup-java with cache: maven keys the cache on pom.xml's hash, setup-node with cache: npm on package-lock.json, so the cache is reused until the dependencies change and rebuilt when they do. The rule that keeps it correct: the cache key is the file that declares the dependencies, never a branch name or a date. A Docker build in CI caches the same way — docker/build-push-action with cache-from: type=gha stores layers between runs, and the Docker course's layer ordering is what makes that cache hit.

#Artifacts: build once, promote everywhere

The jar that passed the tests is the jar that should be deployed — not a second build from the same commit, which is probably identical and cannot be proven to be. So the build stage publishes an artifact and every later stage consumes it: a workflow artifact for the next job, a container image pushed to a registry (GHCR, ECR, Artifactory) tagged with the commit SHA for the deploy, a Maven artifact to a repository manager (Nexus, Artifactory) for a library other services depend on. Tags are immutable and traceable — orders-api:3f2a9c1 says which commit — and :latest is for humans typing docker run, never for a deployment.

The repository manager also does the other half of dependency hygiene: it proxies Maven Central so the build does not depend on the internet, and it is where a dependency audit (npm audit, OWASP Dependency-Check, the mvn dependency:tree diff) reports.

#Credentials in the pipeline

A pipeline needs credentials — a registry token, a deploy key, a database password for the integration tests — and it is the most-attacked place to keep them, because a pull request from a fork runs code you did not write. The rules: they live in the CI system's secret store, injected as environment variables into the steps that need them and no others; a pull_request from a fork gets none (GitHub does this by default; keep it); prefer short-lived tokens through OIDC federation (Actions to AWS, to GCP, to Vault) over long-lived keys in the store; and mask them in logs, which the store does for exact matches and does not for a value you printed base64-encoded. This repository's hooks scan every diff for credential patterns before a commit lands, which is the layer before the pipeline.

#Jenkins, and the other one

Jenkins is a server you run, with a Jenkinsfile in the repository describing stages in a Groovy DSL, agents you provision, and a plugin for everything. GitHub Actions (GitLab CI, and the cloud providers' equivalents) is hosted, YAML, and integrated with the pull request. The trade is control against maintenance: Jenkins can run on your hardware, behind your firewall, with build agents shaped exactly for your workload, and it is a service with upgrades, plugins that break, and a security surface your team owns. A team that has one is not wrong to keep it; a team choosing today usually starts hosted and moves only when a specific need — air-gapped builds, custom hardware, a thousand jobs a day — makes the server worth owning. The stages, the caching and the artifact rules are the same in both.