Skip to main content
Pipeline chaining expresses dependencies: when pipeline A succeeds, pipeline B starts; when B fails, on-call routes to the team that owns stage B. You model this with orchestration primitives instead of copying nodes into one oversized graph.

​
Trigger Pipeline node

The Trigger Pipeline node (or equivalent orchestration action) starts another pipeline with parameters derived from upstream results—file paths, partition dates, batch IDs, or quality-check outcomes.
1

Author the parent pipeline

Build stages that must complete first (ingest, validate, promote staging tables).
2

Add Trigger Pipeline

Point to the child pipeline and map outputs to child parameters.
3

Define failure behavior

Choose whether to fail the parent, continue with notifications, or branch to a remediation pipeline.
4

Test in dev

Run end-to-end with small data; confirm child runs inherit the correct environment and credentials.

​
Dependency management

Use when later stages require committed outputs from earlier warehouses loads (for example, dimensions before facts).
Document dependencies in the pipeline description so new hires understand why ordering exists.

​
Macro pipelines

A macro pipeline is an orchestration-only workflow composed mostly of trigger nodes and lightweight control steps, while heavy lifting stays in child pipelines owned by different teams. This pattern:
  • Keeps blast radius small per repository or team
  • Lets you version child pipelines independently
  • Surfaces clear ownership in run history
Pass explicit run labels or correlation IDs from parent to child so you can trace all stages for a single business batch in Observability.

​
Governance

Restrict who can trigger production pipelines from dev parents. Use teams and permissions so only service accounts or release roles can start sensitive children.

Schedules

Start parent pipelines on a cadence.

Git integration

Version pipeline definitions that participate in chains.