Setting Dependencies

Arrows in code

a >> b means "run b after a succeeds". A list fans out or in: extract >> [clean_a, clean_b] >> load runs both clean tasks in parallel after extract, and load waits for both. chain(...) and cross_downstream(...) help with larger graphs. A DAG must be acyclic: you cannot make a task depend, directly or indirectly, on itself, and Airflow reports an import error if you try.

Execution order, run

I ran a tiny dependency resolver (not Airflow itself) for the graph extract → [transform_a, transform_b] → load → notify. Both transforms become ready together once extract finishes, and load waits for both.

from collections import deque

class Task:
    def __init__(self, name): self.name, self.down = name, []
    def __rshift__(self, other):
        for o in (other if isinstance(other, list) else [other]):
            self.down.append(o)
        return other

extract, t1, t2, load, notify = (Task(n) for n in ("extract", "transform_a", "transform_b", "load", "notify"))
extract >> [t1, t2]; t1 >> load; t2 >> load; load >> notify

def topo(tasks):
    indeg = {t.name: 0 for t in tasks}
    for t in tasks:
        for d in t.down: indeg[d.name] += 1
    q = deque(t for t in tasks if indeg[t.name] == 0); out = []
    while q:
        t = q.popleft(); out.append(t.name)
        for d in t.down:
            indeg[d.name] -= 1
            if indeg[d.name] == 0: q.append(d)
    return out

print(topo([extract, t1, t2, load, notify]))

Output:

['extract', 'transform_a', 'transform_b', 'load', 'notify']

Keep graphs readable

If a DAG has dozens of tasks, group them with TaskGroup so the UI stays understandable, or split the pipeline into several DAGs connected by assets or triggers.

Quick check: What does `extract >> [clean_a, clean_b] >> load` mean?

• Both clean tasks run after extract, and load waits for both
• load runs first
• The tasks run one after another in list order only
• It is a syntax error