Lesson 8 / 25

Scheduling: Cron, Intervals and Catchup

Choose a schedule, understand data intervals and decide whether to catch up on past runs.

The run is for the interval that just ended

Set schedule to a preset ("@daily", "@hourly"), a cron string ("0 6 * * *" for 06:00 daily), or a timedelta. Airflow thinks in data intervals: a daily run whose logical date is 1 October covers 1 Oct 00:00 to 2 Oct 00:00 and actually starts after that interval has ended, at midnight on 2 October. start_date is when scheduling may begin. catchup controls whether Airflow creates runs for every missed interval since start_date (True) or only the latest (False). Be careful with catchup=True and a distant start_date: it can create hundreds of runs at once.

Interval and catchup arithmetic, run

I ran this plain-Python calculation. A run with logical date 1 Oct covers 1 to 2 Oct and executes at 2 Oct 00:00. With start_date 1 Sep and catchup=True on 1 Oct, 30 daily runs (1 to 30 Sep) would be created.

from datetime import datetime, timedelta, date

logical = datetime(2026, 10, 1)
end = logical + timedelta(days=1)
print("interval", logical.date(), "->", end.date(), "runs at/after", end.isoformat())

start, now = date(2026, 9, 1), date(2026, 10, 1)
runs = [start + timedelta(days=i) for i in range((now - start).days)]
print(len(runs), runs[0], runs[-1])

Output:

interval 2026-10-01 -> 2026-10-02 runs at/after 2026-10-02T00:00:00
30 2026-09-01 2026-09-30

Use UTC and be explicit

Airflow works in UTC internally. If your business thinks in India Standard Time (UTC+5:30), define the timezone deliberately with a timezone-aware start_date, and remember that 06:00 IST is 00:30 UTC.

Quick check: What does `catchup=False` do?

  • Deletes old runs
  • Disables the DAG
  • Creates only the latest run instead of one per missed interval
  • Makes the DAG run twice
Answer

Creates only the latest run instead of one per missed interval — Without catchup, Airflow does not backfill every interval since start_date.