Jinja Templating and Macros

The date belongs to the run

Many operator fields (such as bash_command, SQL files and paths) are templated with Jinja. Airflow supplies variables describing the run: {{ ds }} (logical date as YYYY-MM-DD), {{ data_interval_start }}, {{ data_interval_end }}, {{ run_id }}, {{ params.x }} and more. Use them instead of datetime.now() so that a re-run for a past day processes that day's data: the foundation of idempotent, backfill-able pipelines. In @task functions, receive the same values as arguments or from the context.

A date-aware command and path

Re-running the run for 2026-09-15 reads and writes only partition dt=2026-09-15.

export_orders = BashOperator(
    task_id="export_orders",
    bash_command=(
        "python export.py --from '{{ data_interval_start | ds }}' "
        "--to '{{ data_interval_end | ds }}' "
        "--out s3://lake/orders/dt={{ ds }}/"
    ),
)

Never use now() for business dates

datetime.now() returns the time the task happens to execute, which changes on retries and backfills and silently produces wrong data. Use the run's own dates.

Quick check: What does `{{ ds }}` give you?

• The logical date of the run as YYYY-MM-DD
• The current wall-clock time
• The DAG file name
• A random number