Connections, Variables and Secrets

Credentials do not belong in DAG files

A Connection stores how to reach an external system (host, login, password, extras) under a name such as warehouse_db; operators and hooks take that name, so DAG code never contains secrets. A Variable stores a small configuration value. Both can be set in the UI, through the CLI, via environment variables (AIRFLOW_CONN_WAREHOUSE_DB), or, best for production, read from a secrets backend such as AWS Secrets Manager, Google Secret Manager or HashiCorp Vault. Restrict who can view connections, since passwords are masked but still accessible to admins.

A connection as an environment variable

The name after AIRFLOW_CONN_ becomes the connection ID (warehouse_db). The value is a URI. Use placeholders here, never real passwords in documentation or Git.

export AIRFLOW_CONN_WAREHOUSE_DB="postgresql://loader:<password>@warehouse.internal:5432/analytics"

# in a DAG, only the ID appears:
# PostgresOperator(task_id="load", postgres_conn_id="warehouse_db", sql="sql/load_orders.sql")

Quick check: Where should a database password for a DAG live?

• In a public repository
• Hard-coded in the DAG file
• In a task name
• In a Connection or a secrets backend