# Executors and Deployment Options — Apache Airflow: Orchestrate Data Pipelines

Source: https://www.geekswithgeeks.com/en/airflow/scale-executors

> Compare Local, Celery and Kubernetes executors and managed Airflow services.

## Where tasks actually run

The **executor** decides how tasks are run. The **LocalExecutor** runs them as processes on the scheduler machine (simple, fine for small setups). The **CeleryExecutor** distributes tasks to a pool of long-running **worker** machines through a message broker (Redis or RabbitMQ): scalable and fast to start. The **KubernetesExecutor** launches a fresh **pod** per task: strong isolation and per-task resources, with startup overhead. Recent Airflow versions can also run multiple executors together. You can run Airflow yourself (Docker Compose, the official **Helm chart** on Kubernetes) or use a managed service (**Amazon MWAA, Google Cloud Composer, Astronomer**) that handles upgrades and scaling for a fee.

## From one machine to a team platform

Choose an executor, ship DAGs through Git and CI, and test them before they reach the scheduler.

![Three steps: executor, deploy, test.](assets/figures/airflow/section-7-map.svg) — Figure 7.1 — Executor, deploy and test.

## Choosing an executor

A starting guide; benchmark your own workload.

```text
Learning / tiny team          LocalExecutor (or standalone)
Steady load, many short tasks   CeleryExecutor (Redis/RabbitMQ + worker pool)
Mixed/heavy tasks, isolation    KubernetesExecutor (pod per task) or KubernetesPodOperator
Small platform team, want less ops   managed service (MWAA / Composer / Astronomer)
```

## Use PostgreSQL for the metadata DB

SQLite is only for learning. Production deployments use PostgreSQL (or MySQL) for the metadata database, with backups and monitoring, because the scheduler and every task read and write it.

**Quiz:** Which executor launches a new pod for each task?

- [ ] No executor does this
- [ ] LocalExecutor
- [ ] CeleryExecutor
- [x] KubernetesExecutor

*Answer:* KubernetesExecutor. KubernetesExecutor creates a pod per task for isolation and per-task resources.