# Logging, Metrics, Tracing & Alerting — System Design

Source: https://www.geekswithgeeks.com/en/system-design/sd-observability

> Knowing what your system is doing before users tell you it's broken.

## The three pillars

**Logs** are timestamped discrete events (good for 'what exactly happened here'). **Metrics** are aggregated numbers over time (good for trends and dashboards — request rate, error rate, latency). **Traces** follow one request across every service it touched (good for finding where in a call chain the time went).

## The golden signals

For any service, watch **latency** (p50/p95/p99, not just average), **traffic** (requests/sec), **errors** (rate of failed requests), and **saturation** (how full is CPU, memory, queue depth). These four catch most incidents early.

## A car dashboard

Metrics are the speedometer and fuel gauge — glance and know your state. Logs are the trip computer's detailed event history. Tracing is like a black box recorder that shows the exact sequence of what each part of the car did leading up to a specific trip.

## Alert on symptoms, not causes

Alert on user-facing symptoms (error rate spiked, p99 latency breached SLO) rather than every internal cause (one server's CPU is high). Symptom-based alerts reduce noise and point straight at what users are experiencing.