Circuit Breakers and Outlier Detection

Closed, open, half-open

A circuit breaker watches calls to a dependency. In the closed state calls pass normally. After enough failures it opens: calls are rejected immediately (fast fail) without hitting the struggling service, which gives it room to recover and stops callers wasting time. After a cool-down it goes half-open and lets a trial call through: success closes the circuit, failure reopens it. Meshes and gateways add related protections: outlier detection (temporarily ejecting a misbehaving instance from the pool), connection and request limits (bulkheads) and load shedding. Always provide a sensible fallback (cached data, a default, a clear error) for when the circuit is open.

A circuit breaker state machine, run

I ran this plain-Python model. Three failures open the circuit at t=2. Calls at t=3 and t=4 are rejected instantly. After the 10-second cool-down the call at t=13 is allowed as a trial, succeeds, and closes the circuit.

import hashlib, bisect, math, random

class Breaker:
    def __init__(self, threshold=3, cooldown=10):
        self.threshold, self.cooldown, self.fails, self.state, self.opened_at = threshold, cooldown, 0, "closed", None
    def call(self, now, ok):
        if self.state == "open":
            if now - self.opened_at >= self.cooldown: self.state = "half-open"
            else: return "rejected (fast fail)"
        if ok:
            self.fails = 0; self.state = "closed"; return "ok"
        self.fails += 1
        if self.state == "half-open" or self.fails >= self.threshold:
            self.state, self.opened_at = "open", now
        return "failed"
b = Breaker()
events = [(0, False), (1, False), (2, False), (3, True), (4, True), (13, True), (14, True)]
print([(t, b.call(t, ok), b.state) for t, ok in events])

Output:

[(0, 'failed', 'closed'), (1, 'failed', 'closed'), (2, 'failed', 'open'), (3, 'rejected (fast fail)', 'open'), (4, 'rejected (fast fail)', 'open'), (13, 'ok', 'closed'), (14, 'ok', 'closed')]

Envoy-style protections (illustrative)

In a mesh these are declared as policy rather than coded. This Istio DestinationRule limits connections and ejects instances that return repeated 5xx errors. Field names follow Istio; not run here.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata: { name: orders }
spec:
  host: orders
  trafficPolicy:
    connectionPool:
      http: { http1MaxPendingRequests: 50, maxRequestsPerConnection: 100 }
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 10s
      baseEjectionTime: 30s
      maxEjectionPercent: 50

Quick check: What does an open circuit breaker do?

• Encrypts the call
• Sends all traffic faster
• Deletes the service
• Rejects calls immediately to protect the failing service and callers