The Demo Environment

A real gateway with fake services

To make the ideas concrete, I built a small environment in Docker: an nginx 1.27 gateway in front of five tiny Node.js upstream services (orders-v1, orders-v2, flaky that always returns 503, slow that takes 3 seconds, and echo that reflects the request). They share a Docker network, and a Node client calls the gateway. Every output in this course came from running this setup. nginx is used as the gateway because it is widely deployed; the concepts (routing, auth, rate limits, retries, timeouts) apply to Kong, Envoy, Traefik, cloud gateways and others, though configuration syntax differs.

Send each request to the right place

Routing rules match on host, path or header and can split traffic between versions.

The upstream service (Node.js)

One script plays every role depending on the MODE environment variable. It runs with node -e "$(cat up.js)" inside a node:22-alpine container.

const http = require('http');
const mode = process.env.MODE, name = process.env.NAME || mode;
let n = 0;
http.createServer((req, res) => {
  n++;
  const send = (code, body) => { res.writeHead(code, {'Content-Type': 'application/json'}); res.end(JSON.stringify(body)); };
  if (mode === 'echo') return send(200, {service: name, path: req.url, request_id: req.headers['x-request-id'] || null, forwarded_for: req.headers['x-forwarded-for'] || null, host: req.headers['host'], user: req.headers['x-user'] || null});
  if (mode === 'flaky') return send(503, {service: name, error: 'unavailable'});
  if (mode === 'slow') return setTimeout(() => send(200, {service: name, slow: true}), 3000);
  return send(200, {service: name, version: process.env.VERSION, n});
}).listen(8080);

The gateway configuration (nginx)

This is the exact file the gateway ran with. The next sections explain each location block. nginx -t reported the syntax as ok.

worker_processes 1;
events { worker_connections 256; }
http {
  log_format gw '$status $request_method $uri -> $upstream_addr';
  access_log off;

  map $http_x_api_key $api_ok { default 0; "demo-key-123" 1; }
  map $http_x_canary $orders_pool { default orders_stable; "1" orders_canary; }

  limit_req_zone $binary_remote_addr zone=perip:1m rate=5r/s;
  limit_req_status 429;

  upstream orders_weighted { server orders-v1:8080 weight=9; server orders-v2:8080 weight=1; }
  upstream orders_stable { server orders-v1:8080; }
  upstream orders_canary { server orders-v2:8080; }
  upstream resilient { server flaky:8080; server orders-v1:8080; }
  upstream echo_up { server echo:8080; }
  upstream slow_up { server slow:8080; }

  server {
    listen 80;
    resolver 127.0.0.11 valid=5s;

    location = /health { return 200 "ok\n"; }

    # 1. weighted canary split 90/10
    location /orders/ { 
      if ($api_ok = 0) { return 401 '{"error":"missing or invalid API key"}\n'; }
      proxy_pass http://orders_weighted/;
    }
    # 2. header-based routing (opt in to the canary with X-Canary: 1)
    location /pick/ { proxy_pass http://$orders_pool/; }
    # 3. rate limited public route
    location /public/ { limit_req zone=perip burst=5 nodelay; proxy_pass http://orders_stable/; }
    # 4. headers added by the gateway + prefix stripping
    location /api/ {
      proxy_set_header X-Request-ID $request_id;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-User "asha";
      proxy_pass http://echo_up/;
    }
    # 5. retry the next upstream on 503
    location /resilient/ { proxy_next_upstream error timeout http_503; proxy_pass http://resilient/; }
    # 6. timeout to a slow upstream
    location /slow/ { proxy_read_timeout 1s; proxy_pass http://slow_up/; }
  }
}

The test client (Node.js)

It runs inside the same Docker network and prints the results shown throughout this course.

const base = 'http://gwg-gw';
const get = async (p, h = {}) => { const r = await fetch(base + p, {headers: h}); let b = null; try { b = await r.json(); } catch {} return [r.status, b]; };
(async () => {
  console.log('--- auth at the gateway');
  console.log((await get('/orders/x'))[0], JSON.stringify((await get('/orders/x'))[1]));
  console.log((await get('/orders/x', {'X-API-Key': 'wrong'}))[0], (await get('/orders/x', {'X-API-Key': 'demo-key-123'}))[0]);

  console.log('--- weighted canary 90/10 over 100 requests');
  const c = {};
  for (let i = 0; i < 100; i++) { const [, b] = await get('/orders/x', {'X-API-Key': 'demo-key-123'}); c[b.version] = (c[b.version] || 0) + 1; }
  console.log(JSON.stringify(c));

  console.log('--- header-based routing');
  console.log((await get('/pick/x'))[1].version, (await get('/pick/x', {'X-Canary': '1'}))[1].version);

  console.log('--- headers and prefix stripping');
  const [, e] = await get('/api/v1/users?id=7'); console.log(e.path, e.user, !!e.request_id, e.request_id && e.request_id.length, e.forwarded_for !== null);

  console.log('--- retry on 503 (flaky first, healthy second)');
  const rs = []; for (let i = 0; i < 4; i++) { const [st, b] = await get('/resilient/x'); rs.push(st + ' ' + b.service); } console.log(rs.join(', '));

  console.log('--- upstream timeout');
  const t0 = Date.now(); const [s504] = await get('/slow/x'); console.log(s504, Math.round((Date.now() - t0) / 100) / 10 + 's');

  console.log('--- rate limiting: 20 rapid requests');
  const codes = await Promise.all(Array.from({length: 20}, () => get('/public/x').then(r => r[0])));
  const tally = {}; codes.forEach(c => tally[c] = (tally[c] || 0) + 1); console.log(JSON.stringify(tally));
})();

Quick check: Why were fake upstream services used for the demos?

• Real services cannot be proxied
• They let the gateway behaviour be shown repeatably, including failures and slowness
• Gateways only work with Node.js
• To avoid learning HTTP