Rate Limiting

Smoothing the request rate

A bounded worker pool limits how many operations run at once; a rate limiter limits how often they start. The simplest limiter is a time.Ticker: wait for a tick before each request, giving a steady rate with no bursts. For production use, golang.org/x/time/rate implements a token bucket: rate.NewLimiter(r, b) refills at r tokens per second (use rate.Every(d) to express one token per interval) and holds at most b tokens, allowing bursts of up to b. limiter.Wait(ctx) blocks until a token is available or the context is done; Allow() returns immediately with true or false, which suits rejecting excess HTTP requests; Reserve() tells you how long to wait. One limiter is safe to share between goroutines.

A ticker limiter and a token bucket

Both throttle calls to an external API.

package main

import (
	"context"
	"fmt"
	"time"

	"golang.org/x/time/rate"
)

func callAPI(i int) { fmt.Println("call", i) }

func main() {
	// 1. Ticker: one call every 200ms, no bursts.
	t := time.NewTicker(200 * time.Millisecond)
	defer t.Stop()
	for i := range 3 {
		<-t.C
		callAPI(i)
	}

	// 2. Token bucket: 5 per second on average, bursts of up to 10.
	lim := rate.NewLimiter(rate.Limit(5), 10)
	ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
	defer cancel()
	for i := range 20 {
		if err := lim.Wait(ctx); err != nil { // blocks or fails on ctx
			fmt.Println("stopping:", err)
			return
		}
		callAPI(i)
	}
}

Limit per client, not only globally

An HTTP server often keeps one limiter per API key or client IP, stored in a mutex-protected map with eviction of idle entries, so one noisy client cannot use the whole budget.

Quick check: In rate.NewLimiter(r, b), what does b control?

• The number of goroutines that may call Wait
• The bucket size, i.e. the maximum burst of events allowed at once
• The timeout for Wait
• The number of seconds between tokens