Thread Nginx rate limiting — how to handle burst traffic without dropping legitimate requests?

I am seeing frequent 503s under high load because nginx_limit_req is too aggressive. What are the best practices for tuning r/s and burst parameters, and should I consider a different approach like an upstream cache or request queuing instead of simple rate limiting?

leaky bucket works well for this. set a reasonable burst size and let it fill naturally instead

Two-layer approach is usually what people want when they ask this:

1. Use limit_req with a burst parameter to absorb short spikes. For example, if you're okay with 20 requests/sec sustained but can handle bursts of 50 within a second, use:

limit_req_zone $binary_remote_addr zone=bursty:10m rate=20r/s;

server {
    limit_req zone=bursty burst=50 nodelay;
}

nodelay is key — it lets the entire burst through immediately and only rates-limits beyond that. The cost is that a sustained attack at 100 r/s will drop requests rather than queueing them, which may be what you want for security but not if every request is mission-critical.

1. If dropping any legitimate traffic is unacceptable during bursts, move the limit

The standard approach is to use two tiers of limit_req:

1. A soft limit for normal traffic with a generous burst buffer (burst=20) — this absorbs short bursts without dropping requests. The leaky-bucket behavior means excess requests queue up and process at the rate defined by rate, rather than being dropped immediately.

2. A hard limit (or simply higher bucket size) for legitimate spikes that don't exceed reasonable thresholds.

For burst traffic you can't predict:

limit_req_zone $binary_remote_addr zone=burst_bucket:10m rate=50r/s;
limit_req zone=burst_bucket burst=32 nodelay delay=0;

burst=32 with nodelay means the first 32 requests above the rate are processed immediately — they sit in a buffer and get served at whatever capacity Nginx has