Rate limiting

Why limit requests

Without limits, one person or one bot can send thousands of requests a second. That can slow the app for everyone, cost you money, or help an attacker guess passwords.

• Abuse: bots guessing passwords or OTPs on the login API.
• Cost: every OTP SMS costs money, so SMS-bombing a number is expensive for you and annoying for the victim.
• Fair use: one heavy user of a free API should not slow down everyone else.
• Protection: a buggy client in a retry loop should not take your server down.

Three common methods

Fixed window: count requests in fixed blocks of time, like 100 per minute starting at 10:00:00, 10:01:00 and so on. It is simple and cheap. Its weakness: someone can send 100 at 10:00:59 and 100 more at 10:01:00, so 200 pass in two seconds.

Sliding window: look back over the last full period from now, for example "how many requests in the last 60 seconds?". This removes the burst at the window edge. It needs a little more memory or a smart estimate.

Token bucket: each user has a bucket that holds up to N tokens. Every request takes one token. Tokens drip back at a steady rate, like 1 per second. If the bucket is empty, the request is refused. This allows short bursts (a full bucket) but a steady average rate.

   +1 token every second
          |
          v
   |  o  o  o  |   bucket: up to 5 tokens
   |  o  o     |
   +-----------+
          |
          v
   each request takes 1 token
   empty bucket -> 429 Too Many Requests

What to limit by

You need a key: who is this request from? For logged-in users, use the user id. For public APIs, use the API key. For login and OTP pages, use both the IP address and the phone number or email, because attackers change IPs. Different routes need different limits: login might be 5 per 15 minutes, while reading products might be 300 per minute.

Telling the client

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/json

{"error":"Too many OTP requests. Try again in 30 seconds."}

Where to put it

• API gateway or CDN (for example a cloud API gateway): stops floods before they reach your servers.
• Nginx: has a built-in limit_req feature for simple per-IP limits.
• Middleware in your app: knows the user id and the route, so it can do smart limits.
• Redis: when you run many app servers, the counters must be shared, so they live in Redis.

async function allow(userId) {
  const windowId = Math.floor(Date.now() / 60000); // which minute is it?
  const key = 'rl:' + userId + ':' + windowId;
  const count = await redis.incr(key);              // add 1, get the new count
  if (count === 1) await redis.expire(key, 60);     // clean up after the minute
  return count <= 100;                              // 100 requests per minute
}