Rate Limiting

API Design

Visual Representation

Rendering diagram...

What is it?

🚦 Think of it like a traffic signal:

A traffic signal controls how many cars enter an intersection per minute. Without it, everyone rushes in at once → gridlock → nobody moves.

Rate limiting is a traffic signal for your API. It controls how many requests a user/client can make in a given time window. "You can make 100 requests per minute. After that, wait." Without it, one misbehaving user (or attacker) could flood your server and ruin the experience for everyone else.

It's like a buffet with a rule: "Max 2 plates per person at a time." Prevents one hungry person from taking ALL the food.

💡 Simple Summary: Rate limiting = controlling how many requests a user can make in a time period to protect your server from overload and abuse.

How it works — Like you're watching it happen

You're using Twitter's API to build a bot:

You send your first request: GET /tweets/search?q=javascript. Response: 200 OK. Headers show: X-RateLimit-Remaining: 99, X-RateLimit-Reset: 1609459200.

You make 99 more requests in the next minute. All succeed.

Request #101: Response: 429 Too Many Requests. Body: {"error": "Rate limit exceeded. Try again in 45 seconds."}

You wait 45 seconds (or check X-RateLimit-Reset timestamp).

New minute starts → counter resets → you can make 100 more requests.

Common algorithms:

Fixed Window — Reset counter every minute. Simple but bursty (100 requests at 0:59 + 100 at 1:01 = 200 in 2 seconds).

Sliding Window — Smoother. Counts requests in the last 60 seconds (rolling window). No burst loopholes.

Token Bucket — Bucket holds tokens (capacity: 100). Each request takes a token. Tokens refill at fixed rate (2/second). Allows SHORT bursts but limits sustained rate.

Leaky Bucket — Requests enter a queue (bucket). They're processed at a fixed rate. If bucket overflows, requests are dropped. Smoothest output rate.

❓ But wait — if I have multiple servers, how does rate limiting work? One server doesn't know what the other counted.

Distributed rate limiting uses a SHARED counter — usually Redis. All servers check/increment the same Redis key (user:123:request_count). Redis is fast enough (100K+ ops/sec) to handle this without becoming a bottleneck. Without shared state, a user could hit each of your 10 servers with 100 requests = 1000 total!

Why should you care? (Interview perspective)

🎯 "How would you prevent API abuse?" — rate limiting is the first answer

"Design a rate limiter" is a common standalone design question

Shows you think about security, fairness, and resource protection

Understanding algorithms (token bucket vs sliding window) shows depth

Key Things to Remember

429 status code — "Too Many Requests." The standard response when rate limited. Include Retry-After header.

Rate limit by what? — Per user (authenticated), per IP (unauthenticated), per API key, per endpoint. Different limits for different tiers.

Token Bucket — Most popular in production. Allows short bursts (bucket drains fast) while limiting sustained rate (tokens refill slowly). AWS, Stripe use this.

Response headers — Always tell the client: X-RateLimit-Limit (max allowed), X-RateLimit-Remaining (how many left), X-RateLimit-Reset (when counter resets).

Tiered limits — Free tier: 100/hour. Pro: 10,000/hour. Enterprise: 100,000/hour. Common in SaaS APIs.

Per-endpoint limits — GET /tweets: 300/15min. POST /tweets: 50/15min. Writes are more expensive, limit them more.

Redis for distributed — All servers share rate limiting state in Redis. Key: "rate:user123:minute42", value: count. INCR + EXPIRE commands.

Client-side handling — Good clients implement exponential backoff: wait 1s, then 2s, then 4s between retries when rate limited.

DDoS vs Rate Limiting — Rate limiting handles normal abuse. DDoS (millions of IPs) needs additional protection (CDN, WAF, IP blocking).

Graceful responses — Don't just return 429. Include: why you're limited, when to retry, what limit you hit, and maybe a link to pricing page for higher limits.

Real Examples You Use Daily

🐦 Twitter API — 300 tweets read/15min, 50 tweet posts/15min per user. Different limits per endpoint. Rate limiting prevents bots from spamming the platform.

💳 Stripe — 100 requests/second in live mode. If you exceed it (maybe a bug in your code creating infinite payment attempts), Stripe stops you before you accidentally charge customers 1000 times.

🔍 Google Maps API — Free tier: 28,500 requests/day. Prevents one developer from consuming all of Google's map resources. Pay more for higher limits.

📱 Instagram — 200 requests/hour per user for their API. Prevents bots from mass-following/unfollowing, automated liking, and scraping.

Common Mistakes in Interviews

❌ Forgetting distributed rate limiting — "I'll count in a variable on the server." But you have 10 servers! User hits each with full limit = 10x abuse. MUST use shared state (Redis).

❌ Not knowing any algorithm — "I'll just count requests per minute." That's fixed window — what about the burst problem? Know token bucket or sliding window and their trade-offs.

❌ Only rate limiting by IP — Users behind corporate NATs share one IP (1000 employees = 1 IP). Rate limiting by IP would unfairly block an entire office. Use user ID when authenticated.

❌ Not including rate limit headers — Clients can't handle rate limiting gracefully if you don't tell them their remaining quota and reset time.

❌ One-size-fits-all limits — All endpoints get the same limit? GET (read, cheap) and POST (write, expensive) should have different limits. Critical endpoints need protection.

🎯 Interview One-Liner

"I'd implement distributed rate limiting using a token bucket algorithm backed by Redis — allowing short bursts while enforcing sustained rate limits — with per-user granularity for authenticated requests, per-IP for anonymous ones, and returning proper 429 responses with Retry-After headers."

Interview Q&A

Q: Design a rate limiter for a distributed system.

Use Redis as the shared counter store. Key: "ratelimit:{user_id}:{window}". Algorithm: sliding window log or token bucket. For each request: INCR the key, check against limit. If exceeded: return 429. Use Redis EXPIRE for automatic cleanup. For token bucket: store last_refill_time and token_count. On each request: calculate tokens added since last_refill, subtract one. If tokens <= 0: reject. This handles multiple servers because Redis is the single source of truth.

Q: Token bucket vs sliding window — when to use each?

Token bucket: when you want to allow SHORT BURSTS (API where users sometimes need 50 requests in 1 second but average 10/sec). The bucket "fills up" during idle time, allowing bursts. Sliding window: when you want EVEN distribution (payment processing where you want exactly ≤ 100 charges per minute, no bursts). Sliding window is stricter about spreading requests evenly over time.

Q: How do you rate limit when your API has millions of users?

Redis handles this easily: each user gets a key, Redis supports millions of keys with sub-millisecond access. Use Redis Cluster for even more scale. Memory estimate: 1 million users × 100 bytes per key = 100 MB (trivial for Redis). Use EXPIRE on keys so inactive users are automatically cleaned up. The bottleneck is never the rate limiter itself — it's the API behind it.

Q: A user gets rate limited. What should the response look like?

HTTP 429 with headers: X-RateLimit-Limit: 100, X-RateLimit-Remaining: 0, X-RateLimit-Reset: 1609459260 (Unix timestamp), Retry-After: 45 (seconds). Body: {"error": "rate_limit_exceeded", "message": "You've exceeded 100 requests per minute. Please retry after 45 seconds.", "documentation_url": "https://api.example.com/docs/rate-limits"}. Be helpful, not hostile.

Q: How do you handle rate limiting for premium vs free users?

Different rate limit tiers stored in user metadata. When request arrives: look up user's plan (cached in Redis), apply corresponding limit. Free: 60/min. Pro: 600/min. Enterprise: 6000/min. This is enforced at the API Gateway level. Same algorithm (token bucket), different bucket capacity. Also: enterprise gets priority queuing during overload.

Quick Quiz

1/5

You hit Twitter's API and get back HTTP 429. What does this mean?

API Pagination Idempotency