Imagine you're running a distributed Python application, with multiple instances concurrently hitting the same external API, and suddenly, you're faced with a barrage of 429 "Too Many Requests" errors. This scenario is all too common, and it's a direct result of naive client-side rate limiting failing to account for the distributed nature of modern applications. As someone who has worked with AI agents and microservices, I've seen firsthand how local rate limiting solutions like `asyncio.sleep` or `tenacity` are insufficient for managing API requests across multiple instances. Today, I'll share my experience on how to build a truly production-grade, asynchronous, and centralized rate limiter using Redis and Lua scripting, ensuring that all your application instances respect API limits without sacrificing concurrency.
Key Takeaways
- Local rate limiting solutions are insufficient for distributed applications, leading to API throttling and service instability.
- Redis and Lua scripting provide a robust foundation for building a centralized, asynchronous rate limiter.
- By leveraging Redis's atomic operations and Lua scripting, developers can ensure accurate and efficient rate limiting across multiple application instances.
The Problem
Distributed Python applications, especially those utilizing microservices or AI agents, frequently interact with numerous external APIs. The lack of a centralized, atomic, and asynchronous rate limiting mechanism leads to costly API throttling, IP bans, or inconsistent behavior across concurrent workers. This post addresses the critical need for a robust rate limiting solution to maintain service stability and API compliance.
Data and Sources
The Cloudflare Blog RSS feed (https://blog.cloudflare.com/rss/) serves as a real-world example of an external API that may be subject to rate limiting. Data accessed on 2026-07-27.
Loading the Data
To demonstrate the effectiveness of the rate limiter, we'll fetch entries from the Cloudflare Blog RSS feed using `aiohttp` and `feedparser`.
import aiohttp
import feedparser
async def fetch_rss():
async with aiohttp.ClientSession() as session:
async with session.get("https://blog.cloudflare.com/rss/") as response:
data = await response.text()
feed = feedparser.parse(data)
return feed.entries
The Core Logic
The core logic involves building a sliding window counter using Redis Lua scripting. This ensures that checking the count and adding a new timestamp are atomic operations, preventing race conditions.
import aioredis
async def rate_limit(lua_script, redis_client, key, window_size):
# Remove timestamps older than the window
# Add current timestamp
# Set expiration for the key to ensure cleanup
# Get current count
return await redis_client.eval(lua_script, 1, key)
Building the Sliding Window Counter with Redis Lua
The Redis Lua script for the sliding window counter is as follows:
local key = KEYS[1]
local window_size = ARGV[1]
local now = tonumber(ARGV[2])
-- Remove timestamps older than the window
local old_timestamps = redis.call('zremrangebyscore', key, 0, now - window_size)
-- Add current timestamp
redis.call('zadd', key, now, now)
-- Set expiration for the key to ensure cleanup
redis.call('expire', key, window_size + 1)
-- Get current count
local count = redis.call('zcard', key)
return count
Integrating the Limiter into Python with `aioredis`
We'll develop an `AsyncRateLimiter` class in Python that executes the Lua script via `aioredis.eval`. This will allow us to create a reusable decorator for easy application to `async` functions.
import aioredis
class AsyncRateLimiter:
def __init__(self, redis_client, lua_script, key, window_size):
self.redis_client = redis_client
self.lua_script = lua_script
self.key = key
self.window_size = window_size
async def __call__(self, func):
async def wrapper(*args, **kwargs):
count = await self.redis_client.eval(self.lua_script, 1, self.key, self.window_size, int(time.time()))
if count >= self.window_size:
await asyncio.sleep(1)
return await func(*args, **kwargs)
return wrapper
Complete Script
The full runnable script combining all steps:
import aiohttp
import feedparser
import aioredis
import asyncio
import time
lua_script = """
local key = KEYS[1]
local window_size = ARGV[1]
local now = tonumber(ARGV[2])
-- Remove timestamps older than the window
local old_timestamps = redis.call('zremrangebyscore', key, 0, now - window_size)
-- Add current timestamp
redis.call('zadd', key, now, now)
-- Set expiration for the key to ensure cleanup
redis.call('expire', key, window_size + 1)
-- Get current count
local count = redis.call('zcard', key)
return count
"""
async def fetch_rss():
async with aiohttp.ClientSession() as session:
async with session.get("https://blog.cloudflare.com/rss/") as response:
data = await response.text()
feed = feedparser.parse(data)
return feed.entries
class AsyncRateLimiter:
def __init__(self, redis_client, lua_script, key, window_size):
self.redis_client = redis_client
self.lua_script = lua_script
self.key = key
self.window_size = window_size
async def __call__(self, func):
async def wrapper(*args, **kwargs):
count = await self.redis_client.eval(self.lua_script, 1, self.key, self.window_size, int(time.time()))
if count >= self.window_size:
await asyncio.sleep(1)
return await func(*args, **kwargs)
return wrapper
async def main():
redis_client = aioredis.from_url("redis://localhost")
limiter = AsyncRateLimiter(redis_client, lua_script, "rss_feed", 10)
@limiter
async def fetch_and_print():
entries = await fetch_rss()
for entry in entries:
print(entry.title)
await fetch_and_print()
if __name__ == "__main__":
asyncio.run(main())
Expected Output
When you run the script, you should see the titles of the RSS feed entries printed to the console, with the rate limiter ensuring that the requests are paced to respect the API limits.
Limitations and Tradeoffs
While this approach provides a robust solution for distributed rate limiting, it may be over-engineered for single-instance or low-concurrency applications. Additionally, the overhead of Redis round-trips for extremely high-frequency limits may become a bottleneck. Furthermore, clock skew issues in distributed systems may affect the accuracy of the rate limiter. It's essential to consider these tradeoffs and adjust the implementation accordingly.
Frequently Asked Questions
What is the purpose of using Lua scripting in Redis?
Lua scripting provides a way to execute atomic operations in Redis, ensuring that multiple commands are executed as a single, all-or-nothing unit. This is crucial for implementing a rate limiter, as it prevents race conditions and ensures accurate counting.
How does the rate limiter handle concurrent requests?
The rate limiter uses Redis's atomic operations to ensure that concurrent requests are handled correctly. The Lua script executes a series of commands as a single unit, preventing other requests from interfering with the count.
What happens if the Redis connection is lost?
If the Redis connection is lost, the rate limiter will fail to function correctly. It's essential to implement connection retry logic and ensure that the Redis instance is highly available to prevent this scenario.
What I'd Change
In conclusion, building a production-grade distributed rate limiter with Redis and Lua scripting is a critical component of ensuring service stability and API compliance in modern distributed applications. While this implementation provides a robust solution, I would consider adding more advanced features, such as automatic retry logic and more sophisticated rate limiting algorithms, to further improve the system's resilience and performance.
