In the fast-paced world of digital innovation, Beavr Labs has always been at the forefront of pushing boundaries and building groundbreaking projects. Our team thrives on the challenges that come with creating cutting-edge solutions. However, there are moments in every developer's journey when unforeseen adversities serve as catalysts for change.
A similar challenge came our way in the form of a Distributed Denial of Service (DDoS) attack, akin to a digital tsunami, crashing against our servers. Loads of traffic flooded one of our projects, overwhelming it to the point of instability. Our once-optimistic morning was now overshadowed by a looming security risk.
We needed to act swiftly and decisively to protect our project and the data entrusted to us. And this is where our journey into the world of Rate Limiting began.
In the world of APIs, unrestricted access is an open invitation to chaos, slowdowns, and security risks. That's where API rate-limiting steps in as a vigilant gatekeeper.
API rate-limiting is like a bouncer at the club. It decides who enters, how often, and how much they can consume, which ultimately ensures fair resource allocation.
In this blog post, we'll take you through our experience—the chaos of the DDoS attack, the realization of the security risk, and how we turned towards Rate Limiting as our shield against future threats.
Is API rate-limiting necessary for real-world applications?
Rate-limiting serves a crucial double role. It's like a protective shield as well as a quality checker for APIs. By imposing limits, the API ensures it doesn't crash under sudden surges in user activity. For instance, launching a popular app using an API can create a massive influx of requests. Rate-limiting prevents this influx from overwhelming the API, safeguarding its reliability.
Whether you're someone who offers APIs and wants to keep things fair and secure, or if you're trying to make sure your apps run and don't cost too much, API rate-limiting is necessary. In this blog, we'll dig deeper into how it works, the different ways to do it, and why it's super important in today's digital world.
Types and methods of API rate-limiting
Rate-Limiting Options:
Rate-limiting from individual sources: When we talk about rate-limiting from individual sources, we mean controlling how much access specific things have. This could be users (based on unique IDs), devices (by IP addresses), places, or even requests to specific servers. It also includes rate-limiting on the number of parallel sessions of any individual unit.Rate-limiting for specific API endpoints: When we talk about rate-limiting for specific APIs, it means that we're looking at the big picture. We're checking all the traffic that's coming into the API from everywhere, making sure that we don't go over the limit we've set for the entire API. Overwhelming an endpoint with traffic is an easy and efficient way to execute a denial of service attack.
Throttling:
Throttling creates a temporary state within the API, allowing selective enforcement of rules on specific request types. It can range from slowing down responses to temporary disconnection.Request Queues:
Request queues provide a simple way to limit the number of requests within defined time intervals. For example, a public transit API sets a rate limit of three requests per second to prevent app overload during rush hours.
Algorithms for Rate-Limiting:
Algorithms add a scientific touch to rate-limiting, catering to various use cases.
Fixed Window: Sets a fixed limit (e.g., 3,000 requests per hour) and blocks additional requests when the limit is reached within the specified period.
Leaky Bucket: First-come, first-served queue ensures the earliest request gets priority.
Sliding Log: Uses time-stamped logs to track user actions, discarding older logs when the rate limit is exceeded.
Sliding Window: This method is like a clever mix of the fixed window and sliding log techniques. It's great for handling lots of requests.
Using Upstash For Efficiently Managing API rate-limiting in our projects
Upstash is a real-time database and caching service that is designed to work with serverless and cloud-native applications with the use of Redis, Kafka, and QStash.
@upstash/ratelimit, is a library solution for rate-limiting in serverless environments, such as Vercel, Cloudflare, Deno, Fastly, and Netlify.
It provides two methods:
limit(identifier: string): Promise<RatelimitResponse>
limit will return true or false and some metadata about remaining requests and can be used if you want to reject all requests beyond your set limit.blockUntilReady(identifier: string, timeout: number): Promise<RatelimitResponse>
In case you don't want to reject a request immediately but want to wait until it can be processed. Remember that some platforms charge you for the execution time of your function.
General Set of Steps for Implementing Upstash Rate-Limiter
We begin by creating an Upstash Database Instance after which we need to copy two env variables from our Upstash console, namely:
UPSTASH_REDIS_REST_URL=https://************************* UPSTASH_REDIS_REST_TOKEN=AXzvA***************************************
These will be used to create a Redis connection instance in our application.
Decide an algorithm desired for rate-limiting.
Define an identifier, which will determine which requests we want to rate-limit by caching it using Redis.
Before carrying out the logical tasks, check if the rate limit for the request has been reached.
For example, in Next.js environment using Typescript, a general way of implementing the above is as follows:
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";
// Create a new ratelimiter, that allows 10 requests per 10 seconds
const ratelimit = new Ratelimit({
redis: Redis.fromEnv(),
limiter: Ratelimit.slidingWindow(10, "10 s"),
});
// Use a constant string to limit all requests with a single ratelimit
// Or use a userID, apiKey or ip address for individual limits.
const identifier = "api";
const { success } = await ratelimit.limit(identifier);
if (!success) { return "Unable to process at this time";}
doExpensiveCalculation();
return "Here you go!";
Integrating Upstash Rate-Limiting into a T3-Stack App
For a brief heads-up into the T3-Stack architecture, you can refer to my previous blogs:
1. https://medium.com/@shagilislam786/getting-started-with-the-t3-stack-part-1-3faf3dbe186a
2. https://medium.com/@shagilislam786/getting-started-with-the-t3-stack-part-2-c216eed7f3df
Use of next-api-decorator and tRPC procedures to efficiently reuse the rate-limiting logic
1. next-api-decorator
Creating serverless functions using classes and decorators in a clear and organized way makes it easier to manage Next.js API routes. This helps keep your /pages/api code neat and easy to understand.
This approach draws inspiration from NestJS, a versatile framework suitable for various scenarios.
import rateLimiter from "@lib/RateLimiter";
import { NextApiRequest, NextApiResponse } from "next";
import { BadRequestException, HttpException, NextFunction, createMiddlewareDecorator } from "next-api-decorators";
import { Session, getServerSession } from "next-auth";
import { options as authOptions } from "pages/api/auth/[...nextauth]";
import Request from "@lib/Request";
export enum RateLimitMethod {
IP,
USER,
NONE,
}
export const RateLimitGuard = (method: RateLimitMethod, maxRequests: number) => createMiddlewareDecorator(
async (req: NextApiRequest, res: NextApiResponse, next: NextFunction) => {
const apiEndpoint = req.url
if(!apiEndpoint) throw new BadRequestException("Could not fetch API endpoint from request.")
const headers = req.headers
const rateLim = rateLimiter.getRateLimiter(maxRequests)
let identifier:string
let ip:string
let session:Session|null
switch (method) {
case RateLimitMethod.IP:
ip = Request.getClientIP(headers);
identifier = `${apiEndpoint}${ip}`;
break;
case RateLimitMethod.USER:
session = await getServerSession(req, res, authOptions);
if(!session) throw new BadRequestException("Could not get Server Session.")
identifier = `${apiEndpoint}${session.user.id}`
break;
case RateLimitMethod.NONE:
identifier = apiEndpoint;
break;
default:
throw new BadRequestException("Invalid rate limiting method");
}
const result = await rateLim.limit(identifier)
if(!result.success) throw new HttpException(
429,
`Max request limit for ${apiEndpoint} reached. Please try again later.`
)
next();
}
)
Here’s an example of how the above decorator can be used in our APIs:
import { Catch, Get, createHandler } from "next-api-decorators";
import ExceptionHandler from "@api/exceptions/exceptionHandler";
import { Logger } from "@api/middleware/Logger";
import { RateLimitGuard, RateLimitMethod } from "@api/middleware/RateLimitGuard";
@RateLimitGuard(RateLimitMethod.NONE, 5)() // TODO: Replace it with AdminGuard
@Catch(ExceptionHandler)
@Logger()
class AdminSigninHandler {
@Get()
public async healthcheck(): Promise<string | null> {
return "healthy";
}
}
export default createHandler(AdminSigninHandler);
2. tRPC middlewares:
For applying Upstash rate-limiting to tRPC endpoints efficiently, we can create reusable procedures (rateLimitedPublicProcedure and rateLimitedAuthProcedure), using the following middleware (rateLimitedMiddleware) in the src/server/trpc.ts
const rateLimitedMiddleware = (maxRequests:number, isAuthenticated:boolean) => t.middleware(async ({ next, ctx }) => {
const headers = ctx.req.headers
const apiEndpoint = ctx.req.url
let identifier:string
if(!apiEndpoint) throw new TRPCError({
code: "BAD_REQUEST",
message: "API Endpoint could not be fetched."
})
if(isAuthenticated) {
const userId = ctx.session?.user.id
identifier = `${apiEndpoint}${userId}`
} else {
const clientIP = Request.getClientIP(headers)
identifier = `${apiEndpoint}${clientIP}`
}
const rateLim = rateLimiter.getRateLimiter(maxRequests)
const res = await rateLim.limit(identifier)
if(!res.success) throw new TRPCError({
code: "TOO_MANY_REQUESTS",
message: `Maximum attempts to ${apiEndpoint} exceeded. Please try again later.`
})
return next({ctx})
})
....
export const rateLimitedPublicProcedure = (maxRequests:number) => t.procedure
.use(rateLimitedMiddleware(maxRequests, false))
export const rateLimitedAuthProcedure = (maxRequests:number) => t.procedure
.use(isAuthed)
.use(rateLimitedMiddleware(maxRequests, true))
.meta({authRequired: true, kycRequired: false, traderRequired: false})
In order to use the above trpc procedures, we can refer to the code below:
import { clerkClient } from "@clerk/nextjs";
import { TRPCError } from "@trpc/server";
import { z } from "zod";
import { rateLimitedPublicProcedure, createTRPCRouter } from "~/server/api/trpc";
export const notesRouter = createTRPCRouter({
create: rateLimitedPublicProcedure(rateLimit: 5)
.input(z.object({ content: z.string(), authorId: z.string() }))
.mutation(async ({ input, ctx }) => {
const note = await ctx.prisma.note.create({
data : {
authorId: input.authorId,
content: input.content
}
});
return note;
})
})
Conclusion
In summary, this blog sheds light on the importance of API rate-limiting in today's digital landscape and provides insights into its implementation using Upstash, ensuring the protection and reliability of digital projects in the face of traffic challenges.