January 10, 2022 API Development

What is HTTP middleware? Best practices for building, designing and using middleware.

In order to capture API calls from arbitrary environments, we had to create middleware for many of the common web API frameworks. Some of these learnings are presented here to help those creating middleware also.

In particular, we created middleware for the following frameworks:

Introduction to HTTP middleware

What is middleware?

I use the term middleware, but each language/framework calls the concept differently. NodeJS and Rails calls it middleware. In the Java enterprise world (i.e. Java Servlet), it’s called filters. C# calls it delegate handlers. Essentially, the middleware performs some specific function on the HTTP request or response at a specific stage in the HTTP pipeline before or after the user defined controller. Middleware is a design pattern to eloquently add cross cutting concerns like logging, handling authentication, or gzip compression without having many code contact points.

Since these cross-cutting concerns are handled in middleware, the controllers/user defined handlers can focus on the core business logic.

What can be done with middleware

Middleware is generally pretty flexible. Some middleware is passive such as logging middleware. Other middleware such as gzip compression can perform transforms on the request or response body. Middleware can add HTTP headers, add internal flags for use by your business logic, etc. It’s an implementation of the pipelines design pattern.

In fact, even basic framework features like body parsing can be considered middleware. You can add custom body parsing for various binary protocols if the ones included with a framework don’t fit your use case.

You can even call other services such as looking up a session token in Redis, or performing a GeoIP lookup from a MaxMind dataset.

Key things to consider

One of the first things to consider if something you want do to should be a middleware. So middleware is best for cross cutting concerns. If it is a specific business logic, only applies to very few cases then, perhaps middleware isn’t the way to go.

If your app have many common tasks (such as logging, authentication, parsing JSON or add some common share data to every request or response), then refactoring out that logic into middleware makes sense.

Performance

Due to the cross-cutting nature of middleware, ensuring it’s performant is particularly important as any added latency could impact the entire application. Latency could come from I/O such as disk or network access.

For example, if you create authentication middleware for your API that needs to lookup user information in a SQL Db, that I/O read will stall the HTTP request pipeline until a response is received from the SQL Db. That added latency will still be seen by clients even with non-blocking frameworks like NodeJS.

Primer on Latency vs bandwidth

Latency vs bandwidth are two orthogonal, yet related metrics. Bandwidth refers to metrics such as Request Per Second or maximum number of active clients or connections at a time. Latency, on the other hand, refers to metrics like response time to first byte, or average time to perform a p[articular query. They are related however, since an increase in latency can also reduce bandwidth of a system. For example, an increase latency may also increase resident time of queues.

Generally, non-blocking frameworks such as NodeJS focus mostly on increasing the bandwidth of an application by not tying up resources (i.e. Threads). Similar to adding an additional VM or load balancer to handle additional traffic. However, each individual user or client may still experience high latency regardless of the bandwidth of the system or using a non-blocking or a blocking architecture.

For our example, a better solution may be to use an in-memory hash table like Redis looking up via session token. An even lower latency solution would be to use JSON Web Tokens (JWT) where secure authentication and authorization requires only CPU cycles.

Some frameworks like Node is designed bottoms up to be non-blocking IO. However, in other frameworks (such as PHP or Ruby on Rails), it is important to do IO or processing of heavy tasks on background threads.

Passive middleware

Even with the limitations of NIO described above, that’s not to say it’s not beneficial in reducing latency. Especially for passive middleware with write only I/O, architecting your middleware to be non-blocking or asynchronous is ideal. For example, the middleware for Moesif does not modify the response, so there is no reason for the HTTP pipeline to wait for any writes to Moesif. Thus, we designed the Moesif middleware to be asynchronous.

For blocking frameworks such as PHP Laravel, you may have to implement your own methodology to be asynchronous. For the Moesif Laravel middleware, we leveraged a unix fork to handle sending any data to Moesif in a separate process.

Forking a process in linux is surprisingly a lightweight task. For more info on why, please read What every web developer should know about CPU Arch

Ordering

Each middleware is instantiated as one step in a pipeline, thus ordering may be important for functional correctness. However, even if two middleware can be re-ordered, you should think about the performance or security implications. Lightweight middleware that have a short circuited return path should go before heavier middleware.

As an example, middleware that redirects non-HTTPS to HTTP domain via a 301 Permanent Redirect should be placed before middleware that decompresses and parses the request body. Otherwise, you’re wasting CPU cycles decompressing a body that will be thrown away.

Think about security and DDoS protection also. If only a specific whitelist of IP addresses are allowed to access your API, and all others are denied, you probably want to check the IP address before performing expensive operations such as querying account information in a database.

Underlying Frameworks

Often times, the framework you are using is actually built on top of another framework. For example, Ruby on Rails is built on top of Rack. So when you are building a rails middleware, you are actually building one for Rack.

This can be used for your advantage. At Moesif, we didn’t need to create a separate middleware for every Java framework. Instead, we released a single Java Servlet SDK. Many modern Java frameworks are built on Java Servlet such as Spring, Struts, and Jersey. Sometimes, you have the option of building on the base framework like Java Servlet or the higher level web framework such as Spring.

Closing Thoughts

Build middleware can force you to dig into more underlying technology of each framework. Many of the common services (such as authentication and body parsing) of a framework are implemented as middleware.

Do you spend a lot of time debugging customer issues?
Moesif makes debugging easier for RESTful APIs and integrated apps

Learn More

Django , PHP , Platforms , Python , Rails , Ruby

Xing Wang

Co-founder @Moesif. Previously worked @Zynga & @Microsoft. Studied @MIT.

API Strategy

Announcing General Availability of the Moesif Developer Portal

We’re thrilled to announce the General Availability of the Moesif Developer Portal, a major leap forward enabling you to deploy a seamless experience for dev...

December 06, 2024

API Analytics and Monitoring

Introducing AI Explain

We are thrilled to introduce our newest feature to Moesif - AI Explain - a powerful conversational tool that lets you interact with your API analytics data l...

November 14, 2024