Load Balancing with Kong API Gateway using Docker
In quest of breaking the inflexibility of the monolithic application architecture, developers are turning in large numbers to microservices. Microservices are small, independent loosely coupled modules within large software which communicate with each other via APIs. Microservice architecture provides fault tolerance, enables continuous delivery and ability to scale across multiple regions to ensure high availability.
In this article, we will show an example of how Kong a popular open-source API gateway, built on NGINX at its core, one of the most popular HTTP servers is used for load balancing. With Kong, you can get more control to handle authentication, rate limiting, data transformation, among other things from a centralized location even though you have multiple microservices than a typical load balancer.
Why Load balancing?
Load balancing is a key component of highly available infrastructure which is used to distribute users’ request load across multiple services among healthy hosts so that no host get overloaded.
The single point of failure could be mitigated by introducing a load balancer and a replica of the service. Every service will return an identical response so the users receive consistent response irrespective of which service was called.
Kong provides different ways of load balancing requests to multiple services - a DNS-based method, round-robin method and a hash-based balancing method. A DNS-based method will configure a domain in DNS in such a manner that the user requests to the domain are distributed among a group of services. Round-robin method will distribute the load to the host evenly and in order. When using hash-based balancing method, Kong will act as a service registry which handle the adding and removing of the services with a single HTTP request.
Setting up Kong locally
Kong is available to install in multiple operating environments. Set up Kong based on the environment you’re currently using. More details on how to install kong
Setting up Services
We are going to start two services which will be two NGINX instances. We will pull the OpenResty official docker image based on NGINX and LuaJIT.
docker pull openresty/openresty
Let’s start two services and map it to different ports
docker run -p 9001:9001 -d openresty/openresty
docker run -p 9002:9002 -d openresty/openresty
Before configuring the services we will install the packages.
docker exec -it <container> /bin/bash
apt-get update
apt-get install vim
apt-get install libnginx-mod-http-ndk nginx-common libnginx-mod-http-lua
Now we will configure the services with two endpoints / and /profile which we will authenticate and listen to port 9001 and 9002 respectively. We will update etc/nginx/conf.d/default.conf the config file.
listen <port_number>;
location / {
return 200 "Hello from my service";
}
location /profile {
content_by_lua_block {
local cjson = require "cjson"
local consumer_id = ngx.req.get_headers()["X-Consumer-ID"]
if not consumer_id then
ngx.status = 401
ngx.say("Unauthorized")
return
end
ngx.header["Content-Type"] = "application/json"
ngx.say(cjson.encode{
id = consumer_id,
username = "moesif",
description = "Advanced API Analytics platform"
})
}
}
We will restart the services to ensure the latest config changes are reflected.
docker restart <container>
We will quickly see what the services response if we hit them based on the config.
# Service A
curl -XGET 'http://localhost:9001'
# Service B
curl -XGET 'http://localhost:9002'
The idea is to have both services be replica of each other so we can load balance our traffic between them.
Start the Kong
We will start Kong with the idea to proxy the traffic through Kong as a reverse proxy instead of hitting the NGINX instances directly.
To achieve this we will have to make a requests to the proxy port 8000 and the admin port 8001.
Create a service
We will create a service with name moesif-service.
curl -i -X POST \
--url http://localhost:8001/services/ \
--data 'name=moesif-service' \
--data 'url=http://localhost:9001'
Create a route
We will create a route that will accept traffic and send all the traffic received from that route to the service created above.
curl -i -X POST \
--url http://localhost:8001/routes \
--data 'paths[]=/' \
--data 'service.id=<service-id>' \
-f
At this point we are able to proxy traffic through Kong to port 9001 service.
curl -i -X GET \
--url http://localhost:8000/
Awesome, we are actually proxying traffic over Kong to the upstreams as the Kong will add few headers in the response.
We could also proxy traffic to any upstream endpoint which makes Kong fully transparent as it doesn’t care which request we are sending in, it will proxy that to the upstreams as long as it matches the rule we have defined in the route. In our case it is / which basically means every request going through Kong will be proxy to port 9001 service.
Now let’s talk about the other endpoint /profile which we had defined in the upstream.
If we want to enforce every request goes through /profile upstream is authenticated, we can create a route and apply key-auth plugin on top that route.
Create a route
curl -i -X POST \
--url http://localhost:8001/routes \
--data 'paths[]=/profile' \
--data 'service.id=<service-id>' \
-f
To be as flexible as possible Kong has a configuration parameter which defines whether we strip the request path or not, which is defaults to true. This means /profile is proxied as / upstream. So, we have to disable that with a simple PATCH request.
curl -i -X PATCH \
--url http://localhost:8001/routes/<service-id> \
--data 'strip_path=false'
Now if we make a request to /profile it will correctly hit the /profile in the upstream.
curl -i -X GET \
--url http://localhost:8000/profile
Now let’s apply the key-auth plugin to the /profile route.
curl -i -X POST \
--url http://localhost:8001/routes/<service-id>/plugins \
--data 'name=key-auth'
We need to pass the API key when calling the /profile endpoint for that we will have to create the Consumer.
curl -i -X POST \
--url http://localhost:8001/consumers \
--data 'username=moesif'
To get the API key we will make a POST request
curl -i -X POST \
--url http://localhost:8001/consumers/<username>/key-auth
We make a request to /profile with API-key as the header,
curl -i -X GET \
--url http://localhost:8000/profile \
--header 'apikey:<apikey>'
We could see that the /profile is authenticated and returns the response as configured in the upstream.
You’ve observed that the another service is sitting idle, is not receiving any traffic. The goal here is to load balance the traffic between both the services.
We are not going to use the DNS-based load balancing but will manually do it using upstreams which is similar to upstream module in NGINX. The only parameter we have to give is the name in our case it is localhost. So every service that has a localhost as it’s hostname will be proxied through that upstream, which will make load balancing decision.
curl -i -X POST \
--url http://localhost:8001/upstreams \
--data 'name=localhost'
We will add targets for both the port and start proxying load-balancing traffic between the services.
curl -i -X POST \
--url http://localhost:8001/upstreams/<upstream-id>/targets \
--data 'target=localhost:9001'
Now from this point if we make a request through kong it will load-balance between the two services.
If we take one of the services down, Kong will use the retry mechanism. But in certain cases if we exhaust all of the retries on the same upstream, it is possible to still have errors. While monitoring the Kong logs, you can see the decision Kong is making. It’s trying to talk to second upstream but couldn’t and retries the request to the first upstream. The size of our load balancing slots is quite large, it might take a while to hit another service. In that case we have to continue sending more requests.
To understand how your APIs are used, capture API requests and responses and log to Moesif for easy inspecting and real-time debugging of your API traffic via Kong For more detail about how to install Moesif plugin, check out this approach.
Meanwhile, if you have any questions, reach out to the Moesif Team
Related Articles
API Analytics and Monitoring
Introducing AI Explain
We are thrilled to introduce our newest feature to Moesif - AI Explain - a powerful conversational tool that lets you interact with your API analytics data l...
API Analytics and Monitoring
Monitoring Cost and Consumption of AI APIs and Apps
Discover how Moesif can help manage and optimize your AI-driven APIs with powerful observability tools that track costs and usage efficiently.
API Development
Unlock Deep API Insights with Moesif’s HTTP Body Analytics
Learn how to use Moesif to analyze HTTP bodies.
API Analytics and Monitoring
How Moesif API Observability Compares to Treblle
Learn how Moesif’s API observability platform compares to Treblle.