February 13, 2025 API Development

Achieving API observability in Kong Konnect with Moesif API analytics plugin

Kong Gateway offers a fast, scalable, robust, and modular API gateway for your application programming interfaces (APIs), irrespective of your cloud architecture. Kong Konnect adds more benefits on top of Kong Gateway by providing a centralized platform to manage your gateways. However, even with such a powerful platform, you’ll face challenges in achieving that benchmark of API observability and monitoring to meet modern industry standards and therefore your growth projections.

As engineers, you often encounter visibility gaps that impedes effective troubleshooting and performance optimization. Even if you’re utilizing the world’s fastest API gateway, without granular insights and the ability to act on them, you can’t truly understand how your APIs are performing in production in real time.

Fortunately, you can remediate these issues completely by integrating Moesif’s API analytics plugin with Kong Konnect. By leveraging Moesif with Kong Konnect, you can turn your API into a fully observable system. In this article, we demonstrate how Moesif can improve your API’s observability and compliment Kong Konnect to drive proactive decision-making and growth.

Monitor and Analyze APIs with Moesif 14 day free trial. No credit card required. Try for Free

An Overview of Kong Konnect API Management
Moesif API Analytics Plugin for Kong
How Moesif Analytics Plugin Works with Kong Konnect
Benefits of Using Kong and Moesif
Key Insights and Engineering Metrics
Leveraging AI Explain for API Usage Explanation
Conclusion

An Overview of Kong Konnect API Management

[Kong Konnect provides a modern cloud-native API management platform for building, managing, and deploying APIs. It works with on-premises, hybrid, or fully-cloud systems. Konnect encompasses the features of Kong Gateway by making it more accessible and streamlined to get started with from one central platform.

The most compelling aspect of Kong Gateway comes from its modularity as an API gateway, through its extensive support and ecosystem of plugins. It essentially removes any limitations in building custom integrations and extending current functionalities. You can choose from a range of native plugins and also incorporate custom plugins through Lua. Kong Konnect’s Dev Portal further allows engineers to customize, experiment, and fine-tune their API solutions.

Moesif API Analytics Plugin for Kong

The Moesif API analytics plugin for Kong Konnect brings deep API observability for Kong Gateway APIs. In a nutshell, it allows engineers to monitor traffic, analyze performance, and track user behavior in real time. Once you set it up, it doesn’t need any manual setup for instrumentation and collection, as they happen automatically and scale with your app.

The plugin captures request and response data in great depth, including the following:

Latency
Errors
Payload content
Authentication details

Unlike basic API logging tools, Moesif offers advanced filtering, anomaly detection, and product-level insights. For example, as we’ll soon demonstrate, Moesif allows you to execute granular analytics on HTTP bodies. Having all of these features at hand empowers teams to correlate API activity with specific customers or applications, performance metrics, and more. Engineers can visualize trends, set alerts for unusual behavior, and generate real-time reports to improve reliability and user experience.

The plugin integrates seamlessly with Kong Konnect through the high-performance Lua language. It works asynchronously to avoid adding latency to your app while making sure to capture comprehensive data. The plugin supports protocols such as HTTP, HTTPS, and gRPC, giving you adaptability for various API architectures.

How Moesif Analytics Plugin Works with Kong Konnect

The Moesif Analytics Plugin integrates with Kong Konnect as a custom plugin. Therefore, it doesn’t work for Dedicated Cloud Gateways in Kong.

Here are the general steps involved to set up the plugin with Kong Konnect:

Set up Kong Gateway Docker Image with Moesif Plugin

Clone the Kong Konnect Docker demo repository that contains the Kong Moesif Plugin.

 git clone --recurse-submodules https://github.com/Moesif/moesif-kong-konnect-docker-demo.git

Go inside the moesif-kong-konnect-docker-demo repository and build the image:
```
 docker build -t custom-kong-gateway:3.9.0.0 .
```

Add Moesif Plugin to a Data Plane Node

In your Konnect UI, open the Gateway Manager and then open a control plane.
Select Data Plane Nodes and then select Create a New Data Plane Node.
Select a self-managed Gateway version and choose Linux (Docker) as the platform.
Select Generate certificate.
Copy the generated docker run command and add the following snippet to it:
```
 -e "KONG_PLUGINS=bundled,moesif" \
 -e "KONG_LUA_PACKAGE_PATH=/tmp/custom_plugins/?.lua;;" \
```
Also make sure you modify the image name with the custom-kong-gateway:3.9.0.0 you built earlier.
Run the command to start a data plane node with Moesif plugin loaded in.

Add and Enable Kong Moesif Plugin to Control Plane

In your Konnect UI, open the Gateway Manager and then open your control plane.
Select Plugins from the navigation menu and then select New Plugin.
Select the Custom Plugins tab and then select Create in Custom Plugin.
Upload the schema.lua file of the Kong Moesif plugin.
Check that you can view your upload correctly in Preview and then click Save.
Navigate back to the Plugins screen, select Custom Plugins, and enable the Moesif Plugin.
Enter your Moesif Application ID in the Application Id field.
Click Save to finalize the configuration.

To get your Moesif Application ID, follow these steps:

Log into Moesif Portal.
Select the account icon to bring up the settings menu.
Select Installation or API Keys.
Copy your Moesif Application ID from the Collector Application ID field.

After you successfully set up the plugin, Moesif starts capturing requests to your Kong API. You should see them appear in Moesif’s Live Event Log workspace.

With the plugin set up, Moesif captures all the details of your API traffic, including full HTTP request-response details, custom metadata, latency, and more. Moesif can also track your Kong routes and services. They become available in event filters in the Metadata.Kong field:

Moreover, if you’re using Kong’s OpenTelemetry Plugin, you can configure Moesif to have the trace data available in Moesif. This allows you to leverage distributed tracing to further contextualize and utilize Kong Konnect and Moesif’s capabilities without configuring a separate observability backend.

Benefits of Using Kong and Moesif

By integrating Kong Konnect with Moesif, you get a powerful combination for managing and observing your APIs. This synergy takes you beyond basic monitoring, with deep insights into API performance, usage patterns, and overall operational status. Moesif and Kong provides you with the key components you need for effective API observability, including metrics monitoring, log analysis, and tracing. Kong Konnect handles the delivery of your APIs, while Moesif provides the intelligence to refine and optimize your API strategy to drive more value from your APIs.

Seamless Integration

We’ve designed the Moesif plugin for Kong Konnect for effortless integration with Kong Konnect. This means you can quickly add, set up, and deploy the plugin for your API straight from Kong Konnect’s Gateway Manager. You don’t need to perform any complex configurations or have to incur significant operational overhead.

Using the high-performance Lua language ecosystem, Moesif’s lightweight plugin ensures minimal impact on API performance while paving the way to valuable insights without sacrificing speed. Kong Gateway being a Lua application itself, Moesif’s plugin integrates natively without introducing any performance or incompatibility hiccups.

The plugin forwards crucial performance and observability data and metadata from your API to Moesif’s analytics platform. Once you set it up, you don’t need to manually instrument for data collection and dispatch, as they happen automatically in real time.

Enhanced API Monitoring and Observability

Combining Kong Konnect and Moesif gives you a holistic view of your entire API ecosystem. With Kong Konnect, you get your peace of mind for traffic management, API security, and API lifecycle management. Then Moesif adds a layer of deep API observability. This combination allows you to move beyond simple uptime checks and dig into granular details about how your APIs serve your applications. You gain visibility into not just if your APIs are working as you expect, but how they are performing through insights into your API’s performance.

Simplified API Troubleshooting

One of the most significant benefits comes from the drastically simplified troubleshooting process. When issues arise, you have access to Moesif’s detailed metrics and payload inspection and analytics capabilities. Instead of sifting through logs across different software systems, you have a centralized platform to analyze API requests, identify errors, and understand the context surrounding them. Moesif works and scales in real time with its observability and monitoring features. So you react to problems as they happen and nurture a proactive approach.

API Performance Optimization

API observability doesn’t involve just finding problems, it also promotes optimizing performance. Moesif, coupled with Kong Konnect, provides the key metrics your teams need to identify performance bottlenecks. For example, Moesif can discover that a specific database query called by a specific endpoint is causing high latency.

By analyzing latency, error rate, and request throughput, you can pinpoint areas for improvement. This might involve optimizing your backend services, adjusting Kong Konnect’s configuration—for example, caching and rate limiting, or even redesigning parts of your API. The performance trends Moesif visualizes and allows you to organize and customize them, help you make informed decisions about resource allocation and scaling.

Data-Driven Decisions

The comprehensive performance data and analytics capabilities this integration provides empower you to make data-driven decisions about your API strategy:

Understand how customers are using your APIs.
Identify the most popular endpoints.
Identify potential security vulnerabilities that might exist.

These insights can inform decisions about API design, feature development, and even pricing strategies for your API products. The ability to develop operational insight transforms your API management from reactive to proactive. Moesif lets you track key performance metrics for business and engineering, significantly reducing your mean time to resolution (MTTR) across your API lifecycles.

Key Insights and Engineering Metrics

Let’s go through some practical examples of the key insights and monitoring metrics Moesif unlocks for your Kong APIs.

Core Engineering and Key Performance Metrics

Moesif equips you with a range of crucial engineering metrics that provide a comprehensive view of your API’s health and performance. These metrics help you understand how your APIs are behaving in real-time so you can identify potential issues before they impact users. Let’s explore some of the most important ones.

Latency Analysis

Moesif has built-in performance analysis types like maximum latency, minimum latency, and P90 latency. You can also define your own custom latency analysis. For example, here we use Moesif’s Time Series analysis to perform a P99 latency analysis across API endpoints in the past 24 hours, in an hourly interval:

A simple average latency analysis, while useful, can become skewed by outliers. Therefore, performing percentile analysis like P90 or P99 gives a more robust measure than the average, as extreme values affect it less. Being able to drill down into performance this way can identify and therefore equip you to address the slowest requests. Slow requests often have the biggest impact on user experience and directly impact your product’s health. High P99 latency can indicate problems with backend services, network bottlenecks, or resource constraints.

Error Rate and Status Code Analysis

Analyzing error rates gives you a precise understanding of the percentage of API requests that result in an error, typically 4xx or 5xx HTTP status codes. If you observe a high error rate, it clearly indicates issues such as bugs in your code, issues with downstream dependencies, or insufficient resources. Moesif allows you to track the overall error rates in a number of ways and drill down into specific error types.

For example, here we break down the number of 5xx server errors on an hourly basis for the past 12 days in a time series analysis. We also categorize the analysis by response status codes so we know the exact error types.

If you notice, we’ve folded the time series to better visualize the periodicity of our metric here. This feature can be a very powerful tool to understand recurring patterns like error rate patterns and periodic anomalies. You can focus on worst-case scenarios as well. In this particular example, we can see if a particular hour exists where errors spike to especially high levels. It can help identify issues like the following:

Peak load times overwhelming the API
Scheduled maintenance or batch jobs that cause temporary disruptions
Time-dependent errors in external services that the API relies on

You can also approach this in a different way and break down all API errors in a time series:

It allows you to quickly observe the distribution of API errors. You can therefore understand the nature of your API interactions and identify potential problems. For example, if you observe a sudden increase in 401 Unauthorized errors, it might indicate a security issue. A spike in 502 Bad Gateway errors on the other hand can point to problems with a backend service.

AI Consumption Analysis for AI Apps

If you’re building an AI product using Kong Konnect, Moesif can help you track AI consumption, analyze usage, and debug AI traffic. Since Moesif captures HTTP request-response data with deep granularity, it’s very easy to track, filter on, and break down your product’s usage data by token usage, rate limits, and so on.

For example, here we do a Segmentation analysis to look at the top customers based on average input token consumption, the number of API calls, and the associated costs for the past 7 days:

Advanced Metrics and Analysis

Beyond the core metrics, Moesif supports advanced analytics like payload analysis, customer analytics, alerting, and governance rules. They bring more valuable insights and observability for your Kong Konnect APIs.

HTTP Payload Analysis

One of the most powerful features for troubleshooting is Moesif’s ability to inspect the full request and response payloads of your API calls. Analyzing HTTP bodies gives you invaluable support for debugging complex issues where the error messages alone aren’t sufficient. You can examine the exact data being sent and received, identify malformed requests, and understand the context of errors. This eliminates guesswork and significantly reduces the time required to resolve problems.

For example, you can have payloads like the following with deeply nested structures. With Moesif, you can focus anywhere and figure out what you want.

{
  "order_id": "ORD-2024-12345",
  "timestamp": "2024-11-08T15:45:30Z",
  "customer": {
    "customer_id": "CUST-7890",
    "email": "customer@example.com",
    "shipping_address": {
      "street": "123 Main St",
      "city": "Anytown",
      "state": "CA",
      "zip": "90210",
      "country": "USA",
      "delivery_instructions": {
        "leave_at_door": true,
        "signature_required": false,
        "notes": "Ring doorbell twice."
      }
    },
    "billing_address": {
      "same_as_shipping": true,
      "street": "123 Main St",
      "city": "Anytown",
      "state": "CA",
      "zip": "90210",
      "country": "USA"
    }
  },
  "items": [
    {
      "product_id": "PROD-111",
      "product_name": "Wireless Mouse",
      "quantity": 2,
      "unit_price": 24.99,
      "discount": {
        "type": "percentage",
        "value": 10,
        "reason": "Black Friday Sale"
      },
      "attributes": [
        {
          "name": "color",
          "value": "black"
        },
        {
          "name": "connectivity",
          "value": "bluetooth"
        }
      ]
    },
    {
      "product_id": "PROD-222",
      "product_name": "Keyboard",
      "quantity": 1,
      "unit_price": 79.99,
      "discount": {
        "type": "fixed",
        "value": 5,
        "reason": "Coupon Code"
      },
      "attributes": [
        {
          "name": "color",
          "value": "white"
        },
        {
          "name": "layout",
          "value": "QWERTY",
          "details": {
            "backlit": true,
            "mechanical": true,
             "switch_type": "Cherry MX Red"
          }
        }
      ]
    }
  ],
  "payment": {
    "method": "credit_card",
    "card_type": "Visa",
    "last_four": "1234",
    "transaction_id": "TRANS-XYZ",
    "status": "success",
     "gateway_response": {
        "code": "200",
        "message": "Approved",
        "auth_code": "AUTH123"
    }
  },
  "shipping": {
    "carrier": "UPS",
    "tracking_number": "1Z12345E0312345678",
    "estimated_delivery": "2024-11-12",
    "shipping_cost": 9.99
  },
    "coupons": [
      "WELCOME10",
      "FREESHIP"
    ]
}

Here are some more illustrative use cases:

Filter and analyze requests with large context size for your AI application in a Live Event Log, for example:
Use the key count operators that work on the number of keys in the object, to identify large payloads for performance troubleshooting and analysis:
Specify custom metrics to gain valuable insights into user behavior and API usage. Since Moesif makes the HTTP body data available in segmentation, filter, and group settings, engineers can easily see distributions and patterns in the data. It can uncover performance issues related to specific data values or user segments.
Debug and troubleshoot modern APIs possessing complex data models by navigating intricate JSON structures.
Get precise error context quickly by finding the exact API call that caused the problem and observe the full request-response data.

Distributed Tracing with OpenTelemetry

Moesif’s Kong integration supports extracting OpenTelemetry trace data as well if you already have an OTel integration installed.You can also use Kong Konnect’s OpenTelemetry plugin and send your trace data to Moesif’s OpenTelemetry endpoint.

How Moesif Leverages OpenTelemetry with Kong Konnect

If you have already instrumented your services with OpenTelemetry or have an OpenTelemetry Collector capable of exporting through OTLP over HTTP, Moesif can seamlessly integrate with this setup. Kong’s OpenTelemetry plugin, after configuring, can forward trace data to an OpenTelemetry Collector. Moesif, in turn, can receive this trace data from the collector, providing a unified view of your API traffic and its journey through your distributed systems.

Note that you must have existing OTel instrumentation; Moesif integrates with OpenTelemetry, it doesn’t replace the need for OpenTelemetry instrumentation within your services.

The trace data Moesif collects becomes available in Live Event Log workspaces. To view the trace data, select the Trace View tab in a Live Event Log workspace. You can expand an event for more details about it. In this detailed view, you can interact with individual event data fields and field values. For example, you can select a trace ID value and then select Open Trace, giving you more details about the trace and its constituent spans:

Furthermore, trace data elements like trace IDs, span IDs, and span statuses become available as event filters in Moesif’s API and customer analytics suites.

Leveraging AI Explain for API Usage Explanation

Despite all these analytics, metrics, and visualizations at your disposal, understanding complex API usage patterns can still pose challenges. That’s why Moesif has introduced AI-powered features to augment your API observability and monitoring needs:

Use Smart Search to navigate through analytics dashboards, workspaces, and more.
Use AI Explain in API analytics, customer analytics, and alerting to quickly uncover valuable insights.

With these features, Moesif aims to make API observability and monitoring easier, more accessible, and faster. For engineers, they provide an additional layer of intelligence for troubleshooting and insights.

Instead of manually sifting through your analysis, you can use AI Explain to generate concise summaries of API usage trends. For example, it might highlight the most significant changes in API traffic over a given period, identify the most frequently accessed endpoints, or summarize the most common error types.

For example, consider you have an embeddings API in Kong Konnect. You want to understand its growth and adoption by companies. Here’s how such an analysis might look in Moesif using a time series analysis:

You can select Ask AI and get instant insights about the analysis:

Conclusion

Modern apps have been growing in their complexities, and so do the demands from the consumers. Needless to say, therefore, utilizing observability and monitoring tools to gain a holistic understanding of your product’s components can no longer stay a secondary choice in your API strategy. With Moesif and Kong Konnect, you cover a lot of ground in that aspect while avoiding the overhead and risks of stiching together a bunch of observability tools. You can get started quickly, more easily, get to value fast, and continue to scale your product with confidence.

To see for yourself how Moesif can help you with observability-driven development, sign up for a free trial no sign up for a free trial.

Deep API Observability with Moesif 14 day free trial. No credit card required. Try for Free

API Analytics , API Monitoring , Best Practices

Abu Sakib

Technical Writer at Moesif. Previously @Arcion Labs and @Dgraph Labs.

Podcasts

From Vision to Venture Ep. 05: Mike Amundsen, Amundsen.com

Derric Gilling sits down with Mike Amundsen, a leader in the API ecosystem, to explore the role of observability in API design and customer experience.

March 07, 2025

API Analytics and Monitoring

Monetizing Proprietary Data Through APIs: How to Unlock New Revenue in the AI World

Monetizing data can bring numerous benefits to an organization. By leveraging their existing data, companies can unlock new revenue streams and gain a compet...

March 04, 2025