Moesif OpenTelemetry Integration

Integrate Moesif API Analytics with your OpenTelemetry-instrumented applications to gain deep insights into your API traffic and performance. This integration allows you to send trace and log data from your OpenTelemetry setup directly to Moesif without any code changes or redeployments.

How it works

The integration works by configuring your OpenTelemetry exporter to send trace and log data to Moesif using the OTLP/HTTP protocol. Moesif treats each OpenTelemetry HTTP or Generative AI span as an API event, capturing key information like request and response details, user identification, and metadata. OpenTelemetry logs are ingested as structured log events that can be correlated with traces.

By integrating with OpenTelemetry, you can leverage existing distributed tracing and logging to analyze API performance and user behavior across services and applications.

How to Install

Prerequisites

  • An application instrumented with OpenTelemetry SDKs or an OpenTelemetry Collector which can export OTLP/HTTP for traces and logs.
  • A Moesif account.

Steps

  1. Configure the OpenTelemetry Exporter

    Set up your OpenTelemetry SDK to export trace and log data to Moesif using the OTLP/HTTP protocol.

    Example Configuration for Traces (YAML):

    exporters:
  otlphttp:
    endpoint: https://api.moesif.net/v1/traces
    headers:
      X-Moesif-Application-Id: 'Your Moesif Application ID'

    Example Configuration for Logs (YAML):

    exporters:
  otlphttp/logs:
    endpoint: https://api.moesif.net/v1/logs
    headers:
      X-Moesif-Application-Id: 'Your Moesif Application ID'

    Example Configuration (Environment Variables):

    # Set the OTLP endpoints
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.moesif.net/v1/traces
export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=https://api.moesif.net/v1/logs

# Include your Moesif Application ID in the headers
export OTEL_EXPORTER_OTLP_HEADERS=X-Moesif-Application-Id='Your Moesif Application ID'

    Replace 'Your Moesif Application ID' with your actual Moesif Application ID. To get your Application ID:

    1. Log into Moesif Portal.
    2. Select the account icon to bring up the settings menu.
    3. Select Installation or API Keys.
    4. Copy your Moesif Application ID from the Collector Application ID field.

  2. Set Up Authentication

    Ensure your Moesif Application ID is included in the request headers as X-Moesif-Application-Id to authenticate requests.

  3. Run Your Application

    Start your application, and OpenTelemetry will begin sending trace and log data to Moesif.

Mapping OpenTelemetry Data to Moesif

Moesif ingests two OpenTelemetry signals today: spans (HTTP and Generative AI) and structured log records. Mapping details for each signal are below.

Spans

Moesif supports capturing span data which follows the OpenTelemetry Semantic Conventions. Two span types are mapped into first-class Moesif fields:

Moesif maps OpenTelemetry spans to its API event model. Each span is treated as a single API event in Moesif with the addition of Trace ID, Span ID, Parent Span, Span Links, and metadata.

The HTTP/GenAI semantic conventions for OpenTelemetry are recently stable. We do our best to be backward compatible with prior tracing implementations as well. To understand exactly how OpenTelemetry spans are mapped to Moesif API events, refer to the tables below. The fields in the following table are common to all spans regardless of type.

Span to Moesif API Event Mapping

OpenTelemetry Span Field Moesif Field Notes
trace_id trace_id Used to group spans into a single trace.
span_id span.id Unique identifier for the span.
parent_span_id span.parent_id Identifier of the parent span, if any.
name action_name Name of the span or the API action.
kind direction Mapped to incoming or outgoing based on span kind.
start_time_unix_nano request.time Start time of the request.
end_time_unix_nano response.time End time of the response.
attributes Various fields (see below) Attributes are mapped to Moesif fields and metadata.
status span.status Status of the span.

HTTP Span Attributes Mapping

Under the OpenTelemetry HTTP conventions, incoming (SERVER) and outgoing (CLIENT) spans represent the request URI differently. OpenTelemetry defines these as two separate span types; Moesif builds the URI accordingly. All other attributes map the same way regardless of span kind. Both current (stable) and legacy attribute names are accepted.

Incoming spans (SERVER and INTERNAL span kinds): request URI

Span Attribute Key Moesif Field Notes
url.scheme (or http.scheme) request.uri URI scheme (http/https).
server.address (or http.host) request.uri Server host.
url.path (or http.target) request.uri URI path. Combined with scheme and host to build the full URI.

Outgoing spans (CLIENT span kind): request URI

Span Attribute Key Moesif Field Notes
url.full (or url.original, http.url) request.uri Absolute URL that includes scheme, host, path, and query in a single field.

Common to both span kinds

Span Attribute Key Moesif Field Notes
http.request.method (or http.method) request.verb HTTP method (GET, POST, etc.).
http.response.status_code (or http.status_code) response.status HTTP response status code.
client.address (or http.client_ip, net.peer.ip) request.ip_address Client IP address.
user_agent.original (or http.user_agent) request.user_agent_string User agent string.
http.request.header.<key> request.headers.<key> Request headers.
http.response.header.<key> response.headers.<key> Response headers.
http.request.body request.body Request body string/bytes.
http.response.body response.body Response body string/bytes.
user.id (or enduser.id, user.hash) user_id ID for the user.
company.id company_id ID for the company.
subscription.id subscription_id ID for the subscription.
weight weight Weight for sampling.
Resource resource Resource attributes.
Scope scope Scope attributes.

Generative AI Span Attributes Mapping

Moesif maps gen_ai.* attributes into first-class ai_context fields and uses input and output messages as the request and response bodies. For descriptions of the ai_context fields, see AI Context Filters.

Span Attribute Key Moesif Field Notes
gen_ai.operation.name request.verb Uppercased (e.g., chat becomes CHAT).
gen_ai.provider.name (or gen_ai.system) ai_context.provider Provider identifier (e.g., openai, anthropic, aws.bedrock).
gen_ai.request.model ai_context.model Model name (e.g., gpt-4, claude-3-opus).
gen_ai.usage.input_tokens ai_context.usage.prompt_tokens Input/prompt token count.
gen_ai.usage.output_tokens ai_context.usage.completion_tokens Output/completion token count.
gen_ai.usage.cache_read.input_tokens ai_context.usage.cache_read_input_tokens Cached input tokens read.
gen_ai.usage.cache_write.input_tokens (or cache_creation.input_tokens) ai_context.usage.cache_creation_input_tokens Cached input tokens written/created.
gen_ai.usage.reasoning.output_tokens ai_context.usage.reasoning_tokens Reasoning token count.
gen_ai.response.finish_reasons ai_context.finish_reason First entry used.
gen_ai.tool.definitions ai_context.tool_definitions Tool schemas the model had access to.
gen_ai.input.messages request.body Input/prompt messages sent to the model.
gen_ai.tool.call.arguments request.body Arguments passed to a tool call (execute_tool spans).
gen_ai.output.messages response.body Output/response messages returned by the model.
gen_ai.tool.call.result response.body Result returned from a tool call (execute_tool spans).
Span status response.status Span status mapped to a response status code.
client.address (or net.peer.ip) request.ip_address Client IP address.
user_agent.original request.user_agent_string User agent string.
user.id user_id ID for the user.
company.id company_id ID for the company.
subscription.id subscription_id ID for the subscription.
weight weight Weight for sampling.
Resource resource Resource attributes.
Scope scope Scope attributes.

Behavior notes

  • If both HTTP and Generative AI attributes are on the same span (e.g., an instrumented OpenAI SDK call), the HTTP attributes take precedence for request.verb, request.uri, and response.status. However, ai_context is still populated from the Generative AI attributes.
  • For pure Generative AI spans (no HTTP attributes), request.uri is set to / automatically.
  • Any Generative AI attribute not covered in the preceding table is preserved in the event metadata field for querying.

Logs

Moesif also supports OpenTelemetry log records. Each log record is ingested as a structured log event that can be correlated with traces.

Log Record Field Moesif Field Notes
timeUnixNano request.time Log timestamp.
severityNumber log.severity.number Numeric severity level (1-24).
severityText log.severity.text Text severity (INFO, ERROR, etc.).
body request.body Log message content.
traceId trace_id Links logs to traces.
spanId span.id Links logs to spans.
attributes.user.id user_id Associates log with user.
attributes.company.id company_id Associates log with company.
EventName action_name Associates log with event name.
Resource resource Resource attributes.
Scope scope Scope attributes.

Common Attributes & Metadata

Any span or log attributes that do not have a dedicated mapping are stored in the metadata field in Moesif.

User, Company & Subscription Tracking

These IDs are determined by your custom data and are translated into the corresponding Moesif event fields for entity tracking. Associate API calls with entities by adding the relevant attribute (user.id, company.id, or subscription.id) to the span or log.

Span.current().setAttribute("user.id", "user_123");

There are additional user and company tracking options available in Moesif. For more information, see Identifying Users.

Sampling & Weight Attribute

If you are using OpenTelemetry’s sampling, you can set the weight attribute to an integer value that is the inverse of the sample rate to ensure Moesif metrics and counts reflect the correct scale. For example, if you are sampling 50 % of requests, set weight = int(1.0 / 0.5) = 2; use 3 for 33 % sampling, and so on.

double sampleRate = 0.5;
int weight = (int) (1.0 / sampleRate);
Span.current().setAttribute("weight", weight);

See also

Span-specific Enhancements

Body Capturing

Moesif supports capturing request and response bodies from OpenTelemetry attributes. The bodies must be provided as raw strings or byte arrays.

To include request and response bodies:

  • Set the http.request.body attribute on the span to the raw request body content.
  • Set the http.response.body attribute on the span to the raw response body content.

Example (in Java):

Span.current().setAttribute("http.request.body", requestBodyStringOrBytes);
Span.current().setAttribute("http.response.body", responseBodyStringOrBytes);

Note:

  • Only primitive string or byte values are supported. If necessary, ensure you convert any structured object to a string or byte array before setting these attributes.

Span Action Events

In addition to capturing main request/response spans, Moesif now supports capturing OpenTelemetry span events as separate span_action events. Span events are annotations or logs added to a span that provide deeper insights into what occurred during the operation represented by the span.

How It Works:

  • Each event recorded on a span in OpenTelemetry is ingested into Moesif as a separate event of type span_action.
  • These span_action events:
    • Contain the trace_id and span_id linking them to the main API event (the span).
    • Include a request.time corresponding to the event’s timestamp.
    • Store any span event attributes as metadata for detailed analysis.
  • Unlike the primary api_call events, span_action events do not represent a full HTTP request/response. Instead, they provide supplemental details (e.g., internal logs, checkpoints, or custom annotations).

Example (in Java):

Span.current().addEvent("Database Query Start", Attributes.of(
  stringKey("db.query"), "SELECT * FROM users",
  stringKey("db.timing.notes"), "Cache miss"
));

Moesif will ingest this as a separate span_action event linked to the original trace and span. The key difference between a Span Event and an additional child Span is that the Span Event does not have a start_time or end_time and is not a full HTTP request/response. They represent a named action with a single time point and potentially distinct metadata within the Span itself. OpenTelemetry Span Events

Example Integrations

If you are using an OpenTelemetry Collector already, the integration will be as simple as including the exporter configuration above.

If you are instrumenting an application from scratch for OpenTelemetry tracing, we have an example Node JS application that demonstrates how to set up OpenTelemetry with Moesif. The example application is available at

Node JS Tracing Example on Github

Limitations

There are certain OpenTelemetry specific limitations to be aware of:

Traces, logs, HTTP spans, and generative AI spans
Traces and Logs are supported, but not OpenTelemetry Metrics. Within Traces, HTTP Spans and Generative AI Spans are mapped into first-class Moesif fields. Other span types are ingested, but their attributes are only stored in metadata. See OpenTelemetry Semantic Conventions for HTTP Spans and OpenTelemetry Semantic Conventions for Gen AI Spans.
Governance rules are not supported
Moesif’s governance features, such as blocking or modifying API calls, are not supported through the OpenTelemetry integration.
Dynamic sampling
Moesif’s dynamic sampling is not applied to data ingested via OpenTelemetry. Sampling must be configured manually within your OpenTelemetry tooling. If you do so, set the weight attribute as described above. See the OpenTelemetry Sampling Documentation

Troubleshooting

Data not appearing in Moesif
Ensure your OpenTelemetry exporter is correctly configured with the Moesif endpoint and Application ID and that your traces include HTTP or Generative AI spans or logs are properly formatted.
Authentication errors
Verify that the X-Moesif-Application-Id header is correctly set with your valid Moesif Application ID for both traces and logs endpoints.
Logs not processing
If logs show Successfully processed 0 log records, verify that log records contain required fields and are properly formatted according to OpenTelemetry specifications.
Incomplete data
If certain fields are missing, check that you have correctly added custom span attributes or log attributes where necessary.

Additional Resources

Updated: