Ingest Custom Actions - Logstash

Moesif can directly ingest your API logs or custom actions from a Logstash instance.

What are actions?

Actions are custom usage events that you directly send to Moesif. Actions have an action name (like “Signed Up” or “Finished Job”) which represents the raw event. You can also include arbitrary metadata with an action which enables you to create billable metrics, usage reporting, and more. For more info, see docs on actions.

How it works

The integration with Logstash works by using the HTTP Output Plugin for the destination. Then your applications can insert your custom actions into the firehose which will be sent to Moesif.

Limitations

Enforcing quotas and governance rules will not work without a server integration. You can always install a server integration later if this capability is required.

How to install (actions)

Add HTTP output plugin

Add an HTTP output plugin to your Logstash configuration. The HTTP output should sent the actions to https://api.moesif.net/v1/actions/batch. Ensure you have compression and json_batch enabled as a best practice for performance.

You’ll want to define a few fields like below:

  • action_name is a string and should include name of the event such as “Processed Payment Transaction”.
  • company_id is the customer identifier (see companies).
  • transaction_id should be a random UUID for this event which Moesif uses for deduplication (docs on Moesif idempotency).
  • request.time represents the transaction time as an ISO formatted string.
  • metadata is an object which includes any custom properties for this event. By setting metadata, you can bill on arbitrary metrics, create metrics on them, etc. For example, if the action name is “Processed Payment Transaction”, you can include an amount and the currency to bill on the total amount.

For full schema and available fields, see Actions API Reference.

output {
  http {
    url => "https://api.moesif.net/v1/actions/batch"
    http_method => "post"
    http_compressioned => true    
    headers => {
      "Content-Type" => "application/json"
      "Authorization" => "Your_Moesif_Application_Id"
    }
    format => "json_batch"
    codec => "json"
    connect_timeout => 10
    request_timeout => 15
    retry_failed => true
    retry_limit => 3
    retry_interval => 2
    automatic_retries => 3
    verify_cert => true
    message => '[
      {
        "action_name": "%{[action_name]}",
        "company_id": "%{[company_id]}",
        "request": {
          "time": "%{@timestamp}", 
        },
        "transaction_id": "%{[transaction_id]}",
        "metadata": "%{[metadata]}",
      }
    ]'
  }
}

Insert an action

Now that your firehose is created, save an example action into your firehose. The firehose message must match the schema for an action which is available here.

An example action is below:

{
  "action_name": "Processed Payment Transaction",
  "request": {
    "time": "2024-03-01T04:45:42.914"
  },
  "user_id": "12345",
  "company_id": "67890",
  "transaction_id": "a3765025-46ec-45dd-bc83-b136c8d1d257",
  "metadata": {
    "amount": 24.6,
    "currency": "USD",
    "time_seconds": 66.3
  }
}

In the above example, the action is created whenever a payment is processed. There are also two metrics we are tracking as part of the action (the amount of the payment and how long the job took). You can create billable metrics and usage reports from these attributes.

Updated: