Moesif Middleware for Python Django

by Moesif, the API analytics and API monetization platform.

Moesif middleware for Django automatically logs incoming and outgoing API calls and sends them to Moesif for API analytics and monitoring.

This SDK uses the Requests library and works for Python versions 2.7 up to 3.10.4.

If you’re new to Moesif, see our Getting Started resources to quickly get up and running.

Prerequisites

Before using this middleware, make sure you have the following:

Get Your Moesif Application ID

After you log into Moesif Portal, you can get your Moesif Application ID during the onboarding steps. You can always access the Application ID any time by following these steps from Moesif Portal after logging in:

Select the account icon to bring up the settings menu.
Select Installation or API Keys.
Copy your Moesif Application ID from the Collector Application ID field.

Install the Middleware

Install with pip:

pip install moesifdjango

Configure the Middleware

See the available configuration options to learn how to configure the middleware for your use case.

How to Use

In your Django settings file, add moesifdjango.middleware.moesif_middleware to the MIDDLEWARE array:

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'moesifdjango.middleware.moesif_middleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

Because of middleware execution order, we highly recommend that you add moesif_middleweare after SessionMiddleware and AuthenticationMiddleware. These Django middleware add useful session data that enables deeper error analysis. On the other hand, if you have other middleware that modifies responses before going out, you may choose to place Moesif middleware before the middleware that modifies responses. This allows Moesif to see the modifications to the response data and see closer to what goes over the wire.

Changes in Django 1.10

Django middleware style and setup was refactored in version 1.10. You need need to import the correct version of Moesif middleware depending on your Django version:

For Django 1.10 or greater, use moesifdjango.middleware.moesif_middleware.
For Django 1.9 or older, you need to follow the legacy style for importing middleware and use moesifdjango.middleware_pre19.MoesifMiddlewarePre19 instead.

To find your current Django version, you can execute the following command in your terminal:

python -c "import django; print(django.get_version())"

Django 1.10 or Newer

Django 1.10 renamed MIDDLEWARE_CLASSES to MIDDLEWARE. Therefore, for Django 1.10 or newer, you must activate your middleware by adding it to the MIDDLEWARE list:

MIDDLEWARE = [
    ...,
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'moesifdjango.middleware.moesif_middleware'
    ...
]

Then add a dictionary MOESIF_MIDDLEWARE to your Django settings file:

MOESIF_MIDDLEWARE = {
    'APPLICATION_ID': 'YOUR_MOESIF_APPLICATION_ID',
    'LOG_BODY': True,
}

Django 1.9 or Older

Activate the middleware by adding it to the MIDDLEWARE_CLASSES list in your Django settings file:

MIDDLEWARE_CLASSES = [
    ...,
    'moesifdjango.middleware_pre19.MoesifMiddlewarePre19',
    # other middlewares
]

Then add a dictionary MOESIF_MIDDLEWARE to your Django settings file:

MOESIF_MIDDLEWARE = {
    'APPLICATION_ID': 'YOUR_MOESIF_APPLICATION_ID',
    'LOG_BODY': True,
}

The dictionary contains the configuration options for the middleware, including your Moesif Application ID that you must specify.

Optional: Capturing Outgoing API Calls

In addition to your own APIs, you can also start capturing calls out to third party services through by setting the CAPTURE_OUTGOING_REQUESTS option.

For configuration options specific to capturing outgoing API calls, see Options For Outgoing API Calls.

Troubleshoot

For a general troubleshooting guide that can help you solve common problems, see Server Troubleshooting Guide.

Other troubleshooting supports:

Solve Timezone Issue with Docker

When using Docker with Ubuntu-based image, events may not be captured if the image fails to find any timezone configuration. To solve this issue, add the following line to your Dockerfile:

ENV TZ=UTC

Otherwise, you can add RUN apt-get install tzdata in the Dockerfile.

Repository Structure

.
├── BUILDING.md
├── images/
├── LICENSE
├── MANIFEST.in
├── moesifdjango/
├── pull_request_template.md
├── README.md
├── requirements.txt
├── setup.cfg
├── setup.py
└── tests/

Configuration options

The following sections describe the available configuration options for this middleware. You have to set these options in a Python dictionary MOESIF_MIDDLEWARE. See the examples for better understanding.

`APPLICATION_ID` (Required)

Data type
String

A string that identifies your application in Moesif.

Options specific to Incoming API Calls

The following options document the configuration options to use for incoming API calls.

Several options use request and response as input arguments. These correspond to Django’s request and response objects.

`SKIP`

Data type	Parameters	Return type
Function	`(request, response)`	Boolean

Optional.

This function takes Requests Request and Response objects and returns True if you want to skip this particular event.

`IDENTIFY_USER`

Data type	Parameters	Return type
Function	`(request, response)`	String

Optional, but highly recommended.

This function takes a request and a response and returns a string that represents the user ID used by your system.

Moesif tries to identify users automatically and fetch the username from django.http.HttpRequest.user. However, if implementation differs, we highly recommend that you accurately provide a user ID using this function.

`IDENTIFY_COMPANY`

Data type	Parameters	Return type
Function	`(request, response)`	String

Optional.

A function that takes a request and a response and returns a string that represents the company ID for this event.

`GET_SESSION_TOKEN`

Data type	Parameters	Return type
Function	`(request, response)`	String

Optional.

A function that takes a request and a response, and returns a string that represents the session token for this event.

Similar to user IDs, Moesif tries to get the session token automatically. However, if you setup differs from the standard, this function can help tying up events together and help you replay the events.

`GET_METADATA`

Data type	Parameters	Return type
Function	`(request, response)`	Dictionary

Optional.

A function that takes a request and a response and returns a Python dictionary.

The dictionary must be such that it can be converted into valid JSON. This allows you to associate this event with custom metadata. For example, you may want to save a virtual machine instance ID, a trace ID, or a tenant ID with the request.

`LOG_BODY`

Data type	Default
`Boolean`	`True`

Optional.

Whether to log request and response body to Moesif.

Set to False to remove the HTTP body before sending to Moesif.

If you want more control over which fields the middleware includes or not, see the next option MASK_EVENT_MODEL.

`MASK_EVENT_MODEL`

Data type	Parameters	Return type
Function	`(EventModel)`	`EventModel`

Optional.

A function that takes the final Moesif event model and returns an EventModel object with desired data removed.

Use this if you prefer to write your own mask function than uses the string based filter options:

REQUEST_BODY_MASKS
REQUEST_HEADER_MASKS
RESPONSE_BODY_MASKS
RESPONSE_HEADER_MASKS

The return value must be a valid EventModel required by Moesif data ingestion API. For more information about Moesif event model, see Moesif Python API documentation.

`BATCH_SIZE`

Data type	Default
`int`	`25`

Optional.

Specifies the maximum batch size when sending to Moesif.

`EVENT_QUEUE_SIZE`

Data type	Default
`int`	`1000_000`

Optional.

The maximum number of events to hold in queue before sending to Moesif.

In case of network issues, the middleware may fail to connect or send event to Moesif. In those cases, the middleware skips adding new to event to queue to prevent memory overflow.

`AUTHORIZATION_HEADER_NAME`

Data type	Default
String	`authorization`

Optional.

A request header field name used to identify the User in Moesif. It also supports a comma separated string. Moesif checks headers in order like "X-Api-Key,Authorization".

`AUTHORIZATION_USER_ID_FIELD`

Data type	Default
String	`sub`

Optional.

A field name used to parse the user from authorization header in Moesif.

`BASE_URI`

Data type
String

Optional.

A local proxy hostname when sending traffic through secure proxy. Remember to set this field when using secure proxy. For more information, see Secure Proxy documentation..

Options For Outgoing API Calls

The following options apply to outgoing API calls. These are calls you initiate using the Python Requests library to third parties like Stripe or to your own services.

Several options use request and response as input arguments. These correspond to the Requests library’s request or response objects.

If you are not using Django, you can import moesifpythonrequest directly.

`CAPTURE_OUTGOING_REQUESTS`

Data type	Default
Boolean	`False`

Set to True to capture all outgoing API calls.

`GET_METADATA_OUTGOING`

Data type	Parameters	Return type
Function	`(req, res)`	Dictionary

Optional.

A function that enables you to return custom metadata associated with the logged API calls.

Takes in the Requests request and response objects as arguments.

We recommend that you implement a function that returns a dictionary containing your custom metadata. The dictionary must be a valid one that can be encoded into JSON. For example, you may want to save a virtual machine instance ID, a trace ID, or a resource ID with the request.

`SKIP_OUTGOING`

Data type	Parameters	Return type
Function	`(req, res)`	Boolean

Optional.

A function that takes a Requests request and response objects, and returns True if you want to skip this particular event.

`IDENTIFY_USER_OUTGOING`

Data type	Parameters	Return type
Function	`(req, res)`	String

Optional, but highly recommended.

A function that takes Requests request and response objects, and returns a string that represents the user ID used by your system.

While Moesif tries to identify users automatically, different frameworks and your implementation might vary. So we highly recommend that you accurately provide a user ID using this function.

`IDENTIFY_COMPANY_OUTGOING`

Data type	Parameters	Return type
Function	`(req, res)`	String

Optional.

A function that takes Requests request and response objects and returns a string that represents the company ID for this event.

`GET_SESSION_TOKEN_OUTGOING`

Data type	Parameters	Return type
Function	`(req, res)`	String

Optional.

A function that takes Requests request and response objects, and returns a string that corresponds to the session token for this event.

General options

`LOG_BODY_OUTGOING`

Data type	Default
Boolean	`True`

Optional.

Set to False to remove logging request and response body.

`LOCAL_DEBUG`

Data type
Boolean

Optional.

Set to True to print internal log messages for debugging SDK integration issues.

Examples

See the Moesif Django Example for a complete Django application that uses this middleware.

The following shows a sample Django settings file with different configuration options.

def identify_user(req, res):
    # Your custom code that returns a user id string
    if req.user and req.user.is_authenticated:
        return req.user.username
    else:
        return None

def identify_company(req, res):
    # Your custom code that returns a company id string
    return "67890"

def should_skip(req, res):
    # Your custom code that returns true to skip logging
    return "health/probe" in req.path

def get_token(req, res):
    # If you don't want to use the standard Django session token,
    # add your custom code that returns a string for session/API token
    return "XXXXXXXXXXXXXX"

def mask_event(eventmodel):
    # Your custom code to change or remove any sensitive fields
    if 'password' in eventmodel.response.body:
        eventmodel.response.body['password'] = None
    return eventmodel

def get_metadata(req, res):
    return {
        'datacenter': 'westus',
        'deployment_version': 'v1.2.3',
    }


MOESIF_MIDDLEWARE = {
    'APPLICATION_ID': 'Your application id',
    'LOCAL_DEBUG': False,
    'LOG_BODY': True,
    'IDENTIFY_USER': identify_user,
    'IDENTIFY_COMPANY': identify_company,
    'GET_SESSION_TOKEN': get_token,
    'SKIP': should_skip,
    'MASK_EVENT_MODEL': mask_event,
    'GET_METADATA': get_metadata,
}

The following examples demonstrate how to add and update customer information using the middleware.

Update A Single User

To create or update a user profile in Moesif, use the update_user() method.

middleware = MoesifMiddleware(None)

# Only user_id is required.
# Campaign object is optional, but useful if you want to track ROI of acquisition channels
# See https://www.moesif.com/docs/api#users for campaign schema
# metadata can be any custom object
user = {
  'user_id': '12345',
  'company_id': '67890', # If set, associate user with a company object
  'campaign': {
    'utm_source': 'google',
    'utm_medium': 'cpc',
    'utm_campaign': 'adwords',
    'utm_term': 'api+tooling',
    'utm_content': 'landing'
  },
  'metadata': {
    'email': 'john@acmeinc.com',
    'first_name': 'John',
    'last_name': 'Doe',
    'title': 'Software Engineer',
    'sales_info': {
        'stage': 'Customer',
        'lifetime_value': 24000,
        'account_owner': 'mary@contoso.com'
    },
  }
}

update_user = middleware.update_user(user)

The metadata field can contain any customer demographic or other info you want to store. Moesif only requires the user_id field. This method is a convenient helper that calls the Moesif API library.

For more information, see Moesif Python API reference.

Update Users in Batch

To update a list of users in one batch, use the update_users_batch() method.

middleware = MoesifMiddleware(None)

userA = {
  'user_id': '12345',
  'company_id': '67890', # If set, associate user with a company object
  'metadata': {
    'email': 'john@acmeinc.com',
    'first_name': 'John',
    'last_name': 'Doe',
    'title': 'Software Engineer',
    'sales_info': {
        'stage': 'Customer',
        'lifetime_value': 24000,
        'account_owner': 'mary@contoso.com'
    },
  }
}

userB = {
  'user_id': '54321',
  'company_id': '67890', # If set, associate user with a company object
  'metadata': {
    'email': 'mary@acmeinc.com',
    'first_name': 'Mary',
    'last_name': 'Jane',
    'title': 'Software Engineer',
    'sales_info': {
        'stage': 'Customer',
        'lifetime_value': 48000,
        'account_owner': 'mary@contoso.com'
    },
  }
}
update_users = middleware.update_users_batch([userA, userB])

For more information, see Moesif Python API reference.

Update A Single Company

To update a single company, use the update_company() method.

middleware = MoesifMiddleware(None)

# Only company_id is required.
# Campaign object is optional, but useful if you want to track ROI of acquisition channels
# See https://www.moesif.com/docs/api#update-a-company for campaign schema
# metadata can be any custom object
company = {
  'company_id': '67890',
  'company_domain': 'acmeinc.com', # If domain is set, Moesif will enrich your profiles with publicly available info
  'campaign': {
    'utm_source': 'google',
    'utm_medium': 'cpc',
    'utm_campaign': 'adwords',
    'utm_term': 'api+tooling',
    'utm_content': 'landing'
  },
  'metadata': {
    'org_name': 'Acme, Inc',
    'plan_name': 'Free',
    'deal_stage': 'Lead',
    'mrr': 24000,
    'demographics': {
        'alexa_ranking': 500000,
        'employee_count': 47
    },
  }
}

update_company = middleware.update_company(company)

The metadata field can contain any company demographic or other info you want to store. Moesif only requires the company_id field. This method is a convenient helper that calls the Moesif API library.

For more information, see Moesif Python API reference.

Update Companies in Batch

To update a list of companies in one batch, use the update_companies_batch() method.

middleware = MoesifMiddleware(None)

companyA = {
  'company_id': '67890',
  'company_domain': 'acmeinc.com', # If domain is set, Moesif will enrich your profiles with publicly available info
  'metadata': {
    'org_name': 'Acme, Inc',
    'plan_name': 'Free',
    'deal_stage': 'Lead',
    'mrr': 24000,
    'demographics': {
        'alexa_ranking': 500000,
        'employee_count': 47
    },
  }
}

companyB = {
  'company_id': '09876',
  'company_domain': 'contoso.com', # If domain is set, Moesif will enrich your profiles with publicly available info
  'metadata': {
    'org_name': 'Contoso, Inc',
    'plan_name': 'Free',
    'deal_stage': 'Lead',
    'mrr': 48000,
    'demographics': {
        'alexa_ranking': 500000,
        'employee_count': 53
    },
  }
}

update_companies = middleware.update_companies_batch([userA, userB])

For more information, see Moesif Python API reference.

Update Subscription

Update A Single Subscription

To create or update a subscription in Moesif, use the update_subscription method.

middleware = MoesifMiddleware(None)

# Only subscription_id is required.
# metadata can be any custom object
subscription = {
  'subscription_id': 'sub_67890',
  'company_id': '3456',
  'status': 'active'
  'metadata': {
    'plan_name': 'Pro',
    'signup_date': '2020-09-09',
    'expiration_date': '2021-09-09',
    'payment_method': 'credit_card',
    'mrr': 12000,
    'currency': 'USD'
  }
}

update_subscription = middleware.update_subscription(subscription)

The metadata field can store any subscription-related information. Moesif only requires the the following fields:

subscription_id
company_id
status

This method is a convenient helper that calls the Moesif API library.

For more information, see Moesif Python API reference.

Update Subscriptions in Batch

To update a list of subscriptions in one batch, use the update_subscriptions_batch method.

middleware = MoesifMiddleware(None)

subscriptionA = {
  'subscription_id': 'sub_67890',
  'company_id': '3456',
  'status': 'active'
  'metadata': {
    'plan_name': 'Pro',
    'signup_date': '2020-09-09',
    'expiration_date': '2021-09-09',
    'payment_method': 'credit_card',
    'mrr': 12000,
    'currency': 'USD'
  }
}

subscriptionB = {
  'subscription_id': 'sub_54321',
  'company_id': '6789',
  'status': 'active'
  'metadata': {
    'plan_name': 'Enterprise',
    'signup_date': '2020-10-01',
    'expiration_date': '2021-10-01',
    'payment_method': 'paypal',
    'mrr': 24000,
    'currency': 'USD'
  }
}

update_subscriptions = middleware.update_subscriptions_batch([subscriptionA, subscriptionB])

The metadata field can store any subscription-related information. Moesif only requires the the following fields:

subscription_id
company_id
status

This method is a convenient helper that calls the Moesif API library.

For more information, see Moesif Python API reference.

Tested Python and Django Versions

Moesif has validated this middleware against the following combinations of Python and Django versions:

Python	Django
Python 2.7	1.11.22
Python 2.7	1.11.22
Python 2.7	1.9
Python 3.4.5	1.11.22
Python 3.4.5	1.11.22
Python 3.4.5	1.9
Python 3.6.4	1.11.22
Python 3.6.4	1.11.22
Python 3.6.4	1.9
Python 3.10.4	3.2.13 LTS
Python 3.10.4	4.0.5

How to Test

Manually clone this repository.
From your terminal, navigate to the root directory of the middleware.
Run pip install Django and then run pip install moesifdjango.
Add your Moesif Application ID to tests/settings.py.
From terminal, navigate to the root directory of the middleware tests tests/.

a. Run python manage.py test if you are using Django 1.10 or newer.

b. Run python manage.py test middleware_pre19_tests if you are using Django 1.9 or older.

Troubleshoot

On MacOS, with gunicorn, it is possible to encounter this error: “in progress in another thread when fork() was called”. Solution is to add --preload option when launching gunicorn like below.

gunicorn myapp.wsgi:application --bind 127.0.0.1:8000 --preload

Explore Other Integrations

Explore other integration options from Moesif:

Moesif Middleware for Python Django

Prerequisites

Get Your Moesif Application ID

Install the Middleware

Configure the Middleware

How to Use

Changes in Django 1.10

Django 1.10 or Newer

Django 1.9 or Older

Optional: Capturing Outgoing API Calls

Troubleshoot

Solve Timezone Issue with Docker

Repository Structure

Configuration options

APPLICATION_ID (Required)

Options specific to Incoming API Calls

SKIP

IDENTIFY_USER

IDENTIFY_COMPANY

GET_SESSION_TOKEN

GET_METADATA

LOG_BODY

MASK_EVENT_MODEL

BATCH_SIZE

EVENT_QUEUE_SIZE

AUTHORIZATION_HEADER_NAME

AUTHORIZATION_USER_ID_FIELD

BASE_URI

Options For Outgoing API Calls

CAPTURE_OUTGOING_REQUESTS

GET_METADATA_OUTGOING

SKIP_OUTGOING

IDENTIFY_USER_OUTGOING

IDENTIFY_COMPANY_OUTGOING

GET_SESSION_TOKEN_OUTGOING

General options

LOG_BODY_OUTGOING

LOCAL_DEBUG

Examples

Update A Single User

Update Users in Batch

Update A Single Company

Update Companies in Batch

Update Subscription

Update A Single Subscription

Update Subscriptions in Batch

Tested Python and Django Versions

How to Test

Troubleshoot

Explore Other Integrations

`APPLICATION_ID` (Required)

`SKIP`

`IDENTIFY_USER`

`IDENTIFY_COMPANY`

`GET_SESSION_TOKEN`

`GET_METADATA`

`LOG_BODY`

`MASK_EVENT_MODEL`

`BATCH_SIZE`

`EVENT_QUEUE_SIZE`

`AUTHORIZATION_HEADER_NAME`

`AUTHORIZATION_USER_ID_FIELD`

`BASE_URI`

`CAPTURE_OUTGOING_REQUESTS`

`GET_METADATA_OUTGOING`

`SKIP_OUTGOING`

`IDENTIFY_USER_OUTGOING`

`IDENTIFY_COMPANY_OUTGOING`

`GET_SESSION_TOKEN_OUTGOING`

`LOG_BODY_OUTGOING`

`LOCAL_DEBUG`