How to Avoid Vanity API Metrics and Best Measure Developer Success for Developer Platforms
In years of advising and consulting for many companies, from seed-stage startups to unicorns and public entities, I’ve observed many teams and businesses missing the mark when defining success metrics.
Interestingly, this is more likely the case for developer platforms. I often hear “Look at how many API keys/hours spent in development/{Marklar} were created on our platform!” As I’d respond by asking “How many active developers do you have?” or “What is the adoption rate for your partner applications?” I’d strike a nerve.
In API world, vanity metrics can be slipped in just like in any other product area. I’ve lost count of how many times companies boast to me about how many API calls are made on their system, considering that as their primary metric - and as I note inefficiencies in their API designs that inflate the metric. Often, when my client projects are successful, the number of API calls is reduced, as we design APIs that make integrations easier on our partners and on our servers. Treating your API as a product can allow for alignment around value metrics, helping increase deployment frequency. This, in turn, can improve the development process of your API product. By valuing what your test metrics uncover, finding the right quality metrics for your API product becomes much easier.
Such errors of judgement sometimes come from high-level business KPIs that don’t align to company goals.
Others come from proper company goals, that when making their way to platform initiatives, lose alignment.
It could be that the company is, rightly, focused on growth or retention, but then executives think that
can be supported by “more partners!” In such cases, when managing a developer platform, one must be ready
to be critical, to say “no,” and to keep focus on the health of your customer and developer community.
On more than one occasion, I was asked to build out a partner ecosystem for an application marketplace, with a designed number of applications to publish. “Sure,” I’d say. “I know a partner who’s creating an consumer-facing, not relevant to revenue integration, so I’ll bring in plenty of partners like that we can find easily. Or… let’s adjust our software development metrics”.
Then, I’d make sure we’re actually setting value according to company goals. How?
- Understand and restate the company’s goals.
- Understand what value points and feature gaps are supported by partners.
- Set KPIs that focus on obtaining those partners and making them successful.
When asked or tempted to bring in X-number of partner integrations, it’s best to adjust the plan to ensure that you’re only including partners that actually generate value rather than code churn. Is the goal of the platform to acquire new customers? Then only count partners who are proven channel partners that bring us new customers? Are you trying to drive upsells? Then only count partners that generated clear revenue for us. Is the goal to improve retention? Then only count partners who supported existing customers, and who our Post-Sales team can confirm to have supported license renewal.
Vanity metrics are easy to create, and we all understand that they have negative consequences. Yet we easily stumble upon vanity metrics, perhaps more with APIs than UI-related metrics. Only by applying data and critically assessing your platform’s priorities and your company’s goals, can you ensure growth for your platform in the right path with product metrics that improve developer productivity by being relevant and helpful to improving software quality. Tools like Moesif API Analytics can help you get started measuring the right API DORA metrics with just a quick SDK installation.
Accurately Track All API Product Metrics With Moesif
14 day free trial. No credit card required.
Learn More
Related Articles
API Analytics and Monitoring
Introducing AI Explain
We are thrilled to introduce our newest feature to Moesif - AI Explain - a powerful conversational tool that lets you interact with your API analytics data l...
API Analytics and Monitoring
Monitoring Cost and Consumption of AI APIs and Apps
Discover how Moesif can help manage and optimize your AI-driven APIs with powerful observability tools that track costs and usage efficiently.
API Development
Unlock Deep API Insights with Moesif’s HTTP Body Analytics
Learn how to use Moesif to analyze HTTP bodies.
API Analytics and Monitoring
How Moesif API Observability Compares to Treblle
Learn how Moesif’s API observability platform compares to Treblle.