The importance of measuring

Unpicking the right things to measure can be a tricky knot to untangle

Joshua

2 minute read

Twisted ropes

This is a blog where I’m going to try and write about building good products and services that people love to use, enjoy and value.

What makes a product good? Well, it depends on the utility that the product is trying to fulfil. A product aimed at helping office workers track their time spent on different projects is going to have very different success metrics to a product such as a mobile game.

Data –> Product

Digital products are blessed with the ability to generate massive amounts of data about their usage and their users. And with the proper infrastructure in place, this data can be made available in real time for analysis and testing. This kind of data is called telemetry - data collected whilst something is being used or run for the purpose of analysis.

Telemetry is an automated communications process by which measurements and other data are collected at remote or inaccessible points and transmitted to receiving equipment for monitoring. Wikipedia

It’s from this analysis that we can learn about changes to the product - are they being used? Are they successful? Are users gaining value? By asking the right questions and knowing what to measure we can learn faster and better about the products we manage.

Feedback cycle

The feedback cycle when managing a product is key. How fast it needs to be varies based on the nature of the product. For a faster feedback cycle, you need three things:

  • Users to use the product
  • The ability to quickly deploy changes
  • The ability to monitor the outcome of changes

For some digital services, say a popular e-commerce site like Amazon.com, deployments happen hourly and so the feedback cycle can be very rapid with millions of users at their disposal to interact with any change they make. Of course, we don’t always need to ship code to test an idea. A key principle in lean startup is to fail fast. Test any ideas or hypothesis as quickly as possible with as little effort as possible. If that means you test changes with mockups, conversations or demonstrations, then great!

Driving success through product data

All roads lead to a simple premise though - we have an idea that will make our product more successful, and we want to know if we have been successful delivering value. Without measuring, we won’t know if we’re adding value. And without adding value, the long term prospects for the product aren’t great.



comments powered by Disqus