Winning in the data economy: Part 3 of 3 - Measuring, learning and keeping up with AI.
Topics covered:
More than ever the DNA of businesses is being scrutinised. In the AI era what separates success and respect from failure and obscurity? What are the indicators that your brand is going to succeed in their competitive market place? How do you stack the cards and keep on stacking them in your favour to give you an advantage? And, at a time where it’s hard to know what’s real and what’s fake how do you know that you’re even measuring the real indicators of success?
We are in the AI era, to be a winner in this era one thing is as certain as taxes, and that is that AI needs to be part of your business, or these kinds of challenges will be impossible to solve.
An AI first business has a different DNA from traditional and digital businesses. It’s not just a business that has an AI strategy, it’s a business that has prioritised AI as a core component of everything: it’s workflow and processes, its services and its products.
The DNA of successful AI first businesses also have another common trait and that is that they know the compound business case of AI. They are deploying AI for cost saving and value creation, for efficiency and quality, for profit and sustainability. In other words, they have a business plan and they are clear throughout that business plan how AI is going to enable their business across short term and longer term horizons to support all its operational metrics.
To make the point for a telco this means you have a business plan that describes proposition, product, distribution, pricing, marketing, the service experience, wholesale commercials etc but interlaced into each dimension is where AI sits – there isn’t a section called AI in your plan it’s throughout your plan.
Your competitive advantage depends on your leverage of AI in creative and commercial ways aligned to your brand.
So how do you know what good looks like?
When we deploy predictive models we use a variety of metrics: Accuracy (we use scores like AUC – Area under Curve), Precision and Recall. These measures are used to monitor the accuracy of a model and they should be part of your vocabulary if you work in BI or Data Science but they are a technical evaluation of a models performance.
To measure the impact of AI you need a critical eye on your business performance metrics, your team metrics and your financial metrics. That’s because once you have implemented predictive models the real-world impact of the model is the single most important success criteria. It no longer matters what the accuracy score is; it matters that it is helping you to save customers or increase revenue or similar.
We strongly advise rethinking KPIs when you’re moving to an AI first approach. The reason for that is that AI should bring efficiency improvements, quality improvements and greater insights which need more dynamic KPIs, more agile delivery and more alignment across teams.
When we observe the challenge that Opcos have had moving from legacy operating models to AI first operating models it’s very obvious that many of their problems come back to measurement and accountability. One of the classic examples in Opcos is the vanity metric of subscriber base which can be a primary churn driver when the quality of the acquisition is poor. MVNOs, being agile and more focused should have a natural advantage!
As a telco we’d recommend moving to dynamic KPIs that express desired changes and better measure outcomes over time. The more the KPI creates collaboration across business units and breaks organisational silos the better – without this you’ll end up with ‘data hoarders’ who aren’t acting as great corporate citizens for your business.
|
From this KPI |
To this KPI |
|
Subscribers |
Active base |
|
Tenure |
CLV |
|
Revenue |
Margin (per product) |
|
etc |
etc |
Experimentation goes hand in hand with data science. There are several reasons for this:
Experimentation is so important that major brands that are AI first (Netflix is a great example – interesting article here) they treat their entire product experience and marketing offer catalogue as a continuous stream of experiments. They continue to run experiments and promote new versions of models, UX etc based on an experiment out-performing a status quo version.
The other important aspect of experimentation is that if you are breathing data into everything you do then there is no better way to get more useful data than to try new / more things. Experimentation is not a nice to have; it’s a necessity. And, I’d go as far as saying ‘if you aren’t experimenting then you’re probably not data-lead’ and you are certainly not setting yourself up for success in the AI Era.
As leaders, it is drilled into us that we should have answers. We should know best because we’ve worked up the ranks to become a leader. But that’s not how AI works; you have to be ok not to understand the algorithm; ok that you can’t describe how it works or how it was built; and you have to happy not to have the answer. The DNA of AI first businesses that are incredibly successful (putting the tech behemoths like Musk, Bezos and the Zuck aside) is that their leaders are embracing ‘learning the answer’ not knowing it. That’s the movement from gut (‘I know best’) to data lead decision making that will signal you’re in a data lead business.
Making this change means creating cultural change in your business:
There are fundamental things that separate the winners from the losers in the data economy. The winners have found ways to use data as a differentiator – even a revenue stream. They have foundational capability around telco data platforms and nurture their data. They leverage their data comprehensively across their product and value chain so that their investment delivers them compound benefit and they continue to invest and reinvest. Importantly, those winning in the AI era have one other commonality – they have created the right culture and operational model coalescing around performance metrics that allows them to turn data into an advantage.