Measuring and Testing Innovation  | May 2019

At a recent DHC Summit, a panel of four leading experts in healthcare innovation discussed the rather simple question of “Measuring and Testing Innovation.” Led by moderator Joe Shields (President, Health Accelerators), the panel’s true focus was to engage DHC members to get real and accept that over the past decade innovation has grown up. “Pharma spending on innovation must be as accountable as any other investment. And there are opportunity costs to the business, since any new tactic takes money away from something else, such as building a new manufacturing facility or hiring more people.”
 
Given the higher stakes in today’s climate, this panel set out to show that innovation must be measured to some degree, and how to “compare investment in the new with the tried and true.” What follows is a summary of their conversation, with analysis for DHC members to apply the significant insights shared. 

Participating Panelists:

Joe Shields, President, Health Accelerators

John Walsh, Senior Director, IT, Teva

David Guthrie, Chief Product Officer, PatientPoint

Matt Walsh, Industry Solutions, Crossix

CONCEPT 1: ENGAGEMENT IS A PIECE OF THE PUZZLE

In traditional marketing measurement, engagement means that someone is simply interacting with a specific marketing piece. In the context of commercial innovation that may be oneindicator of success, but it’s not necessarily theindicator of success. That type of engagement still matters, but it’s also important to know if you are reaching the right people across their journey, ultimately influencing decisions and actions over time. In other words, actions like visiting a doctor, writing a prescription, or improving a patient’s adherence are usually not triggered by a simple “engagement” using a message delivered by media. It’s possible to test broader engagement and action attribution in the digital world, and marketers and their partners often use dynamic content, A/B testing and other techniques to optimize the effect of these tactics working together to change behavior.

DAVID GUTHRIE: “It’s important to think about both what engages the provider from a tool perspective and an education perspective as well as what engages the patient and provides them the information they need. Ask a question like: ‘Are we delivering the education and information that the doctor and patient need in order to help them make decisions?’”

MATT WALSH: “In a more traditional model of engagement measurement, there are large groups of people that can be influenced by media that aren’t strictly ‘engaging.’ And if you’re not also innovating your measurement methods, you’re potentially undercounting the benefit you see.”

CONCEPT 2: MEASURING INNOVATION, CONSISTENTLY

The panelists agreed that the key metrics continue to answer these questions:

• Are you getting in front of the right audience from a quality perspective?
• Are you driving action post-exposure or post-interaction with media?

It’s important to remember that business objectives and related metrics shouldn’t change just because new tactics may have different ways of collecting and looking at the data. There’s a a seemingly endless stream of new media, platforms, technologies and tactics coming into the market each month, yet we still must be able to understand the basics of who is interacting with them and what they do next. At the end of the day, it’s best to keep measurement consistent so that an ‘apples-to-apples’ comparison can be achieved across different channels, technologies and tactics.

In the case of early-phase or small trial innovation, “stage-gate” is a solid approach, enabling teams to stop underperforming projects in order to reallocate funding to more successful ones. Being able to define at the outset what success looks like — and then consistently enforcing accountability — is much more powerful than the approach of, “This is new, cool, and shiny. I want one of those. Let’s do it.”

MATT WALSH: “It’s important that people start new innovative tactics and clearly say, ‘This is how we’re actually going to measure it. This is how we’re all going to agree whether or not this was successful, to get stakeholder alignment, and we can all say yes, this worked, or no, this didn’t work.’”

In order to have stakeholder alignment and validate measurement methodologies for a new tactic, it’s also important to be on the same page as the analytics experts in your agency or client’s company. Success with this comes from very tight collaboration – taking some of the best practices learned by the analytics groups over time and marrying them with new techniques and new data from the other collaborators in the partnership.

CONCEPT 3: FIRST, CLEARLY DEFINE THE PROBLEM:

In order to understand how to shape new approaches to measure innovation, marketers must start in the right place. As partnerships are built and KPIs  determined, it’s crucial to understand how to design the appropriate approach to consistent measurement.

JOHN WALSH: “I usually start with, ‘What problem are we trying to solve?’ If the first thing we do is properly define the problem, we’re usually going to have a greater opportunity for success, clarity around what that means, what we can learn from it.”

Next step, identify what part of the problem this new innovation is going to solve, while not forgetting the old adage, “Aim small, miss small.” Be honest about what’s unknown and the assumptions. At the same time, smart marketers sell the vision internally by building a passion and understanding for the mission-critical nature of the new project or tactic. Internal allies are crucial to help you overcome hurdles later on.

BONUS…HOW TO SCALE:

The key to scale, according to John Walsh, is to, “Have clarity around what you’re trying to achieve, what the long story is, what the potential contribution is, and then what part you intend to solve.” To get to scale, strong marketers must have the boldness to invest, and gauge along the way whether management really supports failure as a form of learning. With the right framework and a bit of trust, the right preparation will prove out the elements, giving management a reason to believe. In all of this, be bold.