A recent behavioural study conducted by the Nest Pensions Insights team provides a reminder of one of the bigger pitfalls of consumer research: the gulf between what people say they will do and what they actually do.

The study looked at how framing of communications can nudge people with respect to opting out of their workplace pension (the Nest goal being to reduce the opt-out rate):

• Nest developed a series of different messages, constructed around well-known behavioural theories such as present bias and social norms.
• The messages were tested via an online survey approach, using stated preferences to identify the four messages most likely to influence opt-out behaviours.
• The four preferred messages were then deployed in the live online opt-out journey via a randomised trial with members.

So how much of an improvement (reduction) in the opt-out rate did Nest experience?

None. In fact it got worse – the actual outcome moved in the opposite direction to the prediction. Despite clear preferences from the online survey, in the real-world setting, the expected consumer result did not follow.

We applaud Nest for publishing the results of this study. Nest encountered what we have learnt over years of primary research projects – there can be a big difference between stated intention and actual behaviour.

The ‘Say-Do’ Gap

Getting the research design right is essential to a research outcome that gets to the truth and delivers value in the real world. Listening to what customers say, without an appreciation of their biases and motivations – emotional, unconscious, and sometimes irrational – delivers only a veneer of insight. This may be useful in directing thinking, but can result in time and effort being invested in the wrong place. It’s the ‘say-do’ gap.

The ‘say-do’ gap applies also to intermediaries. The workplace pensions market is a good example of where it is essential to utilise a variety of methodologies to uncover the real drivers of behaviour. There can be a significant misalignment between ‘say’ and ‘do’ when advisers feel they ought to speak to the customer interest in responding to research, but we know this does not always drive their real behaviours.

For example, in our annual Corporate Wealth study, we analyse the criteria driving provider selection vs those which support retention (Figure 1):

• Surface data suggests that the workplace pensions market is highly price-driven (price / customer charges is clearly of highest stated importance).
• However, correlation with NPS (measurement of likelihood to recommend a provider) shows that service is far more likely to drive recommendations, leading to higher levels of new flow and retention.

This doesn’t mean price is unimportant. However price is relatively overstated in importance, while service is understated. What this more advanced analysis reveals is that to gain competitive advantage, participants need to focus on delivering high quality of service.

Figure 1: Corporate Pensions Drivers of Intermediary Behaviour

Bridging the gap

How do you ensure the results of your research will play out in the real world?

• A qualitative stage provides insight to emotional drivers and cognitive effects that are unlikely to emerge in a quantitative-only study. Behavioural science frameworks help provide a strong foundation for understanding decision making.
• If time and budget permits, take a deliberative approach in the qualitative stage which spends more time with people, and helps them reflect, project, and take a more considered view of what might motivate and influence their behaviours. This helps uncover latent needs and drivers.
• An ethnographic study may seem extravagant and it is generally under-utilised in the intangible world of financial services, but helps observe real-life behaviour and social and cultural influences on decision-making in a manner impossible through projective research alone.

These insights will deliver much better calibration of a subsequent quantitative stage.

Actual or perceived time pressures mean that an initial qualitative stage is not always undertaken. In these instances we look to advanced analytics, using a variety of techniques, to avoid relying solely on stated opinions. This more accurately predicts (albeit not as accurately as utilising an initial qualitative stage for calibration) the key drivers of behaviour, and how to ‘turn up the dial’ on satisfaction and advocacy.

Asset managers and quality of service

Asset managers are particularly at risk of the “say-do gap” in 2020. The FCA has clarified that asset managers need to improve performance in acting in investors’ best interests, with assessment of investor value becoming an important area of strategic focus.

In assessing the quality of service criteria, it is easy to ask consumers or intermediaries how their preferred providers are performing. This approach might be compliant but may not be useful. A more meaningful assessment identifies what is important to investors and allows asset managers to invest in the ‘moments of truth’ that really matter. This will ensure that service assessments have commercial value, resulting in happier, stickier clients.