What’s going wrong in food innovation?

Despite millions in investment, many new launches in the FMCG sector fail to achieve short term or lasting success, with even ‘product of the year’ plaudits being no guarantee of longevity.  So, what’s going wrong in food innovation, and can it be fixed?

The core issue lies in a fundamental misunderstanding of consumer behaviour and a reliance on outdated product development approaches. Many brands are still asking consumers what they want by relying on surface-level survey insights, trend data, and top-level market insights. This creates an illusion of certainty, but it is an approach only capable of delivering incremental ideas and ‘me too’ products.  It rarely leads to breakthrough innovation.

Traditional market research often focuses on obvious pain points. However, the real opportunities for innovation lie in uncovering ‘hidden tensions’ – the unspoken compromises consumers have internalised as ‘just the way things are’. These are moments where consumer hopes clash with reality, creating unresolved friction that is ripe for an innovative solution.

A simple example is that of laundry detergent. For years, the focus was on cleaning performance, but finally brands identified a deeper tension: the mess, waste, and low-level annoyance associated with measuring liquid or powder detergent. Laundry pods were born and were a huge commercial success. However, it wasn’t a problem that consumers were explicitly articulating, it had become a deeply ingrained compromise.

The same is true in food and drink. For years, the plant-based food sector made the mistake of assuming vegans would happily sacrifice indulgence for ethics. Ben & Jerry’s challenged this by understanding the emotional gap between permission and indulgence. They developed non-dairy ice creams that delivered the same creamy texture and indulgent flavours as their originals, proving that premium and plant-based can coexist and thrive. This was a fantastic example of innovation that didn’t just meet a need, it redefined what plant-based could be and opened up a whole new audience.

These examples of innovation could not have been achieved by relying on lagging indicators such as tracking competitors, reacting to shifting preferences, or skin-deep customer surveys.

Private label success

Adding to the complexity in the food sector is the disruptive rise of private labels. Historically, store brands were seen as inexpensive alternatives. This is no longer the case. Private label market share has reached record levels, and they are teaching a masterclass in innovation.

Private labels thrive on direct product experience. Consumers are initially drawn by competitive pricing, but their sustained loyalty is built on reliable quality and consistent performance. Our research shows consumers are often ‘surprised’ by the quality of store-brand products, leading to increased confidence and repeat purchases.

This underscores a crucial point: actual product quality trumps marketing claims. Private labels benefit from lower initial consumer expectations because no bold brand promises have been made.

Competing against own label

So, how can food brands compete and make the success of their product innovations more predictable?

It requires a fundamental shift in approach, moving beyond superficial market research and embracing a culture of continuous innovation. Brands must delve deep into consumer behaviour, emotions and needs, not just what they articulate.

This means incorporating ethnographic studies, behavioural economics, and real-time feedback to understand not just what consumers buy, but why they buy it. It’s about decoding cultural and behavioural signals before they become category norms. What is the hidden friction in the food items they interact with? What aspirational contradictions do consumers struggle to reconcile?

Food manufacturers need to go beyond ‘me too’ to identify emerging consumer trends and develop products with genuine differentiation and added value. Innovation shouldn’t be about creating a one-hit wonder but about forming an ongoing, systemised approach to staying ahead of the imitation game.

Consumer trust and purchasing behaviour vary significantly across categories. Innovation strategies must be tailored accordingly. For instance, value and functionality might be prioritised in cleaning supplies, taste, trust and quality are critical in food.

Five key steps

There are five key steps for brands looking to take their product innovation beyond the obvious:

• Identify hidden tensions: Uncover unarticulated needs and desires driving consumer decisions.
• Reframe the familiar: Transform existing products or experiences to meet evolving expectations.
• Balance opposing desires: Reconcile seemingly incompatible consumer needs, like indulgence versus health.
• Simplify the complex: Reduce friction and complexity to offer seamless experiences.
• Create emotional resonance: Build deeper connections by tapping into consumer values and emotions.

These principles will help to ensure that innovation is rooted in deep consumer insight and practical execution. It’s about designing without compromise, integrating opposing attributes seamlessly, and anchoring in deep insights to deliver solutions that feel inevitable, not engineered.

But crucially, category success belongs to brands that truly listen to both what is being said - and what is not.  By taking a more insightful approach to innovation and unlocking these hidden tensions, food and beverage brands can move from safe bets to standout successes.