Why Most Market Research Fails
The market research that most businesses conduct — a customer survey with 10 questions, a few Google searches on market size, a look at what competitors are doing — produces information that feels like insight but usually isn’t. The survey that asks customers what they would theoretically want produces answers biased toward social desirability (what respondents think they should want or what seems reasonable to claim wanting) rather than what they would actually pay for and use. The competitor analysis that looks at what competitors are doing assumes competitors’ decisions reflect actual customer needs rather than their own strategic assumptions.
Genuine market research — the kind that reveals what customers actually need, how they actually make decisions, and what they actually value — requires methods that access revealed preferences (what people do rather than what they say they’d do) and deep qualitative insight (why people make the decisions they make, in their own words, in sufficient depth to be genuinely informative). This research is harder to conduct than surveys, harder to quantify, and produces more useful information than any survey will.
Customer Interviews: The Research Method That Produces Real Insight
In-depth customer interviews — 30 to 60 minute conversations with people who match the target customer profile, focused on understanding their existing behaviour and experience rather than their opinions about a hypothetical product — are the highest-quality market research available to most businesses. The interview that asks ‘tell me about the last time you tried to solve this problem’ produces a specific narrative of real behaviour; the survey that asks ‘how important is this problem to you on a scale of 1–5’ produces a rating that may not correspond to any real behaviour.
The interview structure that produces the most useful insight: open with broad questions about the customer’s context and situation (what their day-to-day looks like in the relevant domain), narrow to specific questions about the problem you’re researching (when did they last encounter it, what did they do, how did they try to solve it), probe for the emotional and practical dimensions of the experience (what was frustrating about it, what they wished had been different), and conclude with questions about what solutions they’ve tried and why they work or don’t work. Avoid describing the product being built until after all these questions are answered — the goal is to understand the customer’s world before introducing the solution.
Surveys: When They Help and How to Write Them
Surveys are most valuable for validating hypotheses at scale that qualitative research has generated, not for generating the hypotheses in the first place. The survey that tests whether a finding from 20 customer interviews applies to a broader sample of 200 customers is producing genuinely useful information; the survey that tries to generate hypotheses from 200 responses to vague questions is producing the noise that leads to confident wrong decisions.
The survey design principles that produce more useful responses: ask behavioural questions about what respondents have actually done rather than attitudinal questions about what they think or feel (‘have you purchased X type of product in the last 12 months’ rather than ‘how interested are you in X type of product’), keep surveys short enough that completion rates are high (completion rates for surveys over 10 minutes decline dramatically, meaning responses skew toward the most engaged respondents who may not represent the general population), and avoid leading questions that suggest the ‘correct’ answer (‘how valuable would you find feature X’ implies X is valuable; ‘which of these features is most important to you’ forces a real ranking choice).
Observation and Ethnographic Research
Watching customers in their natural environment — doing the tasks the product is designed to help with, without the researcher explaining or interpreting anything — reveals what survey responses and even interview responses miss: the workarounds people use that they don’t think to mention because they’ve normalised them, the moments of confusion or frustration that pass unremarked because they’ve become accepted parts of the process, and the physical or contextual constraints that shape behaviour in ways that hypothetical research never captures.
The observation research that’s most practical for most businesses: job shadowing (accompanying a target customer through a relevant part of their workday to observe how they currently handle the problem the business is addressing), usability testing (watching users interact with a prototype or an existing product without assistance, noting where they struggle or make unexpected choices), and contextual inquiry (a combination of observation and interview that happens in the customer’s environment while they work, asking questions about specific behaviours as they occur rather than asking them to recall from memory).
Synthesising Research Into Actionable Decisions
The market research synthesis that produces the most useful decisions: identifying the patterns across multiple interviews and data points that reveal consistent customer truths — the problem that every interviewee mentioned unprompted, the solution approach that every respondent tried and found inadequate, the emotional response that consistently accompanied a specific situation. These consistent patterns are the research findings worth building on; the individual data points that appear in one or two responses are the findings worth flagging for follow-up rather than acting on immediately.
The customer persona or jobs-to-be-done framework that converts research into actionable product and marketing strategy: rather than describing the target customer by demographics (35-year-old professional in a major city), describe them by the job they’re trying to do and the circumstances under which they’re trying to do it (trying to ensure their small manufacturing team produces consistent quality when they’re not in the building to oversee personally). This job-based framing produces more actionable design and positioning decisions than demographic descriptions because it’s about what people are trying to accomplish rather than who they are.