Ask the right consumer the right questions

On delayed flights, market research, and why customer satisfaction surveys are good in theory but chronically flawed in practice

One of the nearly ubiquitous experiences of modern consumer living is the follow-up customer survey. “You may receive a survey in the coming days” is recited almost reflexively, as often is its tagalong buddy, “Anything less than a 9 or 10 is considered a failure”.

■ In theory, these surveys are a clever attempt to collect valuable market data that otherwise might be too hard to uncover before it’s too late. But a truly astonishing number of these surveys fail to ask the right questions, corrupting the data, hiding the information they’re supposed to reveal, and generally wasting well-meaning customers’ time.

■ That’s because most of the surveys ask about a narrow range of judgment points, and far more often than not represent the experience as belonging entirely to the front-line, customer-facing employees. Most reasonable people realize that there are second-order and third-order causes that lead to their experiences, but the surveys rarely probe that far.

■ An airline survey, for instance, might ask about whether a cabin crew was friendly and whether the lavatories were clean. These things matter, of course. But rarely do those surveys go on to ask equally important questions like, “Was everything about the online experience satisfactory?”, “Did we account for enough overhead bin space?”, or “Did you spend too much time waiting for a connection because our crews were mis-scheduled?”.

■ There are countless times when things go wrong beyond the control of frontline customer-service employees, but the surveys are often structured so that they get punished in the scoring for factors well outside their control. Well-meaning consumers then start to pull their punches (not wanting to punish people for problems beyond their control), and solvable issues go undetected.

■ A good guideline in life comes in two parts: “If you see a problem, solve it. When attempting to solve a problem, be sure to solve the right problem.” Surveys all too often look like good attempts to do the first part right while failing miserably at the second.