The Focus-Group Fraud

The infelicity of the candidate’s thought experiment was immediately apparent In the upbeat atmosphere of a campaign rally, with perky cheerleaders and brass bands poised to go giddy on cue, it’s usually considered unwise to muse aloud to parents in the audience about what would happen if they croaked and theft kids were left alone in the world. . . .

Dole was asked to explain his reasoning in a TV interview. “It’s what a couple of people have told me who had focus groups,” he replied. Was he impugning the president’s character? “I’m just repeating what focus groups said — liberals, men, women, Democrats, Republicans, conservatives.” Did it mean that Clinton’s not a good person? “You’d have to ask the people in the focus groups,” he replied “I wasn’t in the focus groups But I think it indicates that people trust Bob Dole.”

Actually, it doesn’t. It indicates that those people in those focus groups who said they would leave their kids with Bob Dole probably do trust Bob Dole. And that’s all it indicates. It tells us nothing about the public at large. Indeed, before you could say “Gotcha!” the Washington Post commissioned a poll showing that 52 percent of Americans would prefer Clinton as foster Dad; only 27 percent chose Dole.

But why be pedantic? There’s no reason to pick on Dole alone He was making a common error, another symptom of the latest disease to afflict the world of politics: focus-group hysteria. “They’re the hottest research mechanism going right now,” says Mark Mellman, a Democratic consultant. “We’ve done more focus groups in the last month than we did in the entire 1986 election cycle. People think they’re extremely fashionable and sexy.”

As recently as twenty years ago, focus groups were an obscure technique used by researchers in the field of retail marketing A group often to fifteen consumers sharing some characteristic — middle-aged housewives, teenage girls with disposable income, suburban men with young children — might be selected through a phone survey and brought together to taste a new breakfast cereal, compare proposed ad campaigns, or judge the new logo for a box of Goobers. Their responses are solicited by a moderator and recorded on video or audio tape. Sessions last as long as two hours, after which the lucky participants will be paid $ 40 or $ 50 for their time. In the end the client has a more complete understanding of the tastes and preferences of his potential consumers.

The result is called qualitative research, to distinguish it from quantitative research, which refers to the raw data gleaned from more conventional public-opinion polling. Polls draw on a large, randomly selected group of respondents, who, according to probability theory, will present a statistically accurate picture of the public as a whole. Polls are useful, indeed indispensable, for a market researcher, but they have their limitations. If you’re about to come out with a new cereal — say, chocolate soyflakes (yuk) — it would be too expensive to gather a randomly selected group of a thousand cereal eaters in a single place and force-feed them your bad idea. But you can, with relative ease and little expense, bring a focus group around a table to gauge their reactions to chocolate soyflakes. And when they all reach as one for the air-sickness bags, you’ll know you should probably stick with Fruit Loops.

No product is brought to market these days without extensive focus-group testing; moviemakers have even been known to reshoot the endings of their movies when focus groups have found the originals unappealing. (In the most famous example, Glenn Close’s character committed suicide in the original climax of Fatal Attraction. When focus groups objected, the ending was reshot so Michael Douglas’s wife could kill her. But everyone agreed, then as now, that Glenn Close is annoying.) TV news anchors are often chosen or dumped based on focus-group research. And no one doubts the usefulness of focus groups in the testing of consumer products. But their utility in political campaigns is more controversial.

Focus groups have been widely used in politics for only fifteen years or so. Once an instrument of national campaigns exclusively, they have become popular at every level of electioneering in the 1990s. Consultants will field a focus group for a variety of purposes. They can show campaign ads to test their effect. Pollsters, before conducting a survey, might use a focus group to test ideas about what to ask and how to ask it. After the survey, a pollster can use a focus-group discussion to probe confusing or contradictory results.

So what’s the problem? Focus groups, wrote the political analyst Stuart Rothenberg in a recent Roll Call, “are the most misused and fraudulent political technique of the decade.” Rothenberg’s complaint is that in all this flurry of activity candidates and consultants forget that focus-group results are not “projectable” onto the larger population. Polls are scientifically designed to apprehend public opinion; the results, within a margin of error, do tell you something about the thinking of voters at large. Not so with focus groups. The groups are too small. There’s no way of knowing that they represent anything more than the opinions of twelve people sitting in a room talking to a moderator in anticipation of making forty bucks.

This was Bob Dole’s error in confidently volunteering to adopt America’s children. And it is an increasingly common mistake. It is now routine for newspaper reporters to build entire stories around the projectability fallacy. Recently — to take an example almost at random — the New York Times sent a reporter to trail Bob Dole through the Midwest. The reporter talked to voters. And announced: “Mr. Dole’s effort to [explain his tax cut] has yielded more frustration than votes.” Quotes from real live people were included to obscure the article’s only indisputable fact, which was that the reporter had made a judgment about public opinion at large on the basis of a relatively small number of interviews. Bogus though it was, the story served its purpose of misleading the paper’s readers.

Even some political consultants — even some political consultants who themselves use focus groups — have grown uneasy with the technique, and their objections run deeper than the simple issue of projectability. “Focus groups are great for finding out what’s on the top of people’s minds,” says Sean Fitzpatrick, an ad man who worked for George Bush in 1992 — and who resigned in part from frustration with the campaign’s obsession with focus groups. “You can observe how people immediately react to a particular product, for example.

“But they are dangerous, even destructive, when you’re using them to make judgments for you, particularly on matters they haven’t thought very much about. In politics people often don’t know what they think. But you’re asking them to be instant experts. And once they’re in that role of expert, they’re no longer useful to you. They’re no longer reacting as normal voters.”

Unlike polls, which have statistical safeguards, a focus group can’t be replicated. Ditto the “data” it yields. The response a person gives to an advertisement shown in a focus group might be quite different from his more passive, less thoughtful reaction when he sees the same ad while slumped in his Barcalounger at home. In the artificial setting of a focus group, he can be steered one way or another by the moderator, or by his fellow respondents. He may be inhibited by their reactions to his reactions. All these elements of group dynamics make the conclusions drawn from focus groups highly suspect.

But in politics today they do not seem to be treated with the suspicion they deserve. “In my experience,” says Mike Murphy, a Republican political consultant, “the campaigns that have done the most focus groups have all lost. They’re a symptom of a weak campaign — a substitute for leadership, for devising a strategy and sticking to it. The theory behind focus groups is, if you don’t have a strategy, let’s throw a bunch of people in a room and have them tell us what to do.”

Even an outsider can tell when a campaign is relying overmuch on focus- group research. You just have to know what to look for. If a candidate flits from issue to issue, hammering tax cuts one week, then drugs the next, then crime the next, if he frames his opponent as wishy-washy one day and a committed liberal ideologue the morning after — does any of this sound familiar? — he is probably taking his cues from focus groups. And President Clinton is thought by many in the industry to be the most focus-group- obsessed politician in history. During the budget crisis of late ’95, the administration was reportedly doing one focus group a night. His speeches, with their famously stuffed sentences (“we will strengthen our families, protect our environment, care for the elderly, keep our streets safe”), sound like a focus-group transcript distilled to its essence.

It’s easy to account for the popularity of focus groups in politics. They suit the special demands of candidate and consultant alike. Though not at all scientific, they offer the veneer of pseudo-science. A consultant who specializes in qualitative research can plausibly claim for himself near- mystical powers of intuition and populist divination; he becomes as Svengali, the private role model of all political consultants. At the end of the process the candidate receives easyto-read reports peppered with illustrative quotes and anecdotes from real human beings, plus lots of phrases tested for use in speeches. Even better, focus groups cost less than polls while yielding lots of poll-like data. And perhaps best of all, they offer the candidate instant gratification. Most focus groups are held in offices designed for the purpose, with a two-way mirror on one wall. A candidate has the voyeuristic pleasure of watching those real human beings discuss him and his issues from behind the glass. This is not always for the best. Frank Rizzo, the former mayor of Philadelphia, once tried to lunge through the glass at a woman who had dismissed his candidacy with an ethnic epithet.

One charge in the general indictment of focus groups doesn’t stand up. Focus groups, Rothenberg wrote in Roll Call, “also are popular because the profit margin . . . is so high for pollsters — reportedly more than double that for quantitative studies.” In fact, profit margins are low. A focus group will cost you between $ 5,000 and $ 6,000; a poll begins at $ 12, 000. But a consultant will have to eat more than half his price in expenses — hiring a firm to canvass and cull the participants, paying them off with their $ 50, renting the room, and so on. Most focus groups are held in far- flung places, so there’s travel, too. But one truth will strike anyone who watches a focus group in action: No matter how much a moderator is getting paid for his services, it’s not enough.

A focus-group veteran once said to me: “You sit in enough of these things, and the sad truth is, you end up really despising people as a class.”

I myself am not a veteran, having had only two focus-group experiences. Journalists aren’t usually admitted to political focus groups, especially in a campaign season, since the information derived from them, while mostly worthless, is deemed sensitive and proprietary. But I’ve watched one on videotape and attended another, and I’ve begun to see the veteran’s point.

The videotaped focus group was more typical in technique. It was held last spring in Macomb County, Mich. As a key swing district in national elections, the original home of the Reagan Democrat and other beasts of political mythology, Macomb County is the political consultant’s Holy Land, Disney World, and Lourdes, all in one. The group was fielded by the Republican pollster Kellyanne Fitzpatrick, to answer the question, “Whither women?” (I paraphrase.) How, to be more precise, could the Republican party tailor its message to attract female voters? What — to try one more time — do women want? Accordingly, the group Fitzpatrick assembled was all-female, politically unaligned, blue-to-white collar, and uniformly disgruntled.

Focus-group participants, male or female, seem always to be disgruntled. This may have something to do with a process of self-selection, since the kinds of people who are willing to drive after work to a sparsely furnished, overlit room to talk to a group of strangers for two hours about a subject they care little and know nothing about do not, as a rule, lead lives filled with sunshine and song. Also, elections often turn on the votes of the undecided and unaffiliated and downright alienated; hence these are the voters on whom focusgroup specialists will, um, focus.

Not surprisingly, then, the videotaped focus group soon enough degenerated into a gripe session, a water-cooler chin-wag in which the cooler pours out nothing but bile. “When I say ‘Republican,'” Fitzpatrick asked the assembled women, “what’s the first thing that comes to mind?”

“Money,” a large woman said loudly.

“Big money,” another large woman spat out.

And “politician”?

“I just, for me, don’t believe anything they say. They just tell you what you want to hear.”

“It doesn’t matter who you elect, from my opinion.”

“The president is just a figurehead to me, it’s the Congress that does everything. But they never do what they say they’re going to do.”

“Me, I vote for the person — doesn’t matter if they’re Democrat or Republican.”

And Bill Clinton? Fitzpatrick asked. “Has he caused more pain or has he felt it?”

“I’m not happy with him,” said a woman, who, incidentally, looked like she wouldn’t be happy if Ed McMahon walked in and handed her a check from Publisher’s Clearinghouse. “He’s going on about kids smoking — it’s all I hear. Well, whatever happened to medical reform? What about that? He doesn’t talk about that anymore. Did I miss something?” Yes, ma’am. You did.

In comparing notes with focus-group vets, I’ve discovered several universal themes. People hate politicians, think politics is a con. They say that politicians tell them only what they want to hear and wonder why politicians never listen to them. They think the mainstream press is wholly biased and inaccurate and rely on it for all their information. They cheer bipartisan compromise and disdain congressmen who won’t stand on principle. They want to balance the budget by cutting congressional perks. They hate negative campaigning and, as the campaign progresses, remember only the negative things they hear.

The most cringe-making moments in the Macomb County focus group came when Fitzpatrick asked the participants why they felt as they did. This is supposed to be the point of focus groups, after all — to map the subterranean currents of public opinion, to divine the why beneath the what.

“Why do you say that?” she would ask.

Long, painful silence. Many shrugs.

Finally: “I can’t give you a reason.” “It’s just the way I feel, is all.” ” It’s ’cause that’s how it is,’ and soon they were all talking at once. A kind of sophistry quickly set in. One woman said her health insurance had not covered a procedure she had recently undergone. This is why she wants nationalized health insurance. Ergo et QED. Given their obstinate lack of interest in the subject, asking a group of average Americans about politics is like asking a gang of stevedores to solve a problem in astrophysics. Before long they’re explaining, not merely that the moon is made of cheese, but what kind of cheese it is, and whether it is properly aged, and how it would taste on a Trisket.

From this roiling stew of ignorance and stupidity, the focus-group wizard is supposed to distill some populist wisdom. As critics of focus groups often point out, it is difficult to grasp what people are thinking when they aren’t. Combining her focus-group research with poll results, Fitzpatrick composed a “playbook for Republicans” called “Winning the Women’s Vote.”

Her report closed with a section on “communicating to women voters,” designed for sweaty-palmed congressmen desperate to please the ladies back home. “Words are very important in communicating to women,” Fitzpatrick wrote. “Some hard and fast rules: Never say ‘Tax Cut’ without preceding it with ‘Middle Class.’ Never Say ‘Cut Government Spending’ without calling it ‘Wasteful’.”

If you want to know how the results of focus groups are used, listen closely to your favorite politician, assuming you have one. A focus group can influence everything from the color of the shirt a candidate wears to the part in his hair to which issues he dwells upon and which he avoids. But in the current craze it is political language that is most heavily determined by focus groups. “Language,” Frank Luntz is fond of saying, “is everything.”

Frank Luntz is the Republicans’ uncrowned king of focus groups. He is widely credited with designing the Contract with America, which, in turn, is widely credited with the GOP victory in 1994. He takes credit, most recently, for shaping the Republican vow to “end the IRS as we know it” — a vow that brought down the house when it bellowed forth from both Jack Kemp and Bob Dole during their acceptance speeches in San Diego. “I knew — I knew — that line would go through the roof,” Luntz told me.

How? “Because I tested it!” These days Luntz specializes in “instant response,” a sub-genre of focus groups, a craze within the craze. In instant response, a large focus group, 35 to 50 people, is shown a video of a speech or commercial. In their hands they hold dials wired to a computer.

On another television, unseen by them but closely watched by the moderator, a graph rolls across the video image. On the graph are lines reflecting the group’s reactions in real time. When the lines curve above 5, heading toward 10, it means the participants like what they see and hear; lines sinking below 5 show they’re not happy.

Luntz’s research and Luntz himself hold many congressional Republicans in thrall. Walking the halls of the Capitol with him can take time, as he is often stopped by the members.

“What have you got for me?” they ask eagerly, and usually Luntz has something focus-group fresh. A few weeks ago he discovered that focus groups don’t like it when a candidate says, “We will deny benefits to illegal aliens.” But when the candidate announces, “We will not give benefits to illegal aliens,” the lines on the graph smile upward. This semantic pearl he has dropped in the trembling hands of every grateful congressman who asks.

Newt Gingrich is a Luntz admirer, and Luntz spent time in the week of Sept. 23 preparing Gingrich for a debate in Williamsburg, Va., with Trent Lott, Tom Daschle, and Dick Gephardt. “Preparing” Gingrich, in this instance, meant feeding the speaker soundbites that had brought coos from focus groups around the country.

The night of the debate, Sept. 29, Luntz convened an instant-response focus group of about 35 swing voters in a Williamsburg motel. TV monitors were placed before rows of chairs in a dreary conference room in the motel basement.

Luntz began with a pep talk. (“I need to let them know that they can’t pull one over on me,” he had told me earlier.) “You must give me your reaction on a second-by-second basis,” he told them now, explaining how the dials worked. “I want you to feel comfortable. Tell me what you think. Tell me what you feel.”

As it happened, Virginia senator John Warner was debating his opponent that night as well. Frank switched on the last few minutes of the Warner debate to warm up the group and gauge their reaction. Warner’s Foghorn Leghorn countenance appeared on the screen, and in the back of the room, on Luntz’s TV monitor, we could see the graph superimposed on it. All the lines were on 5 — neutral. Warner spoke. “React to every single thing he says,” Frank called out. The lines rose into favorable territory. And then Warner said: “I think it’s important that we end the IRS as we know it.” And the lines spiked up.

“You see?” Frank whispered to me excitedly. “You see? Didn’t I tell you?” Warner is a Luntz client. “He just eats this stuff up.”

The Gingrich debate was a focus group’s dream. Focus groups, especially with swing voters, complain incessantly about the incivility, the partisanship, in politics. Mention “Republican” or “Democrat” and the participants frown, the lines on the graph head south. Speak softly, speak of consensus and cooperation, and the lines will climb like rockets.

Gingrich and Lott had, of course, been marinating in focus-group-approved language; so too Gephardt and Daschle. All national politicians in 1996 are focus groupies. As a consequence the debate was sleepier than a fly-fishing show on cable. The two Republicans smiled at the two Democrats across a table, as they pelted each other with little marshmallows of rhetoric.

“I hope,” said Lott in his opening remarks, “we will not get involved in what quite often has been partisanship and name-calling.” Fear not! The focus group was happy. It was a happy debate.

“We want to solve the practical everyday problems that people in this country face, Families First,” said Gephardt.

“Wasteful Washington spending,” said Gingrich.

“We need Families First,” said Daschle.

“Common-sense health-insurance reform,” said Lott.

“We have got to deal with people’s practical everyday problems,” said Gephardt.

“My 81-year-old mother-in-law,” Gingrich said, “my mom and dad in their seventies.”

“We asked people,” said Gephardt, “‘What are your everyday problems?'”

“Wasteful Washington spending,” said Lott. “A common-sense approach.”

“Working families,” said Daschle. “We call it Families First.”

“The four of us can chat in a positive way,” said Gingrich. “My mother-in- law is 81. Common-sense practical things.”

“My mother is 83 years of age,” said Lott. “Tax relief. And one other thing: education.”

Only occasionally did the lines frown, when the Democrats would turn dark, forgetting their focus groups. “Deep cuts in Medicare to pay for tax cuts for the wealthiest Americans,” said Gephardt. “A huge tax break for the wealthiest people in this country,” said Daschle. “Biggest polluters to dirty the air and dirty the water.”

So goes debate in a politics ruled by focus groups. Owing perhaps to the occasional Democratic negativity, the Republicans, and particularly Gingrich, scored far better than the Democrats.

Frank was beside himself. “I love this,” he said. “I live for this.” As soon as the focus group was paid off and dismissed, he rushed to tell Gingrich the good news.

We found the speaker talking on a pay phone outside the press briefing room in another hotel. Gingrich waved. Frank ran up. “You are the only one of the four whose approval went up in a statistically measurable way,” he told the speaker.

Gingrich beamed. “Frank says I am the only one of the four whose approval went up in a statistically measurable way,” he said into the phone.

Tony Blankley, Gingrich’s press secretary, cornered Frank. “My impression is that nobody hit it out of the park,” said Blankley. “I mean, there was no blood.”

Frank examined his notes. “There’s a lot here,” he said. “Common sense – – worked very well. Excellent. Wasteful Washington spending — very good. But they’ve got to remember, tax relief, not tax cuts.”

Blankley nodded.

“And Families First worked very well.”

“For us?” Blankley said, hopefully.

“No, for them.”

“Oh,” said Blankley.

“Practical solutions — yes . . .” Frank continued, as if to himself.

He went off with Gingrich that night to explain his findings further, and perhaps to celebrate their triumph, and by now he has surely written his standard three-page report, to be distributed to anxious candidates. “It will take the best pieces and the worst pieces,” he told me, “and show how they can be used.”

You can easily imagine a hapless Republican candidate — or Democrat, too, for the Democrats are as wired with focus groups as their opponents — you can see him standing on a hay bale somewhere under a broiling autumn sun, addressing a half-interested crowd of voters, his sweaty forehead straining with the pressure of so many focus-group soundbites, and wondering: “Did I just say I’d ‘cut government spending’? Didn’t I say ‘cut wasteful government spending’? Was it ‘middle-class tax relief’ or ‘tax cut for working families’? Oh God. . .” And deep in his trance he sees the focus groups rise up . . . they are displeased . . .

And here at last is the overriding irony of focus groups, of their use and abuse in contemporary politics. They have come to full flower just at the moment when conventional wisdom tells us that the system resists as never before the hopes and needs and desires of the average voter. And the average voter heartily concurs. In making the complaint he ignores the groveling figure of every politician and political operative in the country hunched around his feet, their eager and upturned faces smeared with the polish from his boots. That, circa 1996, is democracy, which we all honor in theory and practice. For as Winston Churchill famously wrote in Bartlett’s Familiar Quotations, “Democracy is the worst form of government, except for all the others.” Of course, that was before they invented focus groups.

By Andrew Ferguson