Boy, was I relieved when I found out cigarettes aren’t bad for you!
It’s not as though I’ve had so much as a single whiff of tobacco in years, you understand. Nor am I even exposed to much secondhand smoke-- unless you count a sport jacket I used to wear during my tobacco-consuming days, which despite dozens of dry cleanings can still set off smoke detectors in three adjacent states. No, the problem is that my brand when I did smoke was Lark. In the gooey world of high-tar cigarettes, Larks were a virtual parking lot. Recognizing their product’s status as something less than a health food, the Lark manufacturers sought to capitalize on that shortcoming, providing proof-of-purchase seals that would allow smokers to send away for their own pulmonary embolism after their very first pack.
Given this, you can imagine my surprise when I discovered that despite all the studies linking smoking to disease, eminent doctors and research scientists have questioned the claimed significance of these experiments. Better still, some cigarettes, it turns out, can even protect the delicate tissues in your throat!
Cigarettes aren’t the only products whose reputations have been unexpectedly redeemed. According to recent studies, chocolate can actually inhibit the formation of cavities; high-fat nuts can lower the level of fat in the blood; and Wonder Bread, the only known bakery product able to double as a sturdy throw pillow, is really a diet food.
All of this is good news for the health conscious--as long as you’re willing to overlook a few teensy problems. The smoking study, for instance, was sponsored by the Tobacco Industry Research Committee. The chocolate study was sponsored by a chocolate manufacturer. The nut research was sponsored by the California walnut and almond boards. And the Wonder Bread study was sponsored by, yes, Wonder Bread.
The worlds of commerce and science are rife with this scarcely objective research--and not surprisingly. How many Snickers would the folks at M&M;/Mars sell if their advertising relied on government nutrition standards alone? (Thousands of calories bursting with lip-smacking lecithin!) How much chewing tobacco would U.S. Tobacco sell if its marketing department stuck to the unvarnished truth? (Sköl: When you’re looking for brown saliva.)
Lately, however, the problem appears to have become epidemic. There are oat bran studies sponsored by Quaker Oats, caffeine studies sponsored by coffee manufacturers, and nutrition studies sponsored by the food industry.
Now the problem of biased statistics is at last being addressed, thanks to Cynthia Crossen, an editor and writer at the Wall Street Journal and author of the recent book Tainted Truth: The Manipulation of Fact in America. The book is one of the most comprehensive looks at the national statistics game, revealing the way numbers are increasingly becoming something we can’t count on.
In the last few decades, Crossen says, we have become steadily inundated with supposedly quantitative information that simply doesn’t add up. If the public is going to be truly informed, it needs to know how the studies are conducted and, most important, who’s paying for them all in the first place.
The history of marketing and public relations is a murky one, and from the earliest days of human mercantilism, consumers have been cautioned to take nothing at face value. In the Middle Ages, English courts established the principle of caveat emptor (medieval translation: Let the buyer beware; contemporary translation: Some assembly required). The idea behind the expression--that shoppers assume the risk the moment they enter the market--was originally expressed by Anthony Fitzherbert, who wrote in his Boke of Husbandrie, If he be tame and have ben rydden upon, then caveat emptor. From a country clearly in need of a good spell-check program (Excyse me, has anybydy sen my vowels?), advice like this may seem hard to take. But the caveat emptor admonition is one that has endured.
One of the first things we learn when we become consumers, Crossen says, is that we shouldn’t believe what we read or hear about a product. It used to be that we knew something about most of what we bought. If you were buying a horse, you could actually look at the horse. Now we buy many things we have no knowledge of. There’s a tremendous amount of questionable information used in the business of selling, and courts will generally tolerate almost anything, barring extreme and outrageous lies.
Among the products people trust the least are politicians, and among the ways politicians most earn that distrust are polls. It was in 1824 that public polling became popular in the United States, when it was used to help predict the outcome of the presidential election between Andrew Jackson and John Quincy Adams. Lacking computers, complex statistical formulas, and even Cokie Roberts, early pollsters were forced to rely on informal methods at best, conducting most of their surveys by placing a box in a public square and inviting voters to deposit a slip of paper with the name of their favorite candidate penciled in. In a frontier culture that was only marginally literate, such techniques yielded somewhat dubious results, usually predicting that any presidential nominees would lose to a large smiley face by a margin of almost six to one.
More-sophisticated surveys would have to wait until well into the next century, when pollsters abandoned the straightforward question Who are you voting for? in favor of more elaborate and more numerous questions. Though such methods were intended to increase the accuracy of the surveys, as it turned out they introduced a variable that in many cases decreased it.
One of the things pollsters quickly found, Crossen says, is that the thing that most influences the answers you get in a poll is the way in which you frame the questions.
Perhaps the most egregious example of slanted questioning in political polling occurred during the 1964 presidential campaign, when Barry (I do not want to bomb North Dakota) Goldwater ran against Lyndon (I do) Johnson. Goldwater, justifiably or not, had developed something of a reputation for public petulance, and in the middle of the campaign the unironically named Fact magazine published a poll purporting to show that this testiness might disqualify him for the presidency. 1,189 PSYCHIATRISTS SAY GOLDWATER PSYCHOLOGICALLY UNFIT TO BE PRESIDENT, the headline of the story read. It was only when readers got further into the article that they learned that this unambiguous finding was based on some questionable questions. Among the items on the Goldwater questionnaire that had been mailed to the psychiatrists were such queries as Can you offer any explanations for his public tantrums? and Do you believe that Goldwater is psychologically fit? No or yes?
The Goldwater questionnaire belongs in my file called So Bad They’re Funny, Crossen says. Using a word like tantrums, when no one agrees that the subject actually has them, hopelessly loads the question. Asking if there are ‘any explanations’ for behavior suggests that it would be hard to come up with one. Finally, when you offer answers in the order ‘No or yes,’ you make it much likelier that people will pick the no.
Loaded poll questions are not just artifacts of an earlier, less enlightened political era. During the 1992 presidential election, independent candidate and pie-chart manufacturer Ross Perot published a questionnaire in TV Guide in which he sought to measure the popularity of the positions he had taken on the issues. Cynics, of course, would have expected a list like this to include such questions as Do you believe a pack of lemurs tried to disrupt my daughter’s wedding? Perot, however, played it straight--sort of. Among the questions he asked voters was the seemingly direct Should the president have the Line Item Veto to eliminate waste? To Perot’s delight, 97 percent of respondents answered yes, a ringing endorsement of one of his central beliefs. As Crossen and others observed, however, the last three words in the question left it utterly slanted, in effect asking respondents to vote for or against government profligacy. When the question was rewritten to a more neutral Should the president have the Line Item Veto, or not? and asked of a random sample, the total dropped to just 57 percent.
Even seemingly synonymous words in a poll question can elicit very different answers. Respondents will have different reactions, for example, when a pollster asks about taxes as opposed to revenue enhancers; welfare as opposed to public assistance; NASA as opposed to the Flying Wallendas; Panamanian general Manuel Noriega as opposed to Beelzebub.
Senator S. I. Hayakawa used to refer to some words as snarl words and others as purr words, Crossen says. Which ones pollsters use depend a lot on the results they hope to get.
People with a political agenda to push are not the only ones who manipulate polls, of course; people with products to push do, too. The history of advertising is rich with enthusiastic consumer endorsements based on polls that are conducted questionably at best. One of the most obvious examples of felicitous phrasing in a product poll came from the makers of Black Flag pesticides. In a recent survey, Black Flag’s manufacturers found that fully 79 percent of all Americans believed that using a roach disk would be an effective way to control this pest. The problem was, when most of the subjects were first approached, they admitted to never having heard of a roach disk, so it was left to the pollsters to describe one to them. The language they chose was less than objective.
A roach disk is a type of product that poisons a roach slowly, the explanation ran. The dying roach returns to the nest and after it dies is eaten by other roaches. In turn, these roaches become poisoned and die. How effective do you think this product would be in killing roaches? Though the Black Flag promoters reportedly decided against additional phrasing in which they promised that the few roaches that did survive would be happy to vacuum the house, pick up the dry cleaning, and take the kids to soccer practice, the glowing product description they did offer virtually guaranteed a favorable response.
More troubling than marketers who manufacture polling data are marketers who influence scientific data. In an increasingly health- and safety-conscious world, consumers are becoming more aware of just what it is they’re consuming, and more often than not they don’t like what they learn. Judging by recent health reports, popcorn will soon replace lethal injection as the accepted method of execution in at least 27 states, and massive helpings of fettuccine Alfredo will soon be the battlefield weapon of choice in most regional wars. For every food or other product that falls into this kind of public disrepute, there are manufacturers trying to rehabilitate its reputation--and for that they need scientists.
Among the most notable industries to turn to science for help are the makers of disposable diapers. As nonbiodegradable waste goes, disposable diapers--containing plastic, paper, and chemical gel, not to mention the special, uh, cargo they were designed to carry--have a half- life approaching cobalt’s. A newborn can go through up to ten diapers a day, giving the average baby an environmental impact rivaling that of Three Mile Island.
In 1988 the cloth diaper industry seized on these problems, questioning whether the nation’s landfills could tolerate such a potentially nonbiodegradable load and sponsoring research designed to assess the diaper threat. Not surprisingly, the study came to the conclusion that the United States was facing something approaching a national nappy avalanche. The solution? Cloth diapers. In 1990, Procter & Gamble--the manufacturer of both Pampers and Luvs disposable brands--hit back, hiring its own scientific consulting firm to conduct its own research. These studies found, to no one’s surprise, that once you took into consideration the compostability of paper and the waste generated by manufacturing and repeatedly cleaning a single cloth diaper, the disposable brands were actually the more environmentally friendly of the two. Indeed, so low was their ecological impact that the term disposable ought to be retired altogether, replaced by the less pejorative single use. The study stopped short of recommending that the labels LOW FAT, LEMON FRESH, or ABSOLUTELY NO CHOLESTEROL! be added to Pampers packaging, but the message of the research was nonetheless communicated.
What made both of these studies disturbing was that neither side disputed the essential facts of the controversy--the number of disposable diapers discarded each year; the number of washings cloth diapers require-- but both drew precisely opposite conclusions from them. Both sides, Crossen says, relied on what appeared to be objective data but used it subjectively. In short, she suggests, it was the conclusions the researchers wanted to reach that led them to the conclusions they did reach.
More heated than the diaper debate was the oat bran controversy. In 1986 a study--partly funded by the Quaker Oats Company--was published in the Journal of the American Dietetic Association reporting that products containing oat bran could lower cholesterol dramatically. Bran, of course, had long been thought of as one of those unpalatable good-for-you foods with the texture of cedar chips and the mouth-watering flavor of a packing crate. When the cholesterol study was released, however, all of that changed, and before long the market was flooded with a wave of new oat bran products, including oat bran cereal, oat bran cookies, oat bran muffins, and even oat bran toothpaste, licorice, and beer. Before manufacturers could release the first oat bran off-road vehicle, a few people began to wonder just how much good the magical stuff could really do.
The answer to this question came from a Harvard study in which subjects spent three months eating various types of grain muffins while their cholesterol levels were monitored. At the end of the period, a final blood test was done, and the results were not encouraging for Quaker. Oat bran muffins indeed appeared to lower cholesterol a few percentage points, but no more effectively than any other type of grain muffin. What’s more, in order to achieve even a minimal cholesterol change, subjects had to consume at least five large muffins a day. Even for a country hooked on soluble fiber, this seemed excessive, and unless consumers wanted to go through their day with the oat bran equivalent of a bolt of corduroy in their lower intestine, such bingeing did not seem like a good idea.
Quaker, of course, wanted no part of these findings, so the company couldn’t help but be pleased when yet another study (funded in part by Quaker and published in the Journal of the American Medical Association) found that more-moderate oat bran consumption indeed lowered cholesterol by 2 to 7 percent. This study, however, was a so-called meta-analysis, in which a number of other studies are collected and collated and their results reevaluated. Given Quaker’s financial investment in the study and marketing investment in the outcome, any results would be at least arguably suspect. The results that were produced were indeed good news for the company, showing that a single oat bran muffin could do everything from lowering cholesterol to prolonging life to batting .362 in an American League championship series. The newspapers faithfully reported the study’s findings. OAT BRAN DOES CUT CHOLESTEROL, USA Today, for one, announced.
Crossen cites other cases of questionable research in her book: There are the ongoing studies from the Council for Tobacco Research, an industry-sponsored group created to challenge the link between smoking and disease, spread the pro-cigarette message, and help Joe Camel get a regular guest-hosting gig on Barney & Friends. There are the studies by the Florida Citrus Commission showing that citrus fruits can reduce cholesterol and the incidence of heart attacks (Nurse, get me nitroglycerin, a defibrillator, and a tangelo, stat!).
Crossen believes research of this kind does a lot of harm to the national data pool--and that the trend won’t be easily reversed. I think that people respect science possibly more than they should, she says, mostly because they don’t understand it. When you don’t fully fathom something, you’re inclined just to bow your head and follow wherever it leads.
Of course, given what the proliferating numbers tell us, it’s not clear how much we may want them to come to an end. I can think of worse ways to spend a Sunday afternoon than sitting down in front of a football game with a couple of Hershey bars, a jar of high-fat pistachios, a BLT on Wonder Bread, and not an oat bran muffin in sight. Does that sound so awful to you? No or yes?
