William Storage, 20 March 2007  (email me)

Catching the Chronicle writers publishing some bad math may hardly seem noteworthy to anyone with a smidgen of knowledge of statistics, but the article in question (Police Fail to Report Traffic Stop Data, Mar. 7 2006) is connected with a much larger and more interesting issue of bad math in the analysis of racism in America. A look at the larger issue reveals that bad math in journalism is not merely the consequence of what conservatives might see as the fuzzy-thinking product of a Berkeley liberal arts education. Bad math in print extends beyond ideologies and party lines.

Background

I first became interested in bad media-math fifteen years ago when a friend, who happens to hold a PhD in Chemistry, reported that he had tossed his electric shaver and returned to blade razors. The reason: an article in the Wall Street Journal on a study of cancer in blade versus electric shaver users. So I got a copy of the article, and was appalled. The Wall Street Journal generally gets high marks for credibility, but their science coverage can, as this report showed, be abysmal. The study involved 65 patients with cancer, and showed a slightly higher cancer rate in electric shaver users. There you have it; facts is facts. The investigators, violating rule number one of control-groups, failed to address the rather significant possibility that the lifestyles of blade and electric razor users might be significantly different. I don't know how different, but I can imagine, for example, that electric users might lead faster-paced lives with greater stress, or perhaps greater exposure to industrial pollutants. I don't know this to be the case, but it is the responsibility of those conducting the study to show that no significant differences (besides electric shaver usage) exist in the populations showing different cancer rates - something not addressed in the WSJ article. Studies, and the news articles that present them, are guilty until proven innocent. That is the nature of science: such conclusions must withstand vigorous skepticism.

After the electric shaver piece, I found similarly poor material in the Journal on mercury-based tooth amalgam and silicone implants used in aesthetic and reconstructive surgery. In the '90s, jury trials resulted in large awards to plaintiffs claiming damage from mercury amalgam and gigantic awards in tort suits over silicone breast implants, despite some initially surprising data showing that groups with silicone implants and mercury-based amalgam in their bodies had better health and lower cancer rates than those without. Silicone and mercury amalgam probably do not cause better health, but they may very well correlate with higher income, access to better health care, and perhaps greater interest in personal health.

Journalists, who are rarely trained in math, sometimes see numerical results of a study and then publish misleading conclusions. The problem is exacerbated when those performing the study are sloppy or biased. Such articles made me feel like the world needed more bad-math-and-science police - a job I'd love, but I didn't know where to submit my resume. Letters to editors were not terribly rewarding. At least now the internet allows me to publish opinion pieces on bad math at no cost to all having interest and search-engine skills. Perhaps if everyone called journalists on bad math, things could be improved, and we could raise the gross national credibility. One article (this one) may be a small increment, nonetheless here is my two-cent piece.

Racial Profiling by SF Police

An American Civil Liberties Union study concluding that black people were stopped by police at a rate 3.3 times greater than that of white people prompted the San Francisco Police Commission to require monthly reports from police on their numbers. Investigative reporters at the Chronicle, in a piece packed with self-praise for their bold challenge to "the man", found that African Americans were 2.5 more times as likely to be searched when stopped than whites. While much of the body of the long article dealt with failures to collect and retain such data, the piece opened and closed with the implication that the investigator had cracked a case of racial profiling.

The obvious question that arises from such reports is whether differential arrest rates are a consequence of differences in criminal behavior between groups, or are due to inappropriate discretionary practices of the police and judicial system. Many hold religious adherence to one notion or the other without basis. National arrest rates, which may be grossly different from the rates of criminal acts, are much higher - typically a factor of 10 over the past several decades - for blacks than for whites. If arrest rates reflect criminal behavior, one might argue that the San Francisco police is negligent in not searching African Americans stopped for traffic violations at an even higher rate. After all, national arrest data strongly shows that those wanted for violent crimes are more prone to commit routine traffic offenses than those not wanted for violent crimes. If arrest rate differences reflect discretionary behavior of arresting officers, then this might be grounds for the charge of racism.

I say only "might" because from the perspective of set theory and logic, racism (intolerance of another race) must involve interaction between races. If black police arrest blacks at a different rate than white police arrest whites, we can conclude nothing about racism-motivated arrests. A meaningful study investigating racism as a possible cause of differential search or arrest rates should have started with a comparison of four rates: white-white, black-black, black-white, and white-black. Surely, the crack team of Chronicle investigators had access to such information. Their failure to collect these statistics up front reveals bias, math-incompetence, or both.

That the Chronicle reporters seem to reach conclusions (probably based on their perception of the conclusion's popularity to the local news market) and then seek data and anecdotes to support these conclusions is of passing interest. But the prospect that differential arrest rates of blacks and whites could mean that our police force is racist, corrupt or otherwise wrong warrants serious attention. Even if black and white officers both arrest blacks at a higher rate than whites (i.e., if racism is ruled out), the situation is still undesirable if the rates don't reflect differences in criminal behavior rates.

While organizations like the ACLU give the topic of differential arrests rates a huge amount of coverage, they usually offer no support for the charge of racism beyond the arrest rates. Unfortunately, this bad math is rarely challenged. Those most vocal on the topic seem to be either racists who assume that blacks are arrested more because they are innately prone to commit more crimes due an inherent condition ("criminal mind", lower IQ, etc.), or politically-correct academics who refuse to allow the possibility that cultural - if not innate - differences between black and white groups could result in different criminal behavior. Neither of these extremes show much interest in the few solid studies that have sought to separate criminal behavior from discretionary justice.

Some solid statistical analyses has shown that, in the cities studied, arrest rates of blacks and whites living in the same neighborhood are in fact very similar, the most thorough being one conducted in London in 1981 by Tuck and Southgate. I wonder what the Chronicle writers might dig up if they looked only at say, San Francisco's Tenderloin district. Similar arrest rates in a limited geographic region would be very valuable to a good statistician. But, oddly, such a finding seems to be uninteresting both to racists and to those who see racism where none exists. Racists don't care for evidence that blacks and white have similar criminal behavior; it conflicts with their belief. Those in the racism industry shun evidence that police are not racist; it conflicts with their belief.

Another noteworthy study, warranting praise for both its testing methods and its data analysis, was done by Lange, Johnson and Voas. They looked at arrest/stop data on the New Jersey Turnpike in 2003-2004. Its results were published in the June 2005 Justice Quarterly, showing that in an unmanned-radar and random-camera survey of drivers on various segments of the turnpike, blacks showed a somewhat higher rate of speeding by more than 15 mph over the limit. It also showed that the New Jersey Turnpike police's arrest data from before and during the radar study was perfectly consistent with the speeding data. Thus the racial-profiling charges that had been leveled against the police there were unfounded. The Ney Jersey police received immense television coverage because of accusations of racial profiling in preceding years (and is cited in the Wikipedia topic on racial profiling), but TV and print journalists showed no interest in Lange's findings.

A 2001 study in Cincinnati by Eck, Liu, & Bostaph found both that differential black-white search rates of stopped drivers disappeared when the study populations were defined on the basis of geography or requests for police service. In other words, in Cincinnati, blacks were arrested at a the same rate as whites in two neighborhoods, but the arrest rates (and calls to the police for help) in those two neighborhoods were vastly different. Blacks represented a larger percentage of residents in the neighborhood with the higher arrest rate.  Interesting statistics, which still warrant investigation of all sorts of race-related issues such as housing and income discrepancies, but that do not support a claim of racial profiling by police.

The Cincinnati and London studies touch on a very important aspect of drawing conclusions on the basis of measurements within and between groups or populations. This is math without numbers - so-called set theory. At this point I'd like to go beyond the narrow to topic of bad math in the analysis and reporting of racial profiling by police and discuss bad math in the world of racial studies. And at this point I'll probably turn a lot of those who thought they were my supporters thus far into detractors. But remember, I didn't accept the job of math-cop to promote a social agenda, just to police bad math.

In addition to arrest rates, white and African Americans show big differences in rates of marriage (though not divorce), income, and cervical cancer mortality. The latter could be the result of genetic differences or social factors related to prevention, detection and availability of (or interest in) therapy. Discussion of differences in income between races has been a hot topic for at least a century, starting with discussion of intelligence and IQ, then centering on unequal opportunity, and later the opportunity/IQ debate brought on by Murray and Herrnstein's 1994 book, The Bell Curve.

I'll cut to the punch line. While I disagree with the social scientists who deny the possibility that race could account for differences in general intelligence or very specific mental skills (i.e. IQ), there is simply no mathematical or scientific basis for concluding that the often-cited 15 point difference in mean IQs between blacks and white is inherent in racial difference. The Bell Curve discussed this issue at length, implying - some say - that this difference was inherent in genetic differences between races. Stephen J. Gould vigorously attacked the book and its authors. This was unfortunate; less vigor would have resulted in a stronger claim. Gould could have stopped with an indictment of fatal flaws in the assumptions and logic used by Murray and Herrnstein, but he went on to throw stones. He also made an impassioned plea (The Mismeasure of Man, 1996) that 100,000 years of Homo Sapiens evolution is far too little for subspecies (races) - despite superficial differences in outward appearance - to have significant genetic differences, especially given the relatively high rate of genetic interchange between the groups. However, in the last few years the relationship between biology and race that Gould deemed negligible appears to be on the verge of reappraisal.

Gould's overzealous response notwithstanding (along with that of countless academics who dismissed Bell Curve as conservative or racist propaganda), The Bell Curve, from a math perspective, really is fatally flawed. It makes the assumptions that intelligence is strongly heritable and that it is immutable - nuggets of "common sense" that have no place in science. More interestingly to the mathematician, it infers conclusions about the differences between two groups based on observations within each group. The combination of immutability and heritability of intelligence (as measure by IQ) needs only one piece of evidence to be dismissed, and many such pieces exist. For example, American Jews in US military testing during World War I scored below their white non-Jewish complement. A generation later, despite very low genetic interchange with non-Jews, their new IQ test scores made American anti-Semites change the topic of discussion rather quickly.

 Drawing conclusions about the differences between two groups based on observations within groups is a fascinating concept to which I will devote the rest of this discussion. In the particular case of black-white IQ discrepancies it goes like this. White and African Americans each define a group or population. For the sake of argument, within each group (despite Gould's argument to the contrary - and he knows biology better than I), I'll accept that IQ is significantly heritable. That is, I'll assume that smart parents are somewhat more likely to have high-IQ children than are low-IQ parents. Thus genetics accounts for the difference between low-IQ whites and high-IQ whites. Now add to this the information - or rather, data - that mean white IQ is 15 points higher than mean black IQ. The very wrong conclusion often drawn from this data is that genetics accounts for the difference between the mean IQs of whites and blacks. If the error in this conclusion is not apparent to you, rest assured that you are in good company, and that poor math education runs deep in America. You are an innocent victim, unless of course you are a racist, in which case you needed no data to support your conclusion/belief anyway.

Tally on a Troglophilic Sallie

To drive the error of this line of thinking home, let's step away from a subject that is racially charged and into one that is completely harmless, a personal favorite of mine - speleology, the study of caves. More specifically, lets look at vertebrate biology in Appalachian caves. Ron Stewart, a passionate amateur herpetologist and inspirational high school teacher once got me interested in plethodontid salamanders, a family of lungless critters that do not at all adhere to the standard model of amphibian metamorphosis. After losing their gills, they breathe through their skin, never acquiring lungs. The genera, Gyrinophilus and Eurycea, common in West Virginia caves, include several neotonic species, which either retain larval characteristics into adulthood, or achieve sexual maturation during a larval state. Salamander populations isolated from other populations of the same species (such as in cave systems that have been isolated from each other by erosion of the limestone beds in which they form) display significant differences in their physical characteristics, even within a species. Also, surface and cave populations of the same species can look remarkably different, including the presence or lack of external gills, a set of long balancing appendages near the mouth, and dorsal and ventral fins. These variations continually frustrate salamander taxonomies. Very skilled biologists find themselves in the same logical dilemma of drawing conclusions between groups based on observations within a group, and taxonomical debates last for decades. Gyrinophilus porphyriticus, the Porphyry Salamander, is a species with a broad range, living in caves and in shady surface streams. Gyrinophilus gulolineatus, with a much smaller range, was eventually declared invalid, a mere phenotypic variation, but Gyrinophilus subterraneus, who only live in caves, ultimately made the grade.

Lets says, for purpose of example, that measurements of Gyrinophilus porphyriticus in shady mountain brooks show them to be, on average, larger and more colorful than cave-dwelling Gyrinophilus subterraneus. They are also observed less often with external gills as adults, and less often observed to be cannibalistic than their cave cousins, the subterranea. From these measurements and observations, one can easily draw certain conclusions about the relative natures and appearances on two species of Gyrinophilus - and they would be dead wrong. In practice, it might go like this: You find a small, pale, adult salamander with gills in a cave and observe it eating its sibling. So you conclude that - odds are - it's a subterraneus. Nope. You have no basis at all for that conclusion. Because when specimens of Gyrinophilus porphyriticus live in a cave they they are much smaller and paler than their siblings on the surface, and more likely to retain gills and be cannibals. Taken out of the cave they grow big, colorful, and lose their gills. Caves do that to a guy. I.e., it's the environment, dummy. You took in-cave (within-species) measurements of subterraneus and compared them to on-surface (within species) measurements of porphyriticus; and then you drew a conclusion between subterraneus and porphyriticus. Bad math - plain and simple.

Blacks have lower IQs than whites. Blacks also have lower incomes. Jewish Americans' IQs were not immutable. IQ's of blacks probably aren't either. The Bell Curve says blacks have lower incomes because they have lower IQs. Non Causa Pro Causa, and no data exists to support this claim of causality. An equally valid conclusion is that blacks have lower IQs because they have lower incomes. Like Gyrinophilus porphyriticus, when removed from the "cave" - whether the cave is self-imposed (cultural), the result of ongoing racism, or the aftermath slavery - they may show higher IQs and incomes than whites. Numerous possibilities exist; I'm not interested in discussing social science here, merely bad math in the press.

Racial matters reveal the worst examples of bad math. Bad math is used to show racism where no evidence for it exists (New Jersey State Police) and to promote racism by reaching bad conclusions about correlations between intelligence and race (The Bell Curve). Not if I can help it. Not on my math beat. Now move along please...