Several years ago I saw Al Gore’s An Inconvenient Truth with my father, an environmental scientist, and my mother, an accounts receivable. Before our very eyes Gore – with his film-worthy charisma and spotless suit – transformed our view of a topic previously brushed off as hippie into a genuine, critical and imminent threat. Even my father, who is well educated in the field despite not working specifically on climate change – was surprised by much of Gore’s claims.
Gore didn’t do it by himself. For one, he had his cinematic slide deck, with pictures so stunning and animations so sexy they made PowerPoint look like grandma in a rocking chair (sorry grandma). More than that, he had numbers on his side, numbers that were hard, solid facts; numbers that were indisputable.
Or were they?
A year ago after having taken a course on Knowledge Media Design at university and going through some of Tufte’s (brilliant) work, the more I thought about the presentation the more I was struck by the hand-waviness of the data. First, half his charts had no axes – or if they did, they gave no idea of the scale, had barely legible numbers, or were zoomed in or animated so quickly that you had no idea whether the distance from the bottom of the graph to the top was 1 unit, 100 units or 10 000 units (and if there were any skews). That makes a difference. Every graph when zoomed in enough will look like there are enormous rises and falls. It’s like a microscope – take a surface you think is flat, like a piece of paper, then zoom in enough and it looks like there are huge peaks and valleys.
The adage is that numbers don’t lie. Scientists, politicians, doctors, writers, artists use them all the time to convince people of things. Well maybe, but they sure as hell can do some serious fibbing.
Once in statistics class we did some critical analysis of the statistics in scientific papers and found very surprising assumptions, overlookings and even errors – like using the wrong type of statistical test for the type of experiment conducted. More than that, the professor discussed how the entire paradigm of statistical testing is flawed – and how you can obtain a “statistical significance” (the holy grail of stats, what every grad student prays they get in their experiment) simply by taking more samples. (Okay, not flawed per se, more like “limited”.)
On news we see it all the time – stats about everything from internet use to how to have hot sex. But at least in newspapers there’s some context or source given, and (rarely) things like number of subjects, background of subjects and actual details of the study. With today’s headline/140-char economy, it’s tough to get even that. twitter is infamous for giving people the bare headlines – it’s quick and convenient but has no depth, and with depth comes credibility. I suppose the argument there is that it’s real-time, and has links to details from reputable sources. Fair enough. twitter definitely has its uses. Still every once in a while I have an intense desire to pick up a copy of the New York Times to make me feel smart again.