Post 1 of 7 · Sampling
The planes that came back
In 1943, US planes came back from the war with bullet holes. The holes were on the wings and the tail. The army wanted to put armour there.
Abraham Wald, a statistician, said no. These planes came home. The planes hit in the engine never came back. So the engine needs the armour.
Bullet holes recorded on returning bombers, 1943
From the news
From Good to Great … to Below Average
Economist Steven Levitt checked the 11 winning companies in the bestseller. Circuit City had gone bankrupt and Fannie Mae needed a government rescue. Together the 11 did worse than the stock market.
Read the original ↗The bomber story, the original paper
Abraham Wald worked for the US Statistical Research Group in 1943. His wartime reports were only published decades later.
Background ↗Who is missing from this data?The data you can see is the data that survived.
Post 2 of 7 · Averages
The average that nobody earns
Add all the salaries and divide by the number of people. That is the mean. One billionaire in the list makes the mean very big.
The median is the person in the middle. Half earn more, half earn less. A billionaire moves the middle by one person only.
Yearly pay on one street, in dollars
From the news
Wealth per adult in the US: mean 551,000, median 108,000
The report lists both numbers side by side. Most news stories quote only the first one.
Read the original ↗Is this the mean or the median?If the word is only average, ask which one.
Post 3 of 7 · Percentages
Bacon and the 18 percent
In 2015 the World Health Organization said processed meat raises bowel cancer risk by 18 percent. Newspapers said bacon causes cancer. Sales fell.
Out of 100 people, about 6 get bowel cancer in their life. With bacon every day, it is about 7. One extra person. That is the 18 percent.
Lifetime bowel cancer in 100 people
From the news
Processed meats rank alongside smoking as cancer causes, says WHO
The 18 percent figure travelled around the world in a day. The absolute numbers did not.
Read the original ↗The same finding in real numbers
Out of 1,000 people who eat little processed meat, about 56 get bowel cancer. Out of 1,000 who eat the most, about 66. Ten extra people.
Read the explainer ↗18 percent of what?A big percent of a small number is still a small number.
Post 4 of 7 · Charts
The axis that starts at 34
In 2012 a US TV channel showed a tax going from 35 to 39.6 percent. The chart started at 34, not at zero. So the second bar looked five times taller.
Start the chart at zero. The big jump becomes a small step.
Top US tax rate if the cuts expire
Chart redrawn from the broadcast, not a screenshot
From the news
Top tax rate: now 35%, in 2013 39.6%
The on air chart started its axis at 34. Media Matters and FlowingData published the original next to a corrected version.
See the original chart ↗The same trick has a twin. In May 2020 the state of Georgia showed daily virus cases. The dates were not in order. So the bars looked like they were going down. Put the dates in order and the drop disappears.
Daily confirmed cases, Georgia, spring 2020
Chart redrawn from the published version, not a screenshot
From the news
A graph shows Georgia bungling its coronavirus data
Posted by the state health department on 10 May 2020. April 28 came before April 27. A state representative wrote to the governor and the chart was replaced.
See the original chart ↗Where does the chart start? Are the dates in order?A chart is a sentence. Read it before you believe it.
Post 5 of 7 · Cause and effect
Chocolate and Nobel prizes
In 2012 a medical journal showed something funny. Countries that eat more chocolate win more Nobel prizes. The line was almost perfect.
Chocolate does not make people smart. Rich countries buy more chocolate and pay for more universities. Money moves both.
Countries, chocolate against Nobel prizes
From the news
Eat chocolate, win the Nobel Prize?
The study appeared in the New England Journal of Medicine on 10 October 2012. It compared 23 countries and found a correlation of 0.79.
Read the original ↗Half a glass of wine a day can add almost five years to your life
The study followed 1,400 Dutch men for 40 years. Light wine drinkers lived longer. Nobody was made to drink wine by chance, so the wine drinkers were different people in many other ways too.
See the headline collection ↗What third thing moves both?Two lines going up together is not a reason.
Post 6 of 7 · Probability
The test that is 99 percent accurate
A disease affects 1 person in 1000. The test is right 99 times out of 100. You test positive. Most people say: you are 99 percent sick. Many doctors say it too.
Look at 1000 people. One is sick. About 10 healthy people also get a positive result. So 11 people are positive and only one is really sick. Your real chance is about 9 percent.
1000 people, one rare disease, one very good test
From the news
2,000 wrongly matched with possible criminals at Champions League
2,470 alerts, 2,297 wrong. Only 173 were real matches. Precision: 7 percent.
Read the original ↗How rare was it before the test?A rare disease makes a good test noisy.
Post 7 of 7 · Research
The studies in the drawer
74 trials of antidepressant drugs were registered in the US. About half showed the drug did not beat a sugar pill.
Almost all the negative trials were never published, or were written to sound positive. A doctor reading the journals saw 94 percent success. The truth was 51.
74 registered trials, and what doctors got to read
From the news
Antidepressant studies unpublished
74 trials sent to the FDA. 37 of 38 positive ones were published. Of 36 negative ones, 22 never appeared and 11 were rewritten to sound positive.
Read the study summary ↗What the Tamiflu saga tells us about drug trials
The UK spent about 500 million pounds on the drug. Researchers fought for four and a half years to see the full trial data.
Read the BMJ report ↗What did not get published?Silence is data too.