As I’ve mentioned before, I like the BBC’s podcast More or Less, which examines the use and misuse of statistics in the news. It generally conveys some reasonably sophisticated ideas engagingly and correctly. Here’s a nice example I learned from them recently.
The British Government has set a goal for the time people spend in immigration queues at Heathrow. According to an article by Tim Harford, host of More or Less,
The Border Force is supposed to ensure that passengers from outside the EU get through immigration checks within 45 minutes 19 times out of 20, while EU-based passengers should get through within 25 minutes, again 19 times out of 20.
They then measure whether this goal has been met:
At regular intervals [once per hour, to be precise] they pick somebody joining the back of the queue and then time how long it takes for that person to clear immigration.
At first glance, this might sound like a reasonable method, but in fact it’s not. The reason should certainly be familiar to astrophysicists (and probably to lots of other sorts of scientists). I’ll put a page break here just in case you want to think about it for a minute.
This measurement is subject to a big selection bias. If you pick one passenger every hour to measure, you systematically undersample the people who show up when the immigration area is crowded and oversample the people who show up when it’s not.
If you happen to arrive when the rate of people entering the queue is 1000 people per hour, then you have a 1 in 1000 chance of your wait time being measured, but if you show up when only 100 people per hour are arriving, you have a 1 in 100 chance of being measured.
The correct procedure would be to measure the wait time of every 1000th person (or some other number of your choosing).
This sort of thing shows up all the time in astrophysics. Suppose you want to know what percentage of galaxies are spiral galaxies. Here’s what you can’t do:
- Point your telescope at a random patch of sky.
- Pick a galaxy in that patch, and determine whether it’s a spiral.
- Repeat ad nauseam.
The reason you can’t do this is that your sampling procedure will underrepresent crowded areas of the sky. (If there are 100 galaxies in your field of view, any given galaxy has a 1/100 chance of being represented, but if there are only 10, each galaxy has a 1/10 chance.)
This error wouldn’t matter if galaxy type were uncorrelated with crowdedness, but as it turns out there is such a correlation.
Similarly, the British Government’s procedure would be fine if wait time were uncorrelated with airport crowdedness, but …
