That’s the headline of a piece on io9. I find the headline a bit obnoxious — we scientists are lecturing you, the unwashed masses, about what you’re doing wrong, when in fact scientists themselves are to blame for at least some of the misunderstandings described. But the actual content is very good.
Sean Carroll says very sensible things about “proof”. Science is mostly about accumulation of evidence, which allows us to update our model of the world via Bayesian reasoning (or as I like to call it “reasoning”).
Jordan Ellsburg takes aim at “statistically significant”:
“Statistically significant” is one of those phrases scientists would love to have a chance to take back and rename. “Significant” suggests importance; but the test of statistical significance, developed by the British statistician R.A. Fisher, doesn’t measure the importance or size of an effect; only whether we are able to distinguish it, using our keenest statistical tools, from zero. “Statistically noticeable” or “Statistically discernable” would be much better.
Well said. The fact that something can be “statistically significant” and simultaneously utterly unimportant is very often lost, particularly in descriptions of medical findings.
This item illustrates what bothers me about the headline of the piece, by the way. It smacks of blaming the victim. Scientists are at least as much to blame as anyone else for talking about “statistically significant” results in a misleading way.
The other items are well worth reading too. I particularly recommend the ones on quantum weirdness and “natural”.
To that list of misunderstood terms, I’d add ‘Scientific.’ The same way that you call ‘Bayesian reasoning’ just ‘reasoning,’ I call ‘scientific evidence,’ ‘scientific theories,’ and ‘scientific reasoning,’ just ‘evidence,’ ‘theories,’ and ‘reasoning.’
To qualify these things as scientific suggests, firstly, a binary distinction, between scientific and non-scientific, which of course is nonsense – there are only degrees of reliability of one’s inferences, scientific rigor is a continuous variable. Science is open-ended, and the quality of an inference can always be improved by gathering more data, or by widening the hypothesis space.
Secondly, it seems to suggest to some that there are types of evidence that are non-scientific, rather than simply inferences whose quality can be improved through more rigorous application of scientific method. It suggests the existence of non-scientific ways of obtaining reliable information, that there are fields of knowledge outside scientific enquiry, and that if I’m not a person who wears a white coat and works in a laboratory, then scientific method does not concern me. In reality, anybody who wants reliable information (i.e. anybody with any kind of goal) must seek to be scientific.
In any intro statistics course, the first mention of the word “significant” should be accompanied by strong emphasis on the connection of that word with the words “signify” or “indicate” as opposed to the (non-)connection with the word “important.”
I think this idea has a place in many other science courses as well, including (especially?) social science courses.