March 13, 2017
Written by Michael Davidson
Science is heavily polarized and the process of evaluating science is murky for non-scientists. That is my main take-away of the AAAS conference for scientists and science-educators last week. As Dan Kahan suggests, people of all political affiliations will (subconsciously) fit empirical observations into arguments in support of their beliefs through a process called “motivated reasoning”. Without understanding the scientific process, as Naomi Oreskes writes, it is possible for even intelligent and respected people to be duped, as in Edward Murrow giving equal weight in broadcasts to both scientists and tobacco industry-funded lobbyists. For people without degrees in science, Dietrem Scheufele noted in his talk (1:08), science is often presented as cut and dry without developing a competency in understanding how to identify good and bad science. There are many good efforts to educate about the real-world benefits of science, but maybe what these scholars are pointing to is the need to start from where the audience is in explaining the scientific method. So, in fitting with this theme, I ask you to imagine if my bike mechanic was peer reviewed.
This bike screeches for unknown reasons.
It’s a cold morning, and plunging outside I notice a layer of snow has formed on my bicycle. It hasn’t been moved in a couple weeks while I was away. I give it a cursory brush, eager to jump on and get going. I push off – a gust of frigid air blows over my face – and as I make contact with the pedal, I hear it.
Screeech!
The piercing noise fills my quiet street. Another stroke and – Screeech! The sound belongs in a creepy old house, like opening a heavy wooden door unmoved in years. Screech, screech, screech. I make my way down the street. The noise varies as I go up and down hills. People cast a sideways look as I pass. On some days, I hear it; and others I don’t.
The sound is annoying but even worse is taking it into the bike shop to get it fixed. A number of potential fixes are proposed – re-oil the chain, tighten some bolts, replace the bottom bracket that connects the pedal to the frame, etc. These options fail. I take it into another shop without luck. Eventually with more parts ordered it may be fixed – at a high price tag – but I would not know which combination of conditions caused it or how to prevent something similar from happening again.
But, I wonder aloud, What if my mechanic did peer review?
I imagine my bike mechanic as a scientist designing an experiment to determine the cause of the screeches. A number of variables could be the culprit: the cold day; several days idling; frost… We recreate the same conditions over a week, modifying each variable one at a time, and measuring the change in noise. It may be diminished, change pitch, or go away altogether. With enough observations, we have an inspiration that combines our set of variables into a mechanistic explanation of screeching in bikes.
But, we don’t stop there. We write this up and send to the Journal of Transportation Annoyances, where other mechanic-scientists from around the country are invited to review the paper. Maybe one in warm southern Florida has heard similar noises and raises questions about the factor of cold mornings. Another asks how we accounted for different morning humidity over the course of the experiment. How can we be sure, yet another reviewer asks, that our explanation holds for much larger and smaller bikes?
Over the review period, which could take six months or longer, we may expand the scope of the experiments, address other questions in the screeches literature, and measure our certainty in the findings against other random changes from day to day. We send back responses to all the reviewer comments and only if they are satisfied, a revised version is published. This peer-reviewed article becomes part of scientific knowledge.
Now, of course, my bike can’t be in the shop for six months, and doing these experiments and writing them up takes more time and money than doing the imperfect solutions suggested by my mechanic. But, this research may help diagnose hundreds of other noises across the country. The theory we put together may explain other annoyances of bicycles – and maybe cars and motorcycles too. Sharing the cost of running this experiment beforehand could have potentially saved me and others thousands of dollars and hours spent answering similar questions.
Of course, these details are rarely included in media accounts of the research (even less in the lede). Methods sections of scientific articles are impenetrable mazes of jargon, and short summaries tend to highlight implications over process. Work like AAAS Project 2061 to engage K-12 students in support of Next Generation Science Standards is critical to develop these competencies early on. Outside the formal curriculum, however, we can all do a lot more to meet people where they are to explain not just the results of science but the amazing process of discovery.