There’s a popular myth that science progressed because of a revolution in the way people approach knowledge. This is a self-serving myth that arose in the 1600s by people who wanted to claim special authority. This is why they came up with the marketing term “The Enlightenment” for their philosophical movement. If you look into the actual history of science, scientific discoveries pretty much invariably arose a little while after the technology which enabled their discovery was invented.

There is a reason we did not get the heliocentric (really, Copernican) theory of the solar system until a little while after the invention of the telescope. There is a reason why we did not get cell biology until a little while after the invention of the microscope. If you dig into the history of specific scientific discoveries, it’s often the case that several people discovered the same thing within months of each other and the person we credit with the discovery is generally the one who published first.

This is not to say that there are never flashes of insight or brilliance. So far as I can tell Einstein’s theory that E=mc² was not merely the obvious result of measuring things using new technology. That said, it would almost certainly never have happened had radioactivity not been discovered a decade earlier, which would not have been possible without certain kinds of photographic plates existing (radioactive decay was discovered by Henri Becquerel and Marie Curie in the 1890s as they were studying phosphorescence and exposed photographic plates wrapped in black paper, which showed that something else was going on besides phosphorescence, many further experiments clarified what was going on by the time Einstein was working on the mass-energy equivalence).

Which gets me to modern science: there are a lot of things that we want to know, for which the relevant technology does not seem to exist. Nutrition is a great example. What are the long-term health effects of eating a high carbohydrate diet? How can you find out? It’s not practical to run a double-blind study of one group of people eating a high carbohydrate diet and the other eating a low-carbohydrate diet for fifty years. The current approach follows the fundamental principle of science (assume anything necessary in order to publish): it studies people for a few weeks or months, and measures various things assumed to correlate perfectly to good long-term health. That works for publishing, but if you’re more concerned with accuracy to reality than you are with being able to publish (and if you’re reading the study, you have to be), that’s more than a little iffy. Then if you spend any effort digging into the actual specifics, let’s just say that the top ten best reasons to believe these assumptions are all related group-think and the unpleasantness of being in the out-group. (Please actually look into this for yourself; the only way you’ll know what happens if you don’t just take people’s word for something is by not taking their word for it, including mine.)

And the problem with science, at the moment, when it comes to things like long-term nutrition is that the technology to actually study it just isn’t there. (It’s different if you want to study things like acute stimulation of muscle protein synthesis related to protein intake timing or the effects on serum glucose in the six hours following a meal.) And when the technology to do good studies doesn’t exist, all that can exist are bad studies.

This is why we see so much of people turning to anecdotes and wild speculation. Anecdotes and wild speculation are at least as good as bad studies. And when the bad studies tend to cluster (for obvious reasons unrelated to truth) on answers that seem very likely to be wrong, anecdotes and wild speculation are better than bad studies.

That doesn’t mean that anecdotes and wild theories are good. It would be so much better to have good studies. But we can’t have good studies just because we want them, just as people before the microscope couldn’t have cell biology no matter how much they wanted it. The ancient Greeks would have loved to have known about bacteria and viruses, but without microscopes, x-ray crystallography, and PCR, they were never going to find out about them.

As, indeed, they didn’t.