Sparks’ Notes Version (A Summary of This)
`
Most science is false. Why?
`
1. Statistics. Even if your test is 99% accurate, if there are only 100 possibilities to test, the chances that your test result will be correct is less than 50%. This is because there are so many possible wrong answers that several wrong answers are certain to incorrectly test positive, so your positive test result is more likely to be wrong (a false-positive) than to be right. For example, if testing 20,000 genes for a possible link to Alzheimer’s and only one is a true link, and your test is 99% accurate, you will likely get 200 false positives in addition to the 1 true positive. Unfortunately, a “good” level of accuracy on a test is lower than 99%, more like 95%, so statistically speaking, most scientific research results today are false. See John P Ioannidis’s paper Why Most Published Research is False.
`
2. Researcher Bias. Research has shown that when a scientist expects a certain result, he’s more likely to get the result he expects. Part of this is accident, such as mathematical errors that he’s less likely to discover because he got the expected result. Probably a bigger part of it is outright fraud. Research has shown the majority of researchers admit to committing various degrees of fraud. This makes it even less likely that a positive result is a true positive. It’s also extremely easy to manufacture a certain result without committing massive, blatant fraud by simply using a different data analysis method or selecting a small group out of a larger population for analysis, making it harder for honest scientists to detect the false-positive.
`
3. Publisher Bias. Publishers are in competition with each other and so they want flashy results. Therefore, they are extremely unlikely to publish boring results such as “X is not the cause of Y.” Therefore, among 20 studies on the subject, the one that falsely says X *does* cause Y is highly likely to get published while the 19 that say X doesn’t cause Y are extremely unlikely to be published.
`
4. Cultural Bias. Both the broader popular culture and the scientists’ own micro-culture affect the results. If a research result was considered exciting, scientists in that field are less likely to want to prove it wrong and research proving it wrong is more likely to get intentionally buried. Furthermore, if something is considered ridiculous or unacceptable in the scientific or popular culture, scientists are less likely to pursue or publish research in that area because it may mean the ends of their careers. (Example: Dr. Semmelweis suggested a very unpopular theory–that doctors can carry disease on their unwashed hands–and studied it and ultimately recommended hand washing to prevent the spread of disease. It was an unpopular idea, and so he was ostracized in the scientific community and ultimately lost everything.)
`
5. Sucky Peer-Review. Research is peer-reviewed prior to publication. In theory, this means other highly educated researchers in the same field comb the article for flaws and make recommendations for changes to improve the article or correct errors. In reality, research has shown that peer review fails to do its job. In one study, even when told they were part of a study and that they might find something “off” about an article submitted for publication, peer reviewers on average only caught one of eight *major* errors intentionally added to the paper and only 30% recommended rejecting the paper for publication. Peer review also has the tendency to reject unpopular ideas, making peer review switch from useless to actively harmful in a shockingly high proportion of cases.
`
6. Failure of Self-Correction. In theory, science is self-correcting, meaning that over time, enough evidence will accumulate to replace a wrong theory with a right one. But there are many cases in human history where the correct theory was replaced with a wrong theory, even for over 1,000 years, before being replaced again with the right theory–for example, vitamin C deficiency as the cause of scurvy and geocentric theory (the belief that the sun revolves around the earth). How many correct theories have been replaced with a wrong theory and we just don’t know it yet?
`
7. When Theory Becomes “Fact.” Often, new theories are proposed and bad science published at such a great speed today that a false theory quickly outpaces the natural self-correction of science, and new theories, subspecialties, careers, and grants spring up based on this bad science before it has a chance to naturally self-correct. At that point, a powerful barrier to self-correction has arisen. The old theory is treated like fact, and research that is unpopular due to popular or scientific cultural bias is rejected in the peer review process, while biased research that aligns with cultural bias is accepted in the peer review process. Sadly, one study found that bad cancer research that could not be reproduced was cited hundreds of times more frequently than was good, reproducible research in part because it had spawned new branches of research, along with their associated careers, grants, and prestige.
`
8. Peer Pressure. Publishing something critical to a theory that is the foundation of your colleagues’/mentors’ careers is not going to endear you to them. If your colleagues/mentors don’t like you, that can interfere with your ability to get a job. Thus, the micro-culture of your scientific field and, more specifically, peer pressure prevents criticism of widely accepted ideas.
`
9. Careerism and Opportunism. The present culture is such that scientists are lauded as heroes and science is held up as the only legitimate basis for policy making. Once upon a time, scientists were generally poorly-reimbursed for their efforts and so the profession attracted those earnestly interested in genuine scientific advancement. When suddenly given incredible influence, as science has today, any discipline will become flooded with opportunists and charlatans.
`
10. The Religion of Science. The popular culture treats science as though it is the greatest truth and its practitioners as though they are infallible. They treat science as the greatest aim and scientists as the best advisors in all areas of life. Some refer to this as “scientism” or “The Cult of Science.” At best, it encourages a love of science without teaching adherents to distinguish between good and bad science. At worst, it actively fights against unpopular theories, impeding the progress of science.
`
A final note…
`
Proof by Replication/Reproducibility. Theoretically, if a certain result can be replicated/reproduced (i.e., other scientists running the same experiment get the same result), it’s more likely to be true. Groups that repeat published experiments have found at least 65% were not reproducible and many of the remainder were less effective than the original results showed them to be in social sciences, and at least 75% of drug research was false. Another study of cancer research found 89% could not be reproduced.
Schaabling Shire Shoppe 