https://www.realclearscience.com/articles/2018/11/26/junk_science_has_become_a_profitable_industry_110810.html
Should we believe the headline, “Drinking four cups of coffee daily lowers risk of death”? How about, “Mouthwash May Trigger Diabetes. . .”? Should we really eat more, not less, fat? And what should we make of data that suggest people with spouses live longer?
These sorts of conclusions, from supposedly scientific studies, seem to vary from month to month, leading to ever-shifting “expert” recommendations. However, most of their admonitions are based on flawed research that produces results worthy of daytime TV.
Misleading research is costly to society directly because much of it is supported by the federal government, and indirectly, when it gives rise to unwise, harmful public policy.
Social science studies are notorious offenders. A landmark study in the journal Nature Human Behaviour in August reported the results of efforts to replicate 21 social science studies published in the prestigious journals Nature and Science between 2010 and 2015.
The multi-national team actually “conducted high-powered replications of the 21 experimental social science studies — using sample sizes around five times larger than the original sample sizes” and found that “62% of the replications show an effect in the same direction as the original studies.” One out of the four Nature papers and seven of the seventeen Science papers evaluated did not replicate, a shocking result for two prestigious scientific journals. The authors noted two kinds of flaws in the original studies: false positives and inflated effect sizes.
Science is supposed to be self-correcting. Smart editors. Peer review. Competition from other labs. But when we see that university research claims – published in the crème de la crème of scientific journals, no less — are so often wrong, there must be systematic problems. One of them is outright fraud – “advocacy research” that has methodological flaws or intentionally misinterprets the results.