Do you know who invented the electric toaster? If you answered Alan MacMasters, a young Scotsman with high cheekbones and quite a head of hair, you鈥檝e been lied to by Wikipedia.
That infamous prank was pulled by two university students after one of their professors told them not to use Wikipedia for research purposes. 鈥淏ecause you never know,鈥� he warned, 鈥渨ho might set themselves as the inventor of the toaster.鈥� So they did. The hoax, which was repeated as fact in major media outlets like the BBC, lasted听.
There鈥檚 also the time when the assistant to Robert Kennedy, John Seigenthaler Sr., was mentioned on Wikipedia as having been briefly suspected of a direct involvement in the assassinations of both Robert and his brother, John F. Kennedy. It turns out that a Tennessee man had tried to听听by adding the false information on Wikipedia.
No wonder so many professors are reputed for ushering their students away from Wikipedia as a reliable reference.
But Wikipedia, which is written and edited by volunteers and has existed since 2001, is surprisingly accurate. It鈥檚 difficult to answer a question as broad as 鈥渋s Wikipedia reliable?鈥� given the 63 million articles it offers, with roughly one-tenth being in English. Bearing in mind our Office鈥檚 focus on science and my own background in the health sciences, I will put aside other subjects and plunge into some of the studies that have been done on Wikipedia鈥檚 scientific accuracy. It turns out it鈥檚 not an easy thing to measure.
Having your cake and eating it too
The first major study of Wikipedia鈥檚 accuracy on scientific issues was听听in the prestigious听狈补迟耻谤别听academic journal a mere four years after Wikipedia鈥檚 launch. It pitted the budding website against the undefeated champion,听Encyclopedia Britannica.听The take-home message? Wikipedia 鈥渃ame close鈥� to the encyclopedia in terms of being accurate. The editors of听叠谤颈迟补苍苍颈肠补听disputed the findings;听狈补迟耻谤别听pushed back.
I have seen this study cited in nearly every scientific paper analyzing the reliability of Wikipedia, and it is historically important. But Wikipedia has grown tremendously both in terms of quality and quantity since 2005. This is a problem with research into this topic: we are getting snapshots in time that themselves quickly become unreliable. One academic听听that 鈥渢he site鈥檚 volatility diminishes its credibility.鈥� That same volatility also taints the work of researchers trying to study it.
Another problem is sampling. Given the vastness of Wikipedia鈥檚 content, it is impossible for a research team to examine every single published entry even when limiting itself to a specialty. Researchers have to pick a sample and hope it is representative.
罢丑补迟听狈补迟耻谤别听investigation, for example, only looked at 42 articles. When we survey studies done on drug information accuracy on Wikipedia, the results are all over the map. A听听into 100 chemotherapeutic drugs chosen at random reported that nearly two-thirds of these articles were of poor quality, missing basic information about routes of administration and contraindications. But a听听on another set of 100 drugs also chosen at random (though not necessarily used for chemotherapy) came to a different conclusion: accuracy was stellar and completeness was quite good compared to pharmacology textbooks. Are we to conclude that Wikipedia has worsened significantly in the last decade? No. Rather, these studies show the limitations of sampling.
But it鈥檚 not just sampling; there鈥檚 also the question of what researchers want Wikipedia to be. Especially on medical topics, many academics evaluating the reliability of Wikipedia want to have it both ways, it seems. They at once denounce errors of omission听补苍诲听the complexity of the language used. They compare Wikipedia to textbooks aimed at healthcare professionals听补苍诲听they lament the fact that the Wikipedia articles are too dense to be understandable by the average person. I want to eat a whole pie without gaining weight, feeling sick, or seeing a spike in my blood sugar, but I realize that we can鈥檛 always get what we want. A single Wikipedia article cannot be as thorough as a medical textbook yet as approachable as a patient leaflet. (Wikipedia鈥檚 answer to the problem:听.)
The literature appraising Wikipedia articles is also heavily biased toward the English-language version of the encyclopedia (and, to be fair, so were my own searches for these papers). But an English-language Wikipedia page and its French equivalent are not identical. A听听on how the eye disease known as diabetic retinopathy was covered by Wikipedia in 19 different languages showed that quality varied immensely, from 鈥渇air鈥� for the English and German versions to 鈥減oor鈥� for most of the languages consulted. And because Wikipedia is decentralized, sources that have been deemed untrustworthy by English Wikipedians are听听in Wikipedia pages produced in other languages.
Looking at Wikipedia through a microscope is thus really hard, but there are trends that emerge from these imperfect snapshots. Wikipedia, overall, has no business being this good. Given the deterioration so often witnessed on social media platforms (and a friend鈥檚 horrifying, four-year experiment in running听), I originally expected Wikipedia to devolve into a cesspool, with articles vandalized by trolls and offering all sorts of dangerous misinformation. But that鈥檚 not the case. The site is far from perfect, and it will not replace dedicated professional databases for advanced knowledge (nor should it), but as a point of entry into a topic, it is quite reliable overall.
And there are good reasons for that.
Turning students into editors
听Despite Wikipedia鈥檚 free-for-all appearance, not everyone can edit or write an article on the site. Only users registered on Wikipedia can create new articles, which sets up a small barrier to entry aimed at dissuading spur-of-the-moment pranksters. Controversial topics can also be protected. The page for Donald Trump, for example, has a little blue lock icon on it with the letter 鈥淓鈥� inside of it, indicating an 鈥渆xtended confirmed protection.鈥� Only users whose accounts are at a least a month old and who have performed a minimum of 500 edits on Wikipedia are allowed to edit this page. Moreover, dedicated Wikipedians听听(which might indicate that a topic is being hotly debated on social media) and ensure that no lies or vandalism stick around for long.
Since听, health science pages on Wikipedia have additionally benefitted from the organized participation of a particular subset of people: students. Some universities are offering elective courses that train health science students on how to make edits on Wikipedia. These students are then unleashed on important pages that have failed to meet the standards of the online encyclopedia. At the University of Notre Dame Australia, every first-year medical student learns how to edit Wikipedia during their orientation week. Some of these isolated initiatives belong to a larger affinity group called WikiProject Medicine, which played an important role in curbing misinformation on Wikipedia during the early days of the COVID-19 pandemic.
Even on pseudoscientific topics, which invite controversy and strong emotions, the encyclopedia is surprisingly good. This is in part due to the work done by听, ensuring that these pages fall on the side of the evidence and not wishful thinking. (In the interest of full disclosure, their off-Wikipedia organizing has been the subject of听.)
Wikipedia is not perfect; but as has been听, denying students the use of the encyclopedia does not teach them anything. There鈥檚 a reason why so many students gravitate toward it. When听听took a short test similar to their licensing examination, they were asked to write down the topics they had had difficulty with afterwards. They were then randomized to looking those topics up either in a medical textbook, on the website UpToDate (commonly used by doctors), or on Wikipedia. They then took the test again. Who did best? The students who brushed up on their knowledge on UpToDate鈥� or on Wikipedia. The latter was praised by the students for being easy to navigate and for having practical hyperlinks. It was both useful and fast.
After all, the word 鈥渨iki鈥� is Hawaiian for 鈥渜uick.鈥�
Take-home message:
- Studying the accuracy of Wikipedia on health science topics is hard to do, as articles change over time and researchers have to choose a sample of articles to look at and hope they are representative
- Some health science programs in university train their students to edit Wikipedia and revise science articles that could be improved
