I just came across an interesting article from, of all places, vice, titled, A Bored Chinese Housewife Spent Years Falsifying Russian History on Wikipedia. The tl;dr is that the woman, who had the username Zhemao, pretended to be a scholar of Russian history, which is a thing that Chinese Wikipedia had little of. It apparently began when she tried to understand real scholarly articles but couldn’t, and so started off filling in the missing pieces.

By the time she had written around 10,000 characters, she had gotten attached to it and didn’t want to delete it. Eventually she had a network of sock puppet accounts to boost herself, and had written or contributed to over 200 articles. She got very good at producing scholarly-sounding citations that were extremely difficult to verify. For the most part no one will spend the effort to verify citations to print books that are hard to get, especially on obscure topics that not many people are interested in.

This is, in general, the best way to deceive people—make it easy to believe you and hard to disprove you. The upshot of that is that one should be most careful about things that are easy to believe and hard to disprove—the more that is the case, the more important the trustworthiness of the source is. What can be dangerous is that this sort of thing can bootstrap itself. If you learn of a person through something that’s easy to believe and hard to disprove, and conditionally believe them, the more this goes on the more you will tend to feel like you’ve already trusted them and haven’t been disappointed, so they must be trustworthy.

Incidentally, this applies remarkably well to Science—by which I mean the academic industry of publishing papers. It’s all well and good to say that people’s results can be independently verified—but for the most part no one independently verifies them. When people actually try to, well, there’s a reason that if you Google “reproducibility crisis” you’ll get a lot of results.

Wikipedia gets a lot of criticism because “anyone can edit it,” but that’s not actually all that different from what the industry of scientific paper publication is like. Yes, there is peer review. There’s also peer review on Wikipedia, at least much of the time. In both places, the amount of rigor varies highly. And peer review, in academia, never includes actually running the experiments described in the paper to see if one gets the same results. No one has the money, or even the time, to do that. For some reason a lot of people think that “peer review” in science means “this paper is guaranteed to be good,” when in fact what it means is, “this paper is not guaranteed to be garbage.”