≡ Menu

How I think Wikipedia works

I have a mental model of how Wikipedia works and behaves. This may not reflect reality, but it is how I, as an end-user, expect Wikipedia to behave. I think these are reasonable expectations regarding things like standards of proof and balance and that if the real Wikipedia differs substantially from these expectations, then we have a problem.

Please let me know if my mental model differs from reality.

First, I assume that we deal with facts, not opinions. So an editor cannot state a personal opinion such as, “Citizen Kane is the greatest movie ever made”, since there is no objective, recognized scale for cinematic greatness.

However, saying, “Citizen Kane topped the list of ‘Greatest Films’ according to a 2002 poll of directors and film critics by Sight & Sound magazine” would be fine. It is a factual statement, albeit a statement about an opinion, but the factual portion of it is verifiable. It is a fact about an opinion and that is OK.

But if I made the statement, “Citizen Kane is the greatest movie ever made” and cited the Sight & Sound article, this would not be proper, since that article does not establish the fact of the greatest movie, but only the fact of a poll that collected opinions on the greatest movie. A fact about the existence of an opinion (or even a polled opinion) does not assert the truth of the opinion.

Similarly, a statement, “Gone with the Wind has been criticized for its long running time” would not be properly cited by merely referencing a source that states its length as 238 minutes. That citation would merely be evidence of its length, not that its length was inordinate. You need a citation for the length being criticized.

Similarly, if another recognized expert stated, “Gone with the Wind was too short and failed to cover the entire Mitchell novel”, then I would expect both opinions to be mentioned, not merely selecting an arbitrary opinion.

I also expect that cited sources have recognized (not merely self-declared) expertise in the area. So, I would find it idiosyncratic if an article on cinema said, “Citizen Kane is the greatest film ever made, according to a fan blog post by Joe Blow, a ophthalmologist in Podunk, Michigan”, since he would be a source cited outside any area of recognized expertise.

I also, as a user, expect Wikipedia to give a balanced view of issues. This does not mean equal time to all fringe opinions. Although I expect there to be multiple views presented on the propriety of the Iraq War, I would not expect that someone who believes that Abraham Lincoln was an alien from the planet Quthbral to have a section in the Lincoln article, even if he could cite a blog post or a photocopied article, or self-published book on the subject. Ditto for Flat Earth Society members, holocaust deniers and those who think the Apollo moon landing was filmed on a Hollywood sound stage.

On the other hand, I don’t expect that every fact requires a citation. For example as a user, I don’t expect to see citations whenever someone says “Mercury is the closest planet to the Sun”. Similarly, I would find it odd if someone removed that assertion for lack of a citation.

However, I would be suspicious if someone writes something in the form, “Mercury is the hottest planet because it is closest to the Sun”. Although the it is well known that Mercury is the closest planet, it does not follow that it is the hottest. In fact, Venus is the hottest planet. It is a subtle form of editorializing, where an editor can inadvertently introduce personal assumptions into an article. I’m assuming Wikipedia editors are on the watch for this kind of thing.

On the other hand, some things clearly logically follow from known facts. If we know that John Brown was buried on January 23rd, 1582, then we should, absent contrary evidence, safely be able to state that his date of death was on or before January 23rd, 1582. I would not expect someone to revert such a statement as being unfounded, speculation, original research, etc. It logically follows based on our knowledge of how the world works.

Does anyone know whether the above statements have any basis in the aspirations or actual practice of Wikipedia editors and admins? Sadly, my recent reading of some articles suggests that these reasonable expectations are routinely flouted and bear little resemblance to reality.

{ 13 comments… add one }
  • SteveL 2009/06/30, 16:15

    wikipedia likes printed facts. Blogs and online docs it values less, because it assumes that printed content has more peer review. It likes citations too. View it as a collection of peer-reviewed documents that have been scribbled over by teenagers and regularly abused by sponsors.

  • Brandon 2009/06/30, 17:36

    Hi Rob. Love the blog (and used to work in your building, coincidentally).

    Wikipedia has an extensively documented (and often followed) editing policy. The specific policy you're probably interested in is the Verifiability policy, although Wikipedia has many policies.

    Your basic premise is correct. Wikipedia likes verified facts. You'll often see admins refer to Wikipedia's strong policy of "No Original Research", which means that regardless of whether you can prove something is true, Wikipedia is not going to accept it as true until it's available externally, preferably in a peer-reviewed, published journal.

    Of course, Wikipedia also has the ignore all rules rule, so I suppose it's all open to interpretation.

  • Anonymous 2009/07/01, 04:12

    You will actually tend to find the opposite. The BDFL & admin/editorial power structure, the accompanying vanity and censorship issues, are in fact deeply incompatible with the principles of open publishing and fair resolution of "NPOV" issues. There is nothing charming or particularly positive about the fact that whoever founded the encyclopaedic project gets to make truth/censorship decisions ad nauseam. This has become apparent time and again as frustration with the admin/editorial resolution of conflicts around contentious issues (MSOOXML, Palestine, scientology, etc ppp) has grown, where vested interests make best use of the internal political mechanisms of a semi-autocratic structure. Refer to the several sites criticising the odd power structure in this supposedly free medium. I shall be looking forward to a truly democratic fork of this project.

  • aristippus303 2009/07/01, 10:26

    Very good post Rob.

    I can imagine there are probably subtle jabs here, but as I rarely consult Wikipedia as anything other than a source of links to authoritative sources, I wouldn't know exactly what particular topics you might be alluding to, although, I can imagine ;-)

    I do agree with the mental model, but the grey area starts to appear with the definition of "fringe".

    I wonder how careful society is to define what is fringe and what is not – as you know (flat earth, planetary behaviour etc etc), many views which have been deemed to have been very "fringe" indeed, have been proven to be factually true.

    One must be open to a certain flexibility about this, but the problem is likely far less pervasive than in bygone years, since there are far more mechanisms to be able to prove and disprove theories, as well as the readiness of some societies to use them.

    That's an interesting area for further discussion.


  • David Gerard 2009/07/01, 13:16

    Your mental model is more or less correct, but leaves out an important detail: this doesn't happen on any particular time scale. There's no compelling obsessive drive to get an article into ideal condition *right now*; such obsession leads to people being kicked off the site, as hAl nearly was.

    The exception is biographies of living people, where we really don't have the luxury of eventualism, and things have to be pretty much ideal at any given moment. This has led to people also outraged that Wikipedia doesn't work like they thought it did; examples are regular features in the news.

    In this particular case, publicity on Groklaw has led to the right thing happening for the articles to right themselves in the usual manner, i.e. lots of interest and lots of editors flocking to them. This is then expected to get things worked out eventually.

    One thing that hypothetical (I'm sure) shills don't seem to understand is that people know what they're doing and are watching closely. That they choose not to act is because the process is as described above, i.e. get a pile of editors and it'll all work out in good time. It's not because they don't know damn well what's going on or don't have anvils hovering above in orbit waiting to drop on the perpetrators should it regrettably become necessary.

    In summary:

    * you don't get to expect a perfect article at any given moment. Caveat lector, we're an unreliable source and tell you so, etc.
    * the correct thing is for lots of people to work on the articles in question. This is happening.
    * the powers that be aren't stupid, but they don't intervene in every such conflict. Because it all works better if they don't.

  • Rob 2009/07/01, 17:17

    Thanks for the thoughts. I wasn't so interested in published rules as in how Wikipedia really works.

    For example, take how democracy works. It is one person one vote, but most people don't vote. Those with the strongest interests in an issue spend money on campaigns and tend to vote in larger numbers. This lends a certain quality to democracy.

    Free market economics is another way we determine outcomes, by weighing capital rather than counting votes. It lends a certain quality to capitalism.

    Newspapers and traditional encyclopedias make content decisions via editorial boards.

    Journals do it via peer review.

    These are all mechanisms which we have constructed that take many views as input and then output a single view.

    The question is how does Wikipedia work? How does it decide? It is not voting, since there is no limit to how many changes a single person can make. They can "vote" as many times as they have the time and inclination. Similarly, it is not a vote by capital, or by expertise.

    So having rules is a fine thing. But it isn't clear to me that the written rules are relevant, since the decision-making process on Wikipedia does not appear to

    @Gareth, in terms of fringe, I accept that an encyclopedia will contain false statements, but I'd want them to be false statements that are generally believed by experts to be true at the time. For example, if we had Wikipedia in the mid 17h century I'd expect it to describe Ptolemaic cosmology and relegate heliocentrism to a fringe theory. An encyclopedia is not where you determine truth. It is where you record what the experts (or at least those whom the writers acknowledge as experts) believed to be true. It records the consensus opinio where there is one, or reports the contending views otherwise.

    And there's the rub. In traditional editing, the determination of expertise is made by other experts, either via peer review, certification, or other mechanisms. There is also reputation involved since authors are rarely anonymous. But Wikipedia gives us anonymous writers with unknown motivations, which in some cases are highly suspected of being commercially motivated. So the risk is we do not achieve the "wisdom of crowds" but end up with something more like "the wisdom of zealots".

    It is a curious model, but I do have concerns as noted. I like the crowd-sourcing aspect of it, the ability of anyone to contribute. Clearly these are things which have lead to its success. But I'm not sure the anonymity helps here.

    Compare to public financing in the US. We have a democracy, and it is put at risk by the large amount of money that is involved in campaigns and the motivations of those who contribute the most money. But this anti-democratic force is kept in check by campaign finance law, which limits individual contributions and requires disclosure of contribution by name, by amount and by employer. This transparency helps keep the system from turning into a total mess. I wonder whether Wikipedia would benefit from increased transparency?

  • Anonymous 2009/07/01, 21:15

    Wikipedia is full of hypocrits. The editors accusing others of sockpuppetry are usually engaging in the use of sockpuppets themselves. Some of the juvenile behaviour, written by some loner teenager Buckethead fanatic in Germany no doubt, reflects badly on a project that pretends to be an "encyclopaedia".

  • phil 2009/07/02, 04:38

    "But Wikipedia gives us anonymous writers with unknown motivations, which in some cases are highly suspected of being commercially motivated. So the risk is we do not achieve the "wisdom of crowds" but end up with something more like "the wisdom of zealots"."

    This is often claimed with regards to anonymous editing, "wikipedia is too democratic/open" or whatever. What actually happens is that anonymous "opinions" are deleted swiftly by the editors. The editors, in turn, are people who tend to be extremely interested in the pages/categories they happen to be watching – for whatever reason. It's a bit like the Catholic church (minus the paedo issues) – once you geek it up enough around the site, you get control and vested trust.

    The wisdom of zealots effect thus occurs at the level of editors and the BDFL.

  • Rob 2009/07/02, 09:19

    I wouldn't say Wikipedia is too open or too democratic. I'd say that it is not open enough, nor democratic enough. A democracy does not allow anonymous people to show up and vote, and to do so as many times as they wish. A democracy, in the US form at least, has its voting rolls open for inspection. I'd be quite happy if Wikipedia were more open and more democratic.

    Who contributes to Wikipedia today who would not do so if their real identity were known? Surely, a lot of nefarious activity would quickly end. But are their legitimate uses that would be hurt? Are there people commenting on, say, civil rights issues in Tibet that would be unable to comment if their names were known? But that would sound like they were editorializing, not providing neutral POV, etc. Is it to hide your hobbies and other proclivities from co-workers and neighbors? I suppose that is a legitimate concern.

  • NotZed 2009/07/03, 09:13

    That's how it's 'supposed to work', but in reality wisdom of zealots seems to sum it up pretty well.

    You only have to look at any `controversial' topic to see that. Well, any topic that anyone thinks is controversial anyway.

    And if you really believe that 'over time' things are worked out, just look at the article on Jesus (and it's many many archived talk pages) and compare with known historical information. That one article did wikipedia in for me as any sort of valid objective source of information; I do still use it but sparingly.

    All the rules seem to be (ab)used to enforce a given editor's agenda. e.g. things are dismissed as NPOV or 'original research' when they disagree with whomever it is who has taken it upon him or herself to maintain the given topic, even if the facts stand alone. Those zealots again.

    … and finally … coming from outside of the USA it is obvious there is also a cultural bias evident in much of the editing decisions. The authority given to external information sources for example.

  • Joe 2009/07/04, 15:17

    The policies are pretty much as you describe. An important policy you did not mention is the injunction to ASSUME GOOD FAITH. This requires you to assume (or at least behave as if you believe) other parties are trying to help even when you do not agree with them. Give them plenty of opportunity to show they are shills.

    The most important mechanism for implementing these policies is the discussion page attached to each encyclopedia page. Anything at all controversial should be discussed on the talk page first. Excellent articles on some of the most contentious subjects have been arrived at by editors with different opinions working together on the talk page to find the wording which fairly represents all sides of the subject.
    If the dispute cannot be resolved there and you need to call for help from other editors then you will be judged on how you behaved on the talk page. Wikipedia's enforcement system is heavily biased towards judging how people behaved; Were they abusive? Did they try to bully others instead of collaborating? Dispute resolution tries really hard not to make judgements on content.

  • Josh Cogliati 2009/07/05, 10:20

    In normal articles (say Pumping Lemma for Regular languages) Wikipedia tends to work reasonably close to how it should. People are reasonable and verifiability and such work about the way they should.

    In controversial issues, Wikipedia tends to become insane.

    "On the other hand, some things clearly logically follow from known facts. If we know that John Brown was buried on January 23rd, 1582, then we should, absent contrary evidence, safely be able to state that his date of death was on or before January 23rd, 1582. I would not expect someone to revert such a statement as being unfounded, speculation, original research, etc. It logically follows based on our knowledge of how the world works."

    According to the Wikipedia rules on Original Research and Verifiability as I currently understand them, this is considered original research. I would like this rule changed, but I have not been successful in doing this ( 1 ). If you have specific examples, I will take a look and try and get this changed.

    Note that nine times out of ten, if something you disagree with is happening in wikipedia, here is what you do. Go to the talk/discussion page, put a link to the edit in question and a summary or the whole edit, and what you think should have happened. Then, see if a discussion happens. If no discussion happens in a week, change the article to how you want it, and put in the edit summary something like see talk. If a discussion happens then you can hopefully understand better why the edit did happen.

    I hope this helps. Thank you.

  • Rob 2009/07/10, 13:27

    This is getting more interesting. According to one recent academic study, Australian Wikipedia editors are "egocentric, disagreeable, socially awkward and closed to new ideas".

    Does this remind you of anyone?

Leave a Comment

Next post:

Previous post: