The well-known myth-busting website has for some time, in the opinion of this writer, appeared to have taken on a decidedly partisan hue. Time and again I have gone to Snopes to see if they had covered common myths propagated by Democrats/liberals—to no avail, but they appeared to be working overtime to catch every single falsehood repeated by the Right. I naturally wondered whether they are simply stocked with liberals who are eager to dispel conservative falsehoods over liberal ones. I also wondered what standards, if any, were in place to minimize the bias that, perhaps, might be unavoidable from anybody with strong political opinions. My observations were of course non-scientific, and there’s no doubt that Snopes has debunked some liberal falsehoods. That said, while I was able to find several exposés of liberal falsehoods on other fact-checking websites, they were starkly missing on Snopes.
It appears that my friends and I weren’t the only ones wondering about such things. Back in December, The Daily Mail reported:
David Mikkelson [co-founder of Snopes] told the Dailymail.com that Snopes does not have a ‘standardized procedure’ for fact-checking ‘since the nature of this material can vary widely.’ He said the process ‘involves multiple stages of editorial oversight, so no output is the result of a single person’s discretion.’
He also said the company has no set requirements for fact-checkers because the variety of the work ‘would be difficult to encompass in any single blanket set of standards.’
‘Accordingly, our editorial staff is drawn from diverse backgrounds; some of them have degrees and/or professional experience in journalism, and some of them don’t,’ he added.
The Daily Mail’s report led Kalev Leetaru, a contributor to Forbes, to contact David Mikkelson about the allegations. Given the lack of corroborating reports and the silence of other news outlets, Leetaru was inclined to wonder whether this was another example of fake news. He writes:
On the surface this looked like a classic case of fake news – a scandalous and highly shareable story, incorporating official-looking materials and sourcing, yet with no other mainstream outlet even mentioning the story. I myself told my colleague I simply did not know what to think. Was this a complete fabrication by a disgruntled target of Snopes or was this really an explosive expose pulling back the curtain on one of the world’s most respected and famous fact checking brands?
Leetaru’s interaction with Mikkelson yielded several items of highly disturbing information. Snopes does not prohibit somebody who has run for or has held political office to serve as a fact-checker. This of course raises serious questions on how such a person could objectively analyze material that is hostile to his or her beliefs. How is an organization’s credibility advanced when there appears to be no standard in place to prevent partisanship? Moreover,
When I asked for comment on the specific detailed criteria Snopes uses to screen its applicants and decide who to hire as a fact checker, surprisingly David demurred, saying only that the site looks for applicants across all fields and skills. He specifically did not provide any detail of any kind regarding the screening process and how Snopes evaluates potential hires. David also did not respond to further emails asking whether, as part of the screening process, Snopes has applicants fact check a set of articles to evaluate their reasoning and research skills and to gain insight into their thinking process.
This was highly unexpected, as I had assumed that a fact checking site as reputable as Snopes would have a detailed written formal evaluation process for new fact checkers that would include having them perform a set of fact checks and include a lengthy set of interview questions designed to assess their ability to identify potential or perceived conflicts of interest and work through potential biases.
One of Snopes’ authors is Kim Lacapria. Writing for Inquisitr, she states:
The scandal envy suggestion seems, at least to many on the left, as a plausible explanation for why a series of allegations seem to be so damning to some and so head-scratching to others. (To put it delicately, diffusion of coverage on … certain news outlets … may also be in part to blame.) And as an openly left-leaning individual myself, I share the befuddlement many liberal pundits and newsmen and women seem to express in working out, even now, how Obama may have “ignored” Benghazi or why we are supposed to be so angry that four Americans were tragically killed in an attack on a diplomatic entity in a dangerous zone — a circumstance that is tragic, but not a surprise.
Lacapria describes the Benghazi issue as “scandal envy,” and wonders what all the fuss is about. She is certainly entitled to be “openly left-leaning,” and she is certainly entitled to her opinions, but when she’s fact-checking political stories, that’s another matter. Would liberals enjoy reading a Snopes piece written by Rush Limbaugh about Clinton’s email scandal? Would they not question Limbaugh’s objectivity? Since Snopes is apparently comfortable with a left-winger fact-checking Republicans, I think it is fair to ask whether any right-wingers are on the Snopes’ payroll fact-checking Democrats. Can a highly partisan person be objective with the facts? I think so, but without standardized procedures to curb bias (see below), readers might feel obligated to do their own investigating—in which case, why go to Snopes?
Snopes not only has a front-end problem, but Mikkelson was allegedly silent on his organization’s quality assessment procedures:
This raises exceptionally grave concerns about the internal workings of Snopes and why it is not more forthcoming about its assessment process. Arguing that because multiple fact checkers might work on an article, reliability is not a concern, is a false argument that shows a concerning lack of understanding about reliability and accuracy. Imagine a team of 50 staunch climate deniers all working collaboratively to debunk a new scientific study showing a clear link between industrial pollution and climate change. The very large team size does not make up for the lack of diversity of opinion. Yet, David provided no comment on how Snopes does or does not explicitly force diversity of opinion in its ad-hoc fact checking teams.
A robust human rating workflow must regularly assess the accuracy and reproducibility of the scores generated by its human raters, even when they work collaboratively together. Typically this means that on a regular basis each fact checker or fact checker team is given the same article to fact check and the results compared across the groups. If one person or group regularly generates different results from the others, this is then evaluated to understand why. Similarly, an individual or group is also periodically given the same or nearly identical story from months prior to see if they give it the same rating as last time – this assesses whether they are consistent in their scoring.
In addition, Mikkelson apparently did not respond to a question on why his staff rarely ever reaches out to the authors of pieces being fact-checked in order to get their side of the story. Leetaru states that it is standard journalistic procedure to reach out to a person you are writing negatively about in order to give that person a chance to respond.
Mikkelson allegedly did not reply to Leetaru’s question on why Snopes doesn’t have a dedicated appeals page for those who contest the accuracy of their conclusions or whether they have any formal written appeals process. Readers are encouraged to read Leetaru’s entire piece. Greater transparency is definitely needed before we place our trust in an organization which claims to be an arbiter of truth.
One might of course argue that their lack of transparency notwithstanding, their reports are nonetheless accurate. Where is the evidence that they’ve been biased? What stories have they gotten wrong? The website yournewswire lists their coverage of Hillary Rodham’s defense of a child rapist, “racist brownies,” and the Democratic Party’s removal of the U.S. Flag on the first day of their last convention as examples of extremely questionable fact-checking by the alleged arbiter of truth. The flag issue is particularly interesting because Snopes couched the issue as a claim that the American flag was banned from the convention. However, Snopes focused on The Daily Caller’s (DC) reporting of the event, to which DC replied that Snopes was clearly lying. Particularly troublesome is the DC’s screenshot of one of Snopes’ pictures allegedly showing flags on Day 1. However, that picture was from a C-SPAN grab of a Day 2 event. Snopes apparently removed that image without explanation, perhaps because DC called them on it.
Until Snopes can provide greater transparency, it appears that we’ll need to fact-check Snopes…or simply ignore it.
