{"id":183,"date":"2005-08-31T01:00:35","date_gmt":"2005-08-31T05:00:35","guid":{"rendered":"http:\/\/blogs.law.harvard.edu\/metasj\/unreliability\/"},"modified":"2009-04-26T11:35:42","modified_gmt":"2009-04-26T15:35:42","slug":"unreliability","status":"publish","type":"page","link":"https:\/\/archive.blogs.harvard.edu\/sj\/unreliability\/","title":{"rendered":"Unreliability"},"content":{"rendered":"

<\/a><\/p>\n

Most strong statements <\/span>you find in the wilds of the media are about reliability<\/span><\/a> :<\/p>\n

“The best paper shredder on the market”… “the finest Earl in all of
\nChristendom”… “better and longer-lasting than ever!”… “<\/span>More Comprehensive than Any Other English-language Encyclopedia.<\/span>“<\/span><\/p><\/blockquote>\n

Worse, they don’t try very hard to earn your trust; they demand it.\u00a0 It is awfully hard to find statements about un<\/span>reliability (or even adjective-free metrics) which are, funnily enough, more reliable than the first sort.\u00a0 You have to make it up to the industrial-strength copiers, scanners, and shredders before you find manufacturers telling you about the mean time<\/span>(copies, scans, page-shreds) to failure<\/span> for their products.<\/p>\n

This arrogation of trust by media and advertisers finds a parallel in individual lives, with people saying whatever is necessary to claim the trust of others, not respectfully, but as though playing a
\nshort-term game<\/span>… mentioning only positive points, lying where they think it’s safe (after all, everyone is doing it!<\/span>), and avoiding discussion or measurement of unreliability.\u00a0 This is, broadly speaking, a serious moral <\/span>and economic offense.\u00a0 It contributes to inefficiency and ignorance in the community at large, and stalls the development of a profession of measurers (what is the right word?) — people skilled in identifying
\nand evaluating metrics of every sort.<\/p>\n

To take a practical example (practical both in terms of familiar topics and in terms of things people reading this essay can directly do something about) : there are an increasing number of articles and works
\npublished which refer to Wikipedia as an implicitly reliable <\/span>source — often in inappropriate contexts.\u00a0 As its quality improves, Wikipedia seems to be shirking a certain quiet duty<\/span> to be modest; something which was not a problem back when none would have mistaken it for a meticulously edited compilation.<\/p>\n

Example:<\/span> Ann Simmons<\/span>, writing in the LA Times on a matter of British peerage earlier this summer, used the clause “according to Burke’s and Wikipedia,”<\/span> something which should immediately give one pause.\u00a0 It seems that an editor hastily added the following clause, “, an online encyclopedia,”<\/span> recognizing that many audience members would have no idea why this unusual name had been placed at that particular point in the
\nparagraph.\u00a0 The full quote:<\/p>\n

<\/span>
\nAccording to Burke’s and Wikipedia, an online encyclopedia, Frederick
\nsucceeded his father, Robert Capell, the 10th Earl, who died in June.
\n(The late earl was a distant cousin of the 9th Lord Essex.)<\/p>\n

The 11th Earl is a bachelor and has no children. With no other
\napparent successor in sight, Capell is the new heir to the earldom. His
\naristocratic genealogy is documented in the 106th edition of “Burke’s
\nPeerage & Baronetage.”
\n<\/span><\/span>\n<\/div>\n

Please understand me; I will be the first to tell you that you can find<\/a> articles and collections<\/a> on Wikipedia – including many<\/a> on peerage<\/a> and royalty<\/a> – which are among the best<\/span> overviews in the English language; if only you know where to look (and how to check the latest revisions <\/span>in each article’s history<\/span>).\u00a0 But they are overviews and (at their best) reference the most authoritative sources; they should never be used as one.<\/p>\n

The process for checking information added to Burke’s and that for adding information to Wikipedia are vastly dissimilar.\u00a0 The Wikipedia article on the Earl of Essex<\/a>, continued to list no references, two months after the above bit of news drew new attention to the articles on Frederick and Robert Capell; and while the average quality of information in the Earl is excellent, on any given day it might include mistakes or overlooked fabrications.\u00a0\u00a0 [yes, just like any publication; WP likely has fewer such fabrications and errors overall, but the standard deviation for correctness is much higher.]<\/p>\n

It is embarrassing to imagine some newscasster, writer, lawyer, politician, student, professor, or publicist
\nciting a random article from Wikipedia, on peerage or anything else, without somehow verifying that the source article had been carefully\u00a0 researched.\u00a0 So what can be done?\u00a0 Short of the full-fledged drive for moderated or static views <\/span>of the project, that is.\u00a0 What I would like to see is an internal quality review group that issues regular recommendations <\/span>to the rest of the world.\u00a0 At first these recommendations would look like a brief whitelist of the categories and subsubfields <\/span>that are really top-notch and being monitored by a healthy community of respected users.\u00a0 Slowly it would add various hard metrics<\/span> for each of two score top-level categories — spot-check accuracy;\u00a0 vandalism frequency\/longevity; proportion\/longevity of POV and other\u00a0 disputes; rates of new-article creation and editing, and article deletion; &c, &c<\/span>.<\/p>\n

The recommendations could go out to educational, librarian, and research bodies <\/span>— including some of you reading this.\u00a0\u00a0 They\u00a0 would be prominently linked to the sitewide disclaimer[s].\u00a0 The metrics would be available to anyone as feedback, including those working on relevant WikiProjects. What do you think? (Original blog post | A boldfaced tip o’ the cursor to lotsofissues<\/span><\/a>)<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Most strong statements you find in the wilds of the media are about reliability : “The best paper shredder on the market”… “the finest Earl in all of Christendom”… “better and longer-lasting than ever!”… “More Comprehensive than Any Other English-language Encyclopedia.“ Worse, they don’t try very hard to earn your trust; they demand it.\u00a0 It […]<\/p>\n","protected":false},"author":135,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","meta":{"footnotes":""},"class_list":["post-183","page","type-page","status-publish","hentry"],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/P7iVvB-2X","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/pages\/183","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/users\/135"}],"replies":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/comments?post=183"}],"version-history":[{"count":0,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/pages\/183\/revisions"}],"wp:attachment":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/media?parent=183"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}