{"id":876,"date":"2013-08-07T13:19:25","date_gmt":"2013-08-07T17:19:25","guid":{"rendered":"http:\/\/blogs.law.harvard.edu\/internetmonitor\/?p=876"},"modified":"2013-08-07T13:19:25","modified_gmt":"2013-08-07T17:19:25","slug":"human-computing-and-the-gamification-of-surveillance-analysis","status":"publish","type":"post","link":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/2013\/08\/07\/human-computing-and-the-gamification-of-surveillance-analysis\/","title":{"rendered":"Human Computing and the Gamification of Surveillance Analysis"},"content":{"rendered":"
\"\"<\/a>

Recently unveiled surveillance blimp; courtesy of Raytheon, via Slate<\/a><\/p><\/div>\n

Since the US invasion of Afghanistan in 2001, the American military has worked to create a system of virtually continual real-time drone surveillance of the entire country.\u00a0The system is not entirely automatic, however: in 2010, Marine Corps General James E. Cartwright\u00a0noted that at least 19 analysts<\/a>\u00a0were needed to process video feeds from a single Predator drone. Looking through thousands of hours of collected video and audio recordings is particularly difficult. Cartwright described the work of analysis as sitting for hours watching “Death TV<\/a>,” searching for single or valid targets, an activity he called “a waste of manpower [and] inefficient.” <\/p>\n

To combat this inefficiency, researchers have experimented with building\u00a0smarter cameras<\/a>, capable of\u00a0recognizing and reporting on suspicious activity, but the development of information gathering technology continues to far outpace the ability of computers to make sense of what has been collected. As an alternative, organizations have experimented with crowd sourcing the work of analysis to online volunteers; the US Air Force even asked ESPN for help<\/a> looking through the footage. But what happens when the work becomes play, and the people involved don’t know they’re working as surveillance analysts?<\/p>\n

\"\"

courtesy of NASA<\/p><\/div>\n

In 2000, NASA began outsourcing the tedious job of identifying craters on the moon and Mars by encouraging pubic volunteers, nicknamed “clickworkers,<\/a>” to identify craters in photographs posted online. What would have taken a graduate student a year to accomplish was\u00a0completed in only a week<\/a>.\u00a0In 2006, the state of Texas installed webcams along the Mexico border, streamed the feeds online, and\u00a0encouraged the public to help monitor<\/a>\u00a0them for suspicious activities. One woman watching at 3:00 AM noticed someone signaling a pickup truck on the webcam and notified the police, which led to a high speed chase and the\u00a0seizure of over 400 pounds of marijuana<\/a>. Following the 2011 riots in London, police asked the public to look through thousands hours of CCTV footage and submit their own photos and videos to identify individuals who had participated in looting<\/a>. Recently, a start-up in the UK began offering a service called “Internet Eyes<\/a>,” which connects the country’s ubiquitous CCTVs to the Internet and offers the public rewards for identifying people committing crimes. <\/p>\n

Important to note is that crowdsourced surveillance efforts don’t necessarily lead to results: following the December 2012 shootings at Sandy Hook Elementary School, police asked for assistance from online crowds and were led to the wrong person<\/a>. Following the April 2013 Boston Marathon bombings, a similar call for crowd assistance interfered with investigations<\/a> and led to the wrongful accusation of several innocent people<\/a>. <\/p>\n

While many of these projects use crowdsourced volunteers to handle tasks computers are not able to do, the volunteers participating are typically aware of how their work is being used. These projects attract volunteers willing to give up a little of their time to help with a project they’re interested in seeing succeed or help catch someone suspected of wrongdoing. In contrast, the next generation of surveillance analysis doesn’t require volunteers to know who they’re working for or even that they’re working.<\/p>\n

\"\"In order to tell the difference between human users and computer programs designed to spam websites, computer scientist\u00a0Luis von Ahn\u00a0created\u00a0CAPTCHA<\/a>,\u00a0which presents users with a challenge-response test, usually a simple mathematical operation or an image of obscured text not readable by a computer, which a user must answer or interpret to proceed. Researchers associated with\u00a0Project Gutenburg<\/a>\u00a0realized that CAPTCHA “had unwittingly created a system that was frittering away, in ten-second increments, millions of hours of a most precious resource: human brain cycles.” They created a new system,\u00a0reCAPTCHA<\/a>, that could test for human users with images scanned from books that could not be read by a computer. Humans could decipher these scanned texts and, by entering them in as answers to the test, Project Gutenburg would be able to digitize enormous amounts of text. Since reCAPTCHA was acquired by Google in 2009,\u00a0thousands of Google Books and nearly the entire archive of the\u00a0New York Times<\/em><\/a>\u00a0have been digitized by millions of people who were not aware they were working for the project. In\u00a02012, reCAPTCHA began\u00a0using photographs of house numbers taken<\/a>\u00a0from Google’s\u00a0Street View<\/a>\u00a0project.\u00a0Last month, the ACLU compiled a report that found that police departments across the US were using automatic license plate scanners<\/a> to track and retain the movements of millions of Americans. The “automatic” scanners are often able to read and convert the images of license plates into computer-readable text on their own, but reCAPTCHA has also been used to digitize<\/a> the more difficult images.<\/p>\n

Luis von Ahn noticed how many hours people spent playing Windows Solitaire and devised an online game called “ESP<\/a>” in which two players would be randomly shown a pair of images and asked to guess the word that best described the pair. When both players made the same guess, they would win points.\u00a0Playing the game\u00a0also contributed to building a database<\/a>\u00a0of labels for graphical search engines.\u00a0Without even knowing it, millions of people playing an online game were helping to build surveillance databases and were working for free helping improve the ability for computers to search images.<\/p>\n

\"\"<\/a><\/p>\n

Big gaming companies and other groups are also taking note of the possibilities for “human computation<\/a>” embedded in games. After researchers at the University of Washington led by David Baker<\/a> successfully solved the puzzle of an AIDS-causing virus<\/a> that had stumped scientists for 15 years in only ten days using an online game called Foldit<\/a> in 2012, the gamification of tedious labor has been a popular concept. In early 2013, the\u00a0Internet Response League<\/a>\u00a0launched a plugin that allows\u00a0online gamers to help support disaster response<\/a>\u00a0operations. In\u00a0Word of Warcraft<\/em>, for example, gamers can receive disaster alerts and momentarily interrupt their play to tag images of disaster areas and rank them according to their severity.\u00a0For the past four years, Ubisoft has been developing a new kind of game called\u00a0Watch Dogs<\/em>, set to be released in December 2013. As part of its marketing campaign for\u00a0Watch Dogs<\/em>, Ubisoft launched a website called\u00a0WeareData<\/a>\u00a0that\u00a0gathers and graphs real-world city data<\/a>\u00a0from London, Berlin, and Paris.\u00a0Real-time data<\/a>, including social network updates, the locations of Wi-Fi hotspots, and feeds from CCTV cameras, is streamed onto the site’s 3D city maps. The actual game will also include these streams and is built to connect with players’ Twitter, Flickr, Instagram, Facebook, and other social media accounts to provide seamless integration of these networks with game play.<\/p>\n

\"\"<\/a><\/p>\n

Ubisoft’s marketing website and eventual game highlights our visibility online (something we’re already acutely aware of since the revelation of\u00a0PRISM<\/a>\u00a0and other government data surveillance programs), but also suggests an alternative future of surveillance and analysis than the kind popularized by George Orwell’s invention of Big Brother<\/a>. It may not be long before someone like Luis von Ahn builds systems that rely upon the unwitting assistance of crowds to analyze CCTV feeds looking for criminals or someone like David Baker makes decrypting communications and files a fun game. Future players tagging photos in connected games like\u00a0Watch Dogs<\/em>\u00a0might be helping to\u00a0identify participants in riots<\/a>\u00a0while also collecting data on other players. People posting comments online, taking and tagging pictures for social networks, or simply drawing unlock patterns on their smartphone screens may help sort through the glut of gathered information. The surveillance analysts of the future may not be people wearing clipped on name badges watching hours of Death TV at the Pentagon. The work of watching and reporting may be done by all of us as we go about the everyday routines of digital life or escape for a while with a fun new game.<\/p>\n","protected":false},"excerpt":{"rendered":"

The problem with modern surveillance is that much more is gathered than can be analyzed. Recent trends in human computing and the use of games to perform complex tasks might fix that problem. In the future, we may all be surveillance analysts. Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":5506,"featured_media":929,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[3619,5107,1860],"tags":[],"class_list":["post-876","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-surveillance","category-united-kingdom","category-united-states"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/files\/2013\/08\/wearedata.jpg","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4L9BV-e8","_links":{"self":[{"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/posts\/876","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/users\/5506"}],"replies":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/comments?post=876"}],"version-history":[{"count":26,"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/posts\/876\/revisions"}],"predecessor-version":[{"id":943,"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/posts\/876\/revisions\/943"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/media\/929"}],"wp:attachment":[{"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/media?parent=876"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/categories?post=876"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/internetmonitor\/wp-json\/wp\/v2\/tags?post=876"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}