{"id":4369,"date":"2020-09-25T14:51:58","date_gmt":"2020-09-25T18:51:58","guid":{"rendered":"http:\/\/blogs.harvard.edu\/sj\/?p=4369"},"modified":"2020-10-12T17:32:37","modified_gmt":"2020-10-12T21:32:37","slug":"forging-social-proof-networked-turing-test-ai","status":"publish","type":"post","link":"https:\/\/archive.blogs.harvard.edu\/sj\/2020\/09\/25\/forging-social-proof-networked-turing-test-ai\/","title":{"rendered":"Forging Social Proof: the Networked Turing Test Rules the First AI War"},"content":{"rendered":"<p>A few years ago I wrote about how our civilization was <a href=\"https:\/\/blogs.harvard.edu\/sj\/2017\/08\/07\/mental-battlefield-how-we-are-forfeiting-the-zeroth-ai-war\/\" target=\"_blank\" rel=\"noopener\">forfeiting the zeroth AI war<\/a>\u00a0\u2014 <span style=\"font-weight: 400\">allowing individual attention hacks, deployed at scale, to diminish and replace our natural innovation and productivity in every society.\u00a0 We gained efficiency in every area of life, and then let our new wealth and spare time get absorbed by newly-efficient addictive spirals.<\/span><\/p>\n<h3>Exploit culture<\/h3>\n<p><span style=\"font-weight: 400\">This war for attention affects what sort of society we can hope to live in. Channeling so much wealth to attention-hackers, and the networks of crude AI tools and gambling analogs that support them, has strengthened an entire industry of exploiters, allowing a subculture of engineers and dealmakers to flourish.\u00a0 That industry touches on fraud, propaganda, manipulation of elections and regulation, and more, all of which influence what social equilibria are stable.<\/span><\/p>\n<h3>The first real AI war<\/h3>\n<p><span style=\"font-weight: 400\">Now we are facing the first <\/span><i><span style=\"font-weight: 400\">real<\/span><\/i><span style=\"font-weight: 400\"> artificial-intelligence war \u2014 dominated by entities that appear as avatars of independent, intelligent people, but are artificial, scripted, automated.\u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">What is new in this? Earlier low-tech versions of this required no machine learning or programming: they used the veil of pseudonymity to fake authorship, votes, and small-scale consensus.\u00a0 In response, we developed layers of law and regulation around earlier attacks \u2014 fraud, impersonation, and scams are illegal.\u00a0 AI can smoothly scale this to\u00a0<\/span><a href=\"https:\/\/www.buzzfeednews.com\/article\/jsvine\/net-neutrality-fcc-fake-comments-impersonation\"><span style=\"font-weight: 400\">millions of comments<\/span><\/a><span style=\"font-weight: 400\"> on public bills, and to forging microtargeted social proof in millions of smaller group interactions online. And these scaled attacks are often still legal, or lightly penalized and enforced.<\/span><br \/>\n<!--more--><\/p>\n<h3>Scale and speed<\/h3>\n<p>Fabricating an alternate <a href=\"https:\/\/www.printmag.com\/post\/how-italian-fascist-propaganda-went-viral\">fantastical<\/a> reality, exaggerated and unfalsifiable, is a common tactic in socializing broad reallocations of agency and power. <em>{cn}<\/em>\u00a0 This\u00a0<i>realfabrik<\/i> often cannibalizes the knowledge, cooperation, and stability of its surrounding society, in order to build something incompatible with it.<\/p>\n<p>T<span style=\"font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif\">his\u00a0was once limited by the human <\/span>capacity<span style=\"font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif\">\u00a0and commitment of one&#8217;s followers. Now, however, you can build such a movement in your (or someone else&#8217;s) backyard.\u00a0 This is thanks to a confluence of developments in spam, propaganda, AI, and delegated decision-making, including:<\/span><\/p>\n<ul>\n<li>New sources of\u00a0 funding for such tools (weapons to defeat unwanted realities) {c}<\/li>\n<li>Tools that pass the Distributed Turing Test &#8211; networks that pass as human + fantasy that passes as fact in shared discourse space {c}<\/li>\n<li>A cache of unaddressed vulnerabilities in every sense-making intermediary: polls, press, politics, law, regulation, corporations, even scientific + military bodies. (Where is the 0-day database for this?) {c}<\/li>\n<li>A rising cohort of exploiters, willing to use these vulnerabilities<\/li>\n<li>Societies that continue to assume good faith and accurate information, allowing that cohort to implement changes that cement their control {counter-c}<\/li>\n<\/ul>\n<h3>Impact and funding<\/h3>\n<p>How much fungible power and control is at stake here, and how much is being invested into this war already?\u00a0 The battle for attention felt like a $<strong>T<\/strong> of investment for some proportional profit, and an opportunity cost perhaps ten times that.\u00a0 This feels a few magnitudes larger: a war that will dominate how we perceive and respond to a cascade of crises that will not end for some time.\u00a0 In the US alone, the course of the urrent pandemic has been strongly influenced by this war.\u00a0 $10T has been spent on explicit remediation and emergency funding, with minimal oversight, manipulation of supplies, and insider trading of associated stocks.\u00a0 By the end of the pandemic, taking the externalities and aftermath into account, this information war may account for a <em>quad<\/em>\u00a0&#8212; four years of global product &#8212; roughly the net cost of WWII.\u00a0 This will be a mix of the opportunity cost of global disorder, the replacement cost of systems lost, and the positive cost of directable streams of resources (often nominally to counteract those destructive effects).<\/p>\n<p>Much more than this could be gained from <em>generative<\/em> uses of reality distortion, but the result would be harder to centralize, control, and hide.\u00a0 A perverse incentive for short-term leaders to sustain {war, pandemic, cataclysm} is that it keeps people busy with daily crises and provides a catch-all smokescreen for decay or failure.<\/p>\n<h3>Phases of the war: state of play<\/h3>\n<p><strong>Forging identity and reputation<\/strong><\/p>\n<p>The first phase of this war was populating digital spaces with fake identities, organizations, newsrooms, and reports.<\/p>\n<p><strong>DDOSing sensemaking<\/strong><\/p>\n<p>Recently, a US <strong>ecosystem<\/strong> has become the world&#8217;s top source of funding and amplification for <strong>&#8216;<\/strong>grassroots<strong>&#8216;<\/strong> conspiracy theories. This ecosystem includes government spokespeople, the country&#8217;s most influential news network,\u00a0and political marketing organizations, as well as niche media and platforms. These are not limited to cranks or any single subgroup, but span religious and secular, conservative and libertarian and liberal.<\/p>\n<p>This ecosystem completes a positive feedback loop: forging social proof in a way that makes all sectors of society respond to fabricated realities, treating them as real and wasting the time of [real] people in responding to them.\u00a0 In combination with the unequal distribution of tools mentioned above, this can exploit and defeat preexisting norms, regulations, and laws, by subverting the sensemaking needed to implement them.<\/p>\n<h3>What we do next<\/h3>\n<p>I don&#8217;t know what will come next.\u00a0But we must change how we <strong>parse<\/strong> our environment, what is at stake and the forces at play, to have a chance of making sense of it.<\/p>\n<p><strong>~<\/strong> We must acknowledge the global froth of battles to control how we understand our shared present, with combatants who collaborate publicly and privately.\u00a0 We must name this an information war, not as an analogy, but as a Named War with sides and <strong>objectives<\/strong>.<\/p>\n<p><strong>~<\/strong> We must quantify where and how communities lose <strong>shared understanding<\/strong> of what is going on.\u00a0 Where the average person&#8217;s understanding of the state of the world is no longer governed primarily by the <em>independent<\/em> observations of reporters, scientists, educators, ministers, and civil servants. Where even the most reliable of sources can no longer draw from pools of knowledge untainted by manipulation. {c} And where reliable human sources trained in those backgrounds are drowned out by others that look similar, but are fully fabricated or scripted by agents of the war. {c}<\/p>\n<p><strong>\u2248<\/strong> We should recognize that compact networks directly influence the flow of\u00a0quads,\u00a0with a <strong>speed<\/strong> that outstrips legal and political adaptation, and is accelerated by crisis. {c}\u00a0 And for the first time, there are a wide range of automatable 0-day exploits for current laws and norms, affecting most parts of society, alongside tools sufficient for a small group to exploit them at scale.\u00a0{c}<\/p>\n<p><strong>\u224b<\/strong> Finally, we must concede that communities of <strong>exploiters<\/strong> willing to implement those social vulnerabilities at scale are winning major battles in this war &#8212; including keeping it from being declared openly while solidifying into alliances of mutual aid, with structural, financial, and memetic power. {c}<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A few years ago I wrote about how our civilization was forfeiting the zeroth AI war\u00a0\u2014 allowing individual attention hacks, deployed at scale, to diminish and replace our natural innovation and productivity in every society.\u00a0 We gained efficiency in every area of life, and then let our new wealth and spare time get absorbed by [&hellip;]<\/p>\n","protected":false},"author":1202,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[60484,216,1],"tags":[],"class_list":["post-4369","post","type-post","status-publish","format-standard","hentry","category-citation-needed","category-fly-by-wire","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7iVvB-18t","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/posts\/4369","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/users\/1202"}],"replies":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/comments?post=4369"}],"version-history":[{"count":25,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/posts\/4369\/revisions"}],"predecessor-version":[{"id":4408,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/posts\/4369\/revisions\/4408"}],"wp:attachment":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/media?parent=4369"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/categories?post=4369"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/tags?post=4369"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}