{"id":1003,"date":"2014-06-26T11:41:46","date_gmt":"2014-06-26T15:41:46","guid":{"rendered":"http:\/\/blogs.law.harvard.edu\/niftyc\/?p=1003"},"modified":"2014-06-26T16:48:46","modified_gmt":"2014-06-26T20:48:46","slug":"1003","status":"publish","type":"post","link":"https:\/\/archive.blogs.harvard.edu\/niftyc\/archives\/1003","title":{"rendered":"Corrupt Personalization"},"content":{"rendered":"

(“And also Bud Light<\/em>.”)<\/p>\n

In my last two posts\u00a0I’ve been writing about my attempt to convince a group of sophomores with no background in my field<\/a>\u00a0that there has been a shift to\u00a0the algorithmic allocation of attention<\/a>\u00a0— and\u00a0that this is important. In this post I’ll respond to a student question. My favorite: “Sandvig says that algorithms are dangerous<\/strong>, but what are the the most serious repercussions that he envisions?” What is the coming social media apocalypse\u00a0we should be worried about?<\/p>\n

\"google<\/a><\/p>\n

This is an important question because people who study this stuff are NOT\u00a0as interested <\/strong>in this student question\u00a0as they should be. Frankly, we are specialists who study media and computers and things — therefore we care about how algorithms allocate attention among cultural products almost\u00a0for its own sake.\u00a0<\/em>Because this is the central thing that we study, we don’t spend a lot of time justifying it.<\/p>\n

And our field’s most common response to the query “what are the dangers?” often lacks\u00a0the required\u00a0sense of danger<\/strong><\/em>.\u00a0The most frequent\u00a0response is: “extensive\u00a0personalization is bad for democracy.” (a.k.a. Pariser’s “filter bubble<\/a>,” Sunstein’s<\/a>\u00a0“egocentric” Internet, and so on). This framing lacks a certain house-on-fire urgency, doesn’t it?<\/p>\n

(sarcastic tone:) “Oh, no! I’m getting to watch, hear, and read exactly what I want<\/strong>. Help me! Somebody do something!”<\/p><\/blockquote>\n

Sometimes\u00a0(as Hindman <\/a>points out) the contention is the opposite, that Internet-based concentration is bad for democracy. \u00a0But remember that I’m\u00a0not speaking to political science majors here. The average person may not be as moved by an abstract, long-term peril to democracy<\/strong> as the average political science professor. As David Weinberger<\/a> once said after I warned\u00a0about the increasing reliance on recommendation algorithms, “So what?<\/strong>” Personalization sounds like a good thing.<\/p>\n

As a side note, the second most frequent response I see is that algorithms are now everywhere<\/a>. And they work differently than what came before. This also lacks a required sense of danger! Yes, they’re everywhere, but if they are a good<\/em> thing<\/strong>…<\/p>\n

So I really like this question “what are the the most serious repercussions?” because I think there are<\/em> some elements of the shift to attention-sorting algorithms\u00a0that are genuinely “dangerous<\/strong>.” I can think of at least two, probably more, and they don’t get enough attention. In the rest of this post I’ll spell out the first one which I’ll call “corrupt personalization<\/strong>.”<\/p>\n

Here we go.<\/p>\n

Common-sense reasoning about algorithms and culture tells us that the purveyors of personalized content have the same interests we do. That is, if Netflix started recommending only\u00a0movies we hate\u00a0or Google started returning only useless search results we would stop using them. However: Common sense is wrong in this case<\/strong>. Our interests are often not the same as\u00a0the providers of these selection algorithms. \u00a0As in my last post, let’s work through a few concrete examples to make the case.<\/p>\n

In this post I’ll use Facebook<\/strong> examples, but the general problem of corrupt personalization is present on all of our media platforms in wide use<\/strong> that employ the algorithmic selection of content.<\/p>\n

(1) Facebook “Like” Recycling<\/strong><\/p>\n

\"Screen<\/p>\n

(Image from ReadWriteWeb.)<\/p>\n

On Facebook, in addition to advertisements along the side of the interface, perhaps you’ve noticed “featured,” “sponsored,” or “suggested” stories that appear inside your news feed, intermingled with status updates from your friends. It could be argued that this is not in your interest<\/strong>\u00a0as a user (did you ever say, “gee, I’d like ads to look just like messages from my friends”?), but I have bigger fish to fry.<\/p>\n

Many ads on Facebook resemble status updates in that there can be messages endorsing the ads with “likes.”<\/strong> For instance, here is an older screenshot from ReadWriteWeb:<\/p>\n

\"pages<\/a><\/p>\n

Another example: a “suggested” post was mixed into my news feed just this morning. recommending World Cup coverage on Facebook itself. It’s a Facebook ad for Facebook, in other words. \u00a0It had this intriguing addendum<\/strong>:<\/p>\n

\"CENSORED<\/a><\/p>\n

So, wait… I have hundreds of friends and eleven of them “like” Facebook<\/strong>? \u00a0Did they go to http:\/\/www.facebook.com<\/a> and click on a button like this:<\/p>\n

\"Facebook<\/a><\/p>\n

But facebook.com doesn’t even have a “Like” button! \u00a0Did they go to Facebook’s own Facebook page<\/strong> (yes, there is one<\/a>)\u00a0and click “Like”? I know these people and that seems unlikely.\u00a0And does Nicolala really like<\/em> <\/strong>Walmart? Hmmm…<\/p>\n

What does this “like” statement mean? Welcome to the strange world of “like” recycling<\/strong>. Facebook has defined “like” in ways that depart from English usage. \u00a0For instance, in the past Facebook has determined that:<\/p>\n

    \n
  1. Anyone who clicks on a “like” button is considered to have “liked” all future content from that source.<\/strong> So if you\u00a0clicked a “like” button because someone shared a “Fashion Don’t” from Vice magazine, you may be surprised when your dad\u00a0logs into Facebook three years later and is shown a current\u00a0sponsored story from Vice.com like “Happy Masturbation Month!” or “How to Make it in Porn” with the endorsement that you like it. (Vice.com example is from Craig Condon<\/a>\u00a0[NSFW].)<\/li>\n
  2. Anyone who “likes” a comment on a shared link is considered to “like” wherever that link points to. \u00a0<\/strong>a.k.a.\u00a0“‘liking a share.” So if you see\u00a0a (real) FB status update from a (real) friend and it says: “Yuck! The McLobster is a disgusting product idea!” and your (real) friend include a (real) link like this one<\/span><\/a>\u00a0— that means if you clicked “like” your friends may see McDonald’s ads in the future that include the phrase “(Your Name) likes McDonalds.” (This example is from ReadWriteWeb.)<\/li>\n<\/ol>\n

    \"fauxLike_mcdonalds\"<\/a><\/p>\n

    This has led to some interesting results, like dead people “liking” current news stories<\/a>\u00a0on Facebook.<\/p>\n

    There is already controversy about advertiser “like” inflation, “like” spam, and fake “likes,” — and these things may be a problem too, but that’s not what we are talking about here. \u00a0In the examples above the system is working as Facebook designed it to<\/strong>. A further caveat: note that\u00a0the definition of “like” in Facebook’s software changes periodically and when they are sued. Facebook now has an opt-out setting<\/a> for the above two “features.”<\/p>\n

    But these incendiary examples are exceptional fiascoes — on the whole the system probably works well. You likely\u00a0didn’t know that your “like” clicks are\u00a0merrily producing ads on your friends pages and in your name<\/strong> because you cannot see them. \u00a0These “stories” do not appear on your news feed and cannot be individually deleted.<\/p>\n

    Unlike the examples from\u00a0my last post<\/a>\u00a0you can’t quickly\u00a0reproduce these results with certainty on your own account. Still, if you want to try, make a new Facebook account under a fake name (warning! dangerous!<\/a>) and friend\u00a0your real account. Then use the new account to watch your status updates.<\/p>\n

    Why would Facebook do this? Obviously it is a controversial practice that is not going to be popular with users. Yet Facebook’s business model is to produce attention for advertisers, not to help you — silly rabbit<\/strong>. So they must have felt\u00a0that using your reputation to produce more ad traffic from your friends was worth the risk of irritating you. Or perhaps they thought that the practice could be successfully hidden from users — that strategy has mostly worked!<\/p>\n

    In sum this is a personalization scheme that does not serve your goals, it serves Facebook’s goals at your expense.<\/p>\n

    (2) “Organic”\u00a0Content<\/strong><\/p>\n

    This second group of examples concerns content that we consider to be “not advertising,” a.k.a. “organic” content.<\/strong> Funnily enough, algorithmic culture has produced this\u00a0new\u00a0use of the word “organic” — but has also made the boundary between “advertising” and “not advertising” very blurry.<\/p>\n

    \"funny-organic-food-ad\"<\/a><\/p>\n

     <\/p>\n

    The general problem is that there are many ways in which algorithms act as mixing valves between things that can be easily valued with money <\/strong>(like ads) and things that can’t<\/strong>. And this kind of mixing is a normative<\/em> problem (what should we do) and not a technical<\/em> problem (how do we do it).<\/p>\n

    For instance, for years Facebook has encouraged nonprofits, community-based organizations, student clubs, other groups, and really anyone to host content on facebook.com. \u00a0If an organization creates a Facebook page for itself, the managers can update the page as though it were a profile.<\/p>\n

    Most page managers expect that people who “like” that page get to see the updates<\/strong>… which was true until January of this year. At that time Facebook modified its algorithm so that text updates from organizations were not widely shared<\/a>. This is interesting for our purposes because Facebook clearly states that it wants page operators to run Facebook ad campaigns<\/strong>, and not to count on getting traffic from “organic” status updates, as it will no longer distribute as many of them.<\/p>\n

    This change likely has a very differential effect on, say, Nike<\/a>‘s Facebook page, a small local business<\/a>‘s Facebook page, Greenpeace International<\/a>‘s Facebook page, and a small local church congregation<\/a>‘s Facebook page.\u00a0If you start a Facebook page for a school club, you might be surprised that you are spending your labor writing status updates that are never shown to anyone. Maybe you should buy an ad.\u00a0<\/strong>Here’s an analytic for a page I manage:<\/p>\n

    \"this<\/a><\/p>\n

     <\/p>\n

    The impact isn’t just about size — at some level businesses might expect to have to insert themselves into conversations via persuasive advertising that they pay for, but it is not as clear that people expect Facebook to work this way for their local church or other domains of their lives. It’s as if on Facebook, people were using\u00a0the yellow pages but they thought they were using the white pages. \u00a0And also there are no white pages<\/strong>.<\/p>\n

    (Oh, wait. No one knows what yellow pages and white pages are anymore<\/a>. Scratch that reference, then.)<\/p>\n

    No need to stop here, in the future perhaps Facebook can monetize my family relationships. It could suggest that if I really want anyone to know about the birth of my child, or I really want my “insightful” status updates to reach anyone<\/strong>, I should\u00a0turn to Facebook advertising.<\/p>\n

    Let me also emphasize that this mixing problem extends to the content<\/em> of our personal social media conversations as well. A few months back, I posted a Facebook status update that I thought was humorous. I shared a link highlighting the hilarious product reviews<\/a> for the Bic “Cristal For Her<\/strong>” ballpoint pen on Amazon. It’s a pen designed just for women.<\/p>\n

    \"bic<\/a><\/p>\n

    The funny thing is that I happened to look at a friend of mine’s Facebook feed over their shoulder, and my status update didn’t go away<\/strong>. It remained, pegged at the top of my friend’s news feed, for as long as 14 days in one instance. What great exposure for my humor, right? But it did seem a little odd… I queried my other friends on Facebook and some confirmed that the post was also pegged at the top of their news feed.<\/p>\n

    I was unknowingly participating in another Facebook program that converts organic status updates into ads<\/a>. It does this by changing their order in the news feed and adding the text “Sponsored” in light gray, which is very hard to see. Otherwise at least some updates are not changed. I suspect Facebook’s algorithm thought I was advertising Amazon<\/strong> (since that’s where the link pointed), but I am not sure.<\/p>\n

    This is similar to Twitter’s “Promoted Tweets” but there is one big difference. \u00a0In the Facebook case\u00a0the advertiser promotes content — my content — that they did not write<\/strong>. In effect Facebook is re-ordering your conversations with your friends and family on the basis of whether or not someone mentioned\u00a0Coke, Levi’s, and Anheuser Busch (confirmed advertisers\u00a0in the program).<\/p>\n

    Sounds like a great personal social media strategy<\/strong> there–if you really want people to know about your forthcoming wedding, maybe just drop a few names? Luckily the algorithms aren’t too clever about this yet so you can mix up the word order for humorous effect.<\/p>\n

    (Facebook status update:) “I am so delighted to be engaged to this wonderful woman that I am sitting here in my\u00a0Michelob drinking a Docker’s Khaki Collection. And also Coke.”<\/p><\/blockquote>\n

    Be sure to use links. I find the interesting thing about this mixing of the commercial and non-commercial to be that it\u00a0sounds to my ears like some sort of corny, unrealistic science fiction scenario<\/strong> and yet with the current Facebook platform I believe the above example would work. We are living in the future.<\/p>\n

    So to recap, if Nike makes a Facebook page and posts status updates to it, that’s “organic” content because they did not pay Facebook to distribute it. Although\u00a0any rational human being would see it as an ad. If my school group does the same thing, that’s also organic content, but they are encouraged to buy distribution — which would make it inorganic<\/strong>. If I post a status update or click “like” in reaction to something that happens in my life and that happens to involve a commercial product, my action\u00a0starts out as organic, but then it becomes inorganic<\/strong> (paid for) because a company can buy my words and likes and show them to other people without telling me. Got it? This paragraph feels like we are rethinking\u00a0CHEM 402<\/a>.<\/p>\n

    The upshot is that control of the content selection algorithm is used by Facebook to get people to pay for things they wouldn’t expect to pay for<\/strong>, and to show people personalized things that they don’t think are paid for. But these things were in fact paid for. \u00a0In sum this is again a scheme that does not serve your goals, it serves Facebook’s goals at your expense.<\/p>\n

    The Danger: Corrupt Personalization<\/strong><\/p>\n

    With these concrete examples behind us, I can now more clearly answer this student question. What are the most serious repercussions<\/strong> of the algorithmic allocation of attention?<\/p>\n

    I’ll call this first repercussion “corrupt personalization<\/strong>” after C. Edwin Baker<\/a>. (Baker, a distinguished legal philosopher, coined the phrase “corrupt segmentation” in 1998 as\u00a0an extension of the theories of philosopher\u00a0J\u00fcrgen Habermas<\/a>.)<\/p>\n

    Here’s how it works: You have legitimate interests that we’ll call “authentic.”<\/strong> These interests arise from your values, your community, your work, your family, how you spend your time, and so on. A good example might be that as a person who is enrolled in college\u00a0you might identify with the\u00a0category “student,” among your many other affiliations. As a student, you might be authentically interested in an upcoming tuition increase<\/a> or, more broadly, about\u00a0the contention that “there are powerful forces at work in our society that are actively hostile to the college ideal<\/a>.”<\/p>\n

    However, you might also be authentically interested in the fact that your cousin is getting married. Or in pictures of kittens<\/strong>.<\/p>\n

    \"Grumpy-Cat-meme-610x405\"<\/a><\/p>\n

    Corrupt personalization is the process by which your attention is drawn to interests that are not your own<\/strong>. This is a little tricky because it is impossible to clearly define an “authentic” interest. However, let’s put that off for the moment.<\/p>\n

    In the prior examples we saw\u00a0some (I hope) obvious places where my interests diverged from\u00a0that of algorithmic social\u00a0media systems<\/strong>. Highlights for me were:<\/p>\n