{"id":223,"date":"2010-02-04T16:06:30","date_gmt":"2010-02-04T20:06:30","guid":{"rendered":"http:\/\/blogs.law.harvard.edu\/niftyc\/?p=223"},"modified":"2011-07-29T22:52:29","modified_gmt":"2011-07-30T02:52:29","slug":"a-plea-for-the-obscure-parts-of-obvious-systems","status":"publish","type":"post","link":"https:\/\/archive.blogs.harvard.edu\/niftyc\/archives\/223","title":{"rendered":"A Plea for the Obscure Parts of Obvious Systems"},"content":{"rendered":"<p>(or: Doing Better User Studies by Looking Away from the User)<\/p>\n<p>This is a brief summary of my presentation at the <a href=\"http:\/\/www.ischools.org\/iConference10\/2010index\/\">iConference 2010<\/a>.\u00a0 I was invited to address the topic of &#8220;<strong>Ethnographies of Large-Scale Technical Systems<\/strong>.&#8221; \u00a0I did this along with a delightful panel of people:\u00a0<a href=\"http:\/\/www.davidribes.com\/\">David Ribes<\/a>, <a href=\"http:\/\/www.si.umich.edu\/people\/faculty-detail.htm?sid=366\">Steven Jackson<\/a>, and <a href=\"http:\/\/www.ischool.pitt.edu\/people\/star.php\">Susan Leigh Star<\/a>. \u00a0Here goes:<\/p>\n<p><strong>A Plea for the Obscure Parts of Obvious Systems<\/strong><\/p>\n<p>As an ethnographer, my plan of attack has often been to choose a somewhat obscure object, technical system, or site (recently, I&#8217;ve studied several wireless Internet systems). \u00a0It might be a <em>large <\/em>technical system but <strong>it&#8217;s usually obscure<\/strong>, and that used to make me feel safe. \u00a0Most people haven&#8217;t heard of the wireless systems I&#8217;ve written about. \u00a0I get to have my site all to myself. \u00a0If another social researcher suddenly started to talk about my object that would be horrible news&#8211;they might contradict me. \u00a0(Yikes!)<\/p>\n<p>When writing up my work on obscure systems, I feel an obligation to\u00a0thoroughly\u00a0<strong>explain and contextualize them<\/strong> because they are almost certainly unfamiliar to my readers. \u00a0With my writing, to do a good job I probably need to cover:<\/p>\n<ul>\n<li>What is it?<\/li>\n<li>Who uses it?<\/li>\n<li>What do they do with it? Why?<\/li>\n<li>Who made it?<\/li>\n<li>Who maintains it?<\/li>\n<li>Who pays for it?<\/li>\n<li>How is it made? How does it work?<\/li>\n<li>How does its design have consequences?<\/li>\n<li>Why does all this matter?<\/li>\n<\/ul>\n<p>When I read other ethnographies of obscure large technical systems, I find they have the same objectives. \u00a0For instance, on this very panel I sit with Steven Jackson who has written useful stuff about <strong>DWRSIM <\/strong>(which I had never heard of before I read his article), and Leigh Star who has written useful stuff about the <strong>Worm Community System<\/strong> (never heard of it before I read her article). \u00a0It&#8217;s the general theoretical insight I&#8217;m after and so it doesn&#8217;t matter so much what obscure system it is.<\/p>\n<p>Lately I&#8217;ve started to think that it might also be useful to have ethnographies of \u00a0<strong>large-scale technical systems that have millions of users and are far from obscure<\/strong>. \u00a0This is my sentiment as a reader of ethnography as well as as an ethnographer.<\/p>\n<p><a href=\"http:\/\/blogs.law.harvard.edu\/niftyc\/files\/2010\/02\/obvious_systems.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-medium wp-image-226\" src=\"http:\/\/blogs.law.harvard.edu\/niftyc\/files\/2010\/02\/obvious_systems-300x155.png\" alt=\"obvious_systems\" width=\"300\" height=\"155\" srcset=\"https:\/\/archive.blogs.harvard.edu\/niftyc\/files\/2010\/02\/obvious_systems-300x155.png 300w, https:\/\/archive.blogs.harvard.edu\/niftyc\/files\/2010\/02\/obvious_systems-1024x529.png 1024w, https:\/\/archive.blogs.harvard.edu\/niftyc\/files\/2010\/02\/obvious_systems.png 1147w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>I mean systems like <strong>Facebook, World of Warcraft, YouTube, GMail, AIM, Modern Warfare 2,<\/strong> and so on. \u00a0These systems are the everyday informational infrastructure for most people. They are most definitely studied in Information Schools, but when the system being studied is not obscure, for some reason we rarely find it necessary to do all of the contextualization work I listed in the bullets above. \u00a0If someone comes up to you in the hallway of an Information School and says: &#8220;I&#8217;m studying ____.&#8221; and the object in the blank is not obscure (say it&#8217;s twitter), you can assume that <strong>they are doing a user study<\/strong> (who uses twitter? what do people say on twitter? &#8230;).<\/p>\n<p><strong>Some of my Best Friends are Users<\/strong><\/p>\n<p>Some of my best friends are users, as the saying goes. You are a user. \u00a0<strong>I love users.<\/strong> I am a user. \u00a0I love user studies. \u00a0People in Information Schools should do user studies. \u00a0But I worry that there is a big trade-off in framing research on what we might call &#8220;obvious&#8221; systems as being only (or usually) about users. \u00a0If someone says &#8220;I&#8217;m studying YouTube.&#8221; When I read what they are writing I often find that they focus mostly on two of my bullet points:<\/p>\n<ul>\n<li>Who uses it?<\/li>\n<li>What do they do with it? Why?<\/li>\n<\/ul>\n<p>Hopefully they also spend some time on:<\/p>\n<ul>\n<li>Why does all this matter?<\/li>\n<\/ul>\n<p>The turn toward user studies (and somewhat away from design\/producer studies) is generally OK with me but I find it interesting that <strong>in a USER study of an obscure system a lot of contextualization work is still required<\/strong>. \u00a0Whereas with an obvious system you pretty much get a pass on that part. \u00a0(We all know what YouTube is, so why bother with all that?)<\/p>\n<p>People who frame studies of large systems as obviously about users are making <strong>a trade-off between access to the site <\/strong>and the <strong>impact of the findings<\/strong> they&#8217;ll get. \u00a0Of course when we propose a study of &#8220;Facebook&#8221; we probably want to study users because these are the most accessible parts of the system. \u00a0I imagine it would be quite hard to talk your way into Facebook HQ to do a producer study, so researchers interested in the topic don&#8217;t even pause to consider it as an option. \u00a0They also usually don&#8217;t learn very much about how Facebook works or who owns it, who designed it, etc.<\/p>\n<p>But I think this trade-off is being mismanaged. \u00a0Sure, <strong>access is much harder when you want to study powerful corporations who don&#8217;t\u00a0particularly\u00a0want your critical eyes on them<\/strong>. \u00a0But focusing only on the accessible part means that you have the tough job of analyzing the familiar and then selling your insights about data that everyone has. \u00a0This can be really hard.<\/p>\n<p><strong>I am no Erving Goffman<\/strong><\/p>\n<p>To pick a familiar name, Erving Goffman had a great talent of generating amazing insights out of the kind of everyday interactions that everyone experiences. \u00a0<strong>But I am no Erving Goffman.<\/strong> I&#8217;d rather unearth something you didn&#8217;t know and ought to know. \u00a0If you focus on the accessible parts of a system you&#8217;ll have access to the users, but then as an\u00a0ethnographer\u00a0you are forced into coming up with something new and insightful to say about an experience where your reader already feels expert. \u00a0These studies hail the reader by saying: Here is an article where I, a Facebook user, will tell you, a Facebook user, amazing new things about Facebook. \u00a0 That&#8217;s hard! \u00a0So maybe they have no access problem but they have <strong>a big findings problem. <\/strong>A <a href=\"http:\/\/www.zephoria.org\/thoughts\/archives\/2009\/01\/18\/taken_out_of_co.html\">user ethnography of Facebook can be done and done very well<\/a> (remember &#8212; I like users) but we don&#8217;t want the literature to be only focused there.<\/p>\n<p>If instead of a &#8220;pure&#8221; user study a researcher spends a little more time on context &#8212; digging up information that was not widely known &#8212; the resulting work has a lot more chance to gain a wide readership. \u00a0It could still be a user study but with a little more contextual research, or it could be immersion in another site of the system along with or instead of users. There is a small but interesting literature in anthropology about the ethnography of the powerful (sometimes called &#8220;studying up&#8221;). \u00a0People in <strong>Information Schools are usually very comfortable doing ethnography of the powerful, but only certain kinds of power are OK to study<\/strong>. \u00a0It is common to see social researchers studying computer scientists or natural scientists. \u00a0They&#8217;re powerful interlocutors and this changes your ethnography (no one in those groups will be too impressed by your fancy sociology or information studies Ph.D.).<\/p>\n<p>However we are ready to ignore other kinds of power. \u00a0<strong>Popular commercial systems are designed, deployed, and managed by the rich, and this is a kind of power we don&#8217;t usually want to grapple with<\/strong>. \u00a0Access to these sites sounds difficult and in fact it is difficult, but if you can get in there, oh the news you can bring back!<\/p>\n<p>This approach is already out there. \u00a0It just isn&#8217;t common. \u00a0Some of my favorite ethnographies of technological systems are those where the researcher picked an obvious system and then <strong>went to the trouble to fully contextualize it<\/strong> (including at least a little bit about many things like users, producers, design, manufacture, maintenance, finances, public policy, and more). Two quick touchstones: \u00a0Grint and Woolgar&#8217;s <em>The Machine at Work <span style=\"font-style: normal\">takes <\/span><\/em>care to cover production and use of the Personal Computer. \u00a0Boczkowski&#8217;s writing about online journalism required him to talk his way into the newsroom of <em>The New York Times<\/em>.<\/p>\n<p>Let me end by paraphrasing Eugene Webb, Donald Campbell, Richard Schwartz, and Lee Sechrest (from <em>Unobtrusive Measures<\/em>): \u00a0Science should\u00a0opportunistically\u00a0exploit all available points of observation.<\/p>\n<p>So <strong>here&#8217;s to daring<\/strong>. \u00a0 Here&#8217;s to accessing hard-to-reach and unfamiliar sites and populations. \u00a0Here&#8217;s to putting the hardest part of a research project at the beginning (access) and not at the end (findings). \u00a0Here&#8217;s to learning more about users by looking briefly away and studying the many other parts of a sociotechnical system that profoundly affect them.<\/p>\n<p><strong>UPDATE<\/strong>:<\/p>\n<p>In the live, in-person version of these remarks I referenced a pretty obscure book chapter by\u00a0<strong>Lucy Suchman, Daniel Miller, and Don Slater<\/strong>. \u00a0It is titled &#8220;Anthropology&#8221; and it appeared in <em>The Academy and the Internet<\/em> edited by Monroe Price and Helen Nissenbaum (Peter Lang, 2004). \u00a0It says in part that focusing on the user\/producer division in ethnographies is silly.<\/p>\n<p><strong>UPDATE UPDATE:<\/strong><\/p>\n<p>A truly fantastic audience member (who was he?\u00a0<strong><span style=\"color: #ff0000\">FOUND:<\/span><\/strong> <a href=\"http:\/\/www.sei.cmu.edu\/about\/people\/iam.cfm\">Ira Monarch<\/a>) \u00a0pointed out in the Q&amp;A that for some systems the producers are easier to access than the users and this skews the ethnographic research in the opposite direction that I am talking about. \u00a0He mentioned military procurement, where access to developers and businesses working in the area was <strong>relatively easy compared to doing a user study on the people who use weapons in wars<\/strong>. \u00a0(I guess the term of art is &#8220;warfighters.&#8221;)<\/p>\n<p><strong>UPDATE UPDATE UPDATE:<\/strong><\/p>\n<p>Looks like I could have been more upbeat when presenting this in person? \u00a0<a href=\"http:\/\/twitter.com\/bblodgett\/status\/8637139868\">http:\/\/twitter.com\/bblodgett\/status\/8637139868<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>(or: Doing Better User Studies by Looking Away from the User) This is a brief summary of my presentation at the iConference 2010.\u00a0 I was invited to address the topic of &#8220;Ethnographies of Large-Scale Technical Systems.&#8221; \u00a0I did this along with a delightful panel of people:\u00a0David Ribes, Steven Jackson, and Susan Leigh Star. \u00a0Here goes: [&hellip;]<\/p>\n","protected":false},"author":2132,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1321],"tags":[],"class_list":["post-223","post","type-post","status-publish","format-standard","hentry","category-research"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4M7Bm-3B","jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/posts\/223","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/users\/2132"}],"replies":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/comments?post=223"}],"version-history":[{"count":15,"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/posts\/223\/revisions"}],"predecessor-version":[{"id":596,"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/posts\/223\/revisions\/596"}],"wp:attachment":[{"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/media?parent=223"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/categories?post=223"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/niftyc\/wp-json\/wp\/v2\/tags?post=223"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}