Protocols and Privacy

This week I read “The Long Tail” by Chris Anderson, “To do with the price of fish” in The Economist, “Shared,Shared, Collaborative and On Demand: The New Digital Economy” by Aaron Smith, “$1 Billion for Dollar Shave Club” by Steven Davidoff Solomon and “Robopocalyspe Not” by James Surowiecki.

These readings were all about the economic effects of the internet on both novel online industries as well as on real-world business. In the case of the Economist article, wireless networking was used to coordinate fish sales in India. Cell phones allowed for fishermen to call ahead and decide where to go to sell the day’s catch for the most profit. Originally, prices differed a lot between markets because if one boat came back with a good catch of a certain fish, others would too and the increased supply often lead to drops in prices at one market while supply was too low at another. Cell phones, and the internet generally, can help to coordinate pricing and make the larger economy for efficient. Ironically, this is what the Soviets were trying to do with their OGAS as I wrote about earlier, while in the US it was illegal to use the internet for commercial purposes until it was privatized circa 1990 (Erik Fair).

Online businesses have also had a huge impact on the retail industry. Chris Anderson discusses this in “The Long Tail” where he shows that low sales volume products are profitable on the internet even when they wouldn’t be worth the shelf space in a physical-world store.  These products are the “long tail.” The three main industries he talks about are the music, books and film industries. Anderson argues that the new ability for record labels and the like to keep selling low volume items past their typical due dates is a great opportunity to make more money, even as peer-to-peer file sharing networks are distributing more and more music less legally. His argument is that as long as the cost is low enough, people prefer buying to downloading illegally because there are costs associated with getting not free things for free on the internet (like getting caught, and the general hassle). Therefore, if record labels provide older and smaller market songs for a low price (49 cents) then people will prefer this to alternative methods. As the record label wasn’t able to sell any of these songs before online sales this would just be extra cash. This argument goes against the oft-touted the-internet-is-killing-our-industry complaint from record labels (although he does predict that the retail (what’s a CD?) music industry won’t last long).

What struck me most is that Anderson wrote this article in 2004 and his analysis still makes sense and his predictions are quite accurate. He predicts that flat fee unlimited streaming services will take over in the future. However, what he doesn’t predict is the rise of crowdsourced and peer-produced content. His article predates YouTube by a couple of months so this omission isn’t a big surprise, but sites like YouTube have been incredibly disruptive because the audience and the content creators are the same. YouTube doesn’t pay people to upload videos, and especially for the first few years of its existence, the site was dominated by amateur videos produced by people who were guarantied no compensation for their efforts. YouTube is a video entertainment company that crowdsources all its content production and only shares advertising profits after the fact — video producers don’t sell their content to YouTube, they only get a share of the revenue it creates. YouTube also became successful before it started sharing profits at all through its partner program in 2007.

12 years after Anderson, Steven Davidoff Solomon wrote about Unilever’s 1 billion dollar purchase of the Dollar Shave Club. Solomon writes that the Dollar Shave Club business strategy was innovative in that the company was only a marketing platform that contracted all the real-world work. It managed to compete against Gillette’s market dominance through a mix of branding, costumer loyalty, and convenience. It also undercut Gillette by forgoing relationships with retail stores (or retail stores entirely). Solomon notes that anyone could have bought Dollar Shave Club razors without branding directly from their supplier’s website — and save money. They didn’t invent a cheaper razor, they just came up with a better way to sell them.

These online “platforms” are very elaborate and often brilliant schemes to profit off of other people’s work. Just like the Dollar Shave club that connected costumers with a cheaper razor manufacturer through an extremely convenient interface, many online companies are just specialized retailers that connect costumers with products. Some, like YouTube, even manage to turn their users into both their products (advertising targets) and the producers of their content. “Platforms” also have the benefit of not having to deal with labor relations and can masquerade as software companies. Of course, not of this is to say that I think this is all bad, YouTube especially has played an important role in lowering the barrier of entry to content production and online music libraries have allowed far more musicians than just the big hits to profit off of their work and connect with fans.

This is the third time I’ve read End-to-end Arguments in System Design (Saltzer, Reed & Clark) and the paper has gotten better every time. The main takeaway for me is that whatever feature you might want to network to offer, it’s usually much better to implement it at the end-point application level than the network level. Ensuring FIFO delivery or ensuring all packets arrive is nice but such features are trouble when all you care about is getting most of the packets there quickly (like in voice calls). The end-points can take care of packet ordering and data integrity by using check sums/hashes of the data and if the application requires both perfect accuracy and timeliness, then network level “solutions” won’t work either because some things can’t be done (networks can get faster and more accurate, but as long as there are errors and the speed of light there will always be a trade-off between speed and reliability). Further, because end-point applications will have to assume some error rate, it will always have to check for errors anyway. The conclusion here is that having the network do more will get in the way of applications that don’t require those features because features inevitably include trade-offs.

I think BitTorrent is a good example of doing things the right way. BitTorrent divides files into “pieces” and checks for errors on each individual “piece”; it doesn’t re-download the whole file because one bit was dropped. It does this in part because all the pieces come from different places, and so it’s important to know that you are getting the right data and not something completely different (when downloading from a single source you may not get what you wanted either, but the only fixable issues are errors, because downloading the wrong file entirely is more a question about life-choices that has nothing to do with networks). I think BitTorrent is a particularly clever protocol because it embraces the weakness of the internet and provides an end-point solution. By weaknesses I mean bottlenecks that happen when a lot of traffic is going to one server (e.g. for downloads), both for the server itself and for its connection to the internet as well as the inevitable reliability issues. There are no bottlenecks when the tubes are all in parallel.

Instead of relying on someone else to fix the system, BitTorrent fixes the problems itself. Organizations that advocate treating packets differently based on what they are might gain from taking this sort of approach. Of course, the subversive nature of BitTorrent might convince them otherwise. Oh well.

I’ve finished, Where Wizards Stay up Late by Hafner and Lyon and read “How the Soviets invented the internet and why it didn’t work” by Benjamin Peters.

“Standards should be discovered, not decreed.” -Unnamed TCP/IP advocate.

After finishing Where Wizards Stay up Late, I began to wonder how on earth any of this could have happened especially with so many companies trying to profit off of it at every step and simultaneously stop others from doing so. BBN tried to keep their IMP software secret, but gave in and let people have the source code (for a fee), ARPA nearly sold the whole ARPANET to AT&T (AT&T still thought packet switching wasn’t the future — close call). But in the end the major decisions were made by engineers who yelled at each other a bunch and implemented things in ways that worked even when some governments supported another way (like TCP/IP vs OSI). email was hacked together using ARNPANET’s file transfer protocol and soon began being used for all sorts of not-allowed things like socializing or anything that wasn’t government business (email didn’t care but ARPA technically did). This all happened somehow, with a lot of government funding and companies that funded long-term internal projects that had no immediate value and often weren’t commercially viable (notably, BBN had financial troubles because they failed to make enough of their products profitable — but at the same time they did quite a bit for the Internet). We only need to look at the infamous business model of AT&T/Bell and their telephone network to see a sad example, not of failure but of an inability to allow for real innovation. The ARPANET was in some senses like a monopoly, albeit government controlled (socialists?!) but predominantly benevolent and open/free. Once TCP/IP took over more and more networks started being connected to the ARPANET (hence the Inter-net). This is precisely what AT&T feared most regarding the telephone network; it simply wasn’t in their interests to let other companies profit using their network. The openness of innovation in the ARPANET and early Internet isn’t at all characteristic of the free market, where money and profit drive rather than discovery and true innovation done by people and companies who did not necessarily get rich off of it — even if their idea caught on.

This train of thought led me to wonder what the Soviets were up to at the time. A short search later I found that the Russians did indeed have something, kinda, and that it was a complete failure. OGAS (All-State Automated System) was meant to facilitate economic planning and pricing decisions across the Soviet Union. It was an idealist’s dream for the communist (cyber)state. In typical Soviet fashion that meant nothing and conflicting rational interests kept OGAS from becoming anything. At least, this is how Benjamin Peters describes it. According to Peters, “The first global computer network emerged thanks to capitalists behaving like cooperative socialists, not socialists behaving like competitive capitalists.” I don’t know enough about Soviet politics to know whether waring government ministries could ever be described as capitalist competition, but it does seem plausible all the same.

Peters continued by referencing Bruno Latour’s argument that technology is society made durable. In other words (says Peters) this means that “social values are embedded in technology.” He quickly ties this into modern mass surveillance projects like Facebook, Microsoft Cloud and NSA saying thatthese may lead to continuing the “20th- century tradition of general secretariats committed to privatizing personal and public information for their institutional gain.” As society moves towards accepting centralized control on massive scales (governments that see all and Facebook that knows all) our technology will begin to reflect this centralized philosophy; exactly the  opposite of the “values” of openness, innovation and discovery that were the initial drivers of the Internet.

In other news, nano has a built in spell check!