The new surveillance capitalism

John Naughton in Prospect:

Consider the so-called “Right to Be Forgotten” granted by the European Court of Justice (ECJ) in 2014, which gives European Union citizens the right to petition Google to have information about them removed from the company’s search results in Europe. To call it a “right to be forgotten” is not strictly accurate; it is merely a right to request that certain information not be listed in Google’s European search results—although in our networked world, this almost amounts to the same thing. If the dominant search engine doesn’t find you, then you have effectively ceased to exist. The ECJ ruling bows to the reality that Google has a unique capacity to make or break reputations.

Google itself has been given responsibility for managing the complaints and adjudicating who gets to be “forgotten,” effectively outsourcing a judicial responsibility to a private company. Territorial sovereignty, the kind exercised by elected governments, has been supplanted by what the legal scholar Frank Pasquale calls “functional sovereignty.” The digital giants, Pasquale maintains, “are no longer market participants.” Rather, “they are market makers” in their fields, “able to exert regulatory control over the terms on which others can sell goods and services.” Moreover, he says, “they aspire to displace more government roles over time, replacing the logic of territorial sovereignty with functional sovereignty. In functional arenas from room-letting to transportation to commerce, persons will be increasingly subject to corporate, rather than democratic, control.”

There is no issue so big that the tech companies think they can’t handle it themselves. When the issue of fake news flared up amid the US election, for example, the first response of Mark Zuckerberg, Facebook’s CEO, was a mixture of denial and incredulity. Then, as the evidence mounted that his advertising machine had been weaponised by dark political actors, he pivoted rapidly from that incredulity to scepticism and then—as the evidence became incontrovertible—to a technocratic determination to “solve” the problem. By the end of September, he was issuing a personal Yom Kippur post on Facebook pleading for “forgiveness” in light of the way that “my work was used to divide people.”

But Facebook has two conflicts of interest that inhibit it from fixing the problems. First, surveillance capitalism requires the maximisation of “user engagement,” to create the data that is to be monetised. And it turns out that Facebook users are often more engaged by fake news than they are by mundane truths. The much-vaunted, pending overhaul of Facebook’s algorithm to give priority to material shared between individuals represents a retreat from real news just as much as it does from fake. That could have adverse implications for the responsible media, and meanwhile, of course, that emphasis on shares only deepens the “engagement.” The second conflict stems from the fact that if Zuckerberg were to accept editorial responsibility for what is posted by his website’s users it would effectively destroy his company, given that there aren’t enough administrators in the world to vet what gets posted to Facebook in a single second.

This hasn’t stopped some from demanding that social media organisations accept responsibility for what appears on their sites. And the Germans have passed legislation that mandates swingeing penalties on platforms that do not take down offending content in a matter of hours. But like the right to be forgotten, this statute delegates to private companies the task of deciding what shall and shall not be published in a democracy—another illustration of functional sovereignty replacing its territorial counterpart.

Back in the 1980s, the cultural critic Neil Postman argued that our future would be bracketed by the nightmares of two British novelists: George Orwell, who thought we would be destroyed by the things we fear; and Aldous Huxley, who believed that our undoing would be the things that delighted us. With the aid of digital technology, we are managing to achieve both nightmares at once. We click compulsively on health scares and other anxieties while Big Brother watches, but are also distracted by dubious political claims that make us feel good and reinforce our prejudices.