net.wars

"How might this be used to further marginalize the already marginalized?" At this week's Computers, Freedom, and Privacy Jon Gosier proposed that all technology designers should ask themselves this question as they begin work.

The example he presented was this: while working on a project in Uganda, his small, under-resourced group found that dissident texts were being blocked; in response they developed a peer-to-peer secure messaging system but never rolled it out. As Gosier told the story, the reasons were partly the concern that the platform could be used by extremist radical groups, and partly financial, in that the group's caurious funders felt the technology was too risky to back. This example was not completely apt: it's not clear who the marginalized are in that story.

I also suspect this is the Silicon Valley response would echo Henry Higgins in George Bernard Shaw's Pygmalion, "Do any of us understand what we are doing? If we did, would we ever do it?" And to a large extent that would be right. Technology developers are consistently surprised by the uses people find for their work post-release: they write letters in spreadsheets, they use tennis racquets to strain spaghetti, and the phonograph that changed the history of music was originally designed as a device to record telegrams. People are weird and inventive about their tools. If Phil Zimmermann had thought like Gosier's cautious funders he would not have launched PGP. On the other hand, if the internet pioneers had forseen the cooption of their new medium, which they imagined as a force for democracy, as a mass surveillance platform, they might have designed it differently. Certainly, as Michael Froomkin pointed out in April at this year's We Robot, the presence of a single lawyer would have substantially changed the design of the domain name system.

Gosier's other examples were bitcoin, which as he said is too technically complicated for most of the world's population to use directly, and wearable health monitors, which he suggested risk making healthcare reliant on gadgets that many people cannot afford.

Gosier was just one of a number of speakers pointing out the limitations of the way a lot of us think about computers, freedom, and privacy - as well as the risks posed by AI, machine learning, and predictive analytics.

The Pakistani lawyer Hamid Khan, head of the Stop LAPD Spying Coalition, prefers to call the latter "speculative" rather than "predictive", as in "speculative policing". Centuries of racism are embedded in the data on which the Los Angeles Police Department base their judgments of how people walk and move their hands. "What is being sold is the claim that technology is race neutral," he said. Instead, it's "Garbage in, Garbage Out." The reality on the ground may be quite different from what the designers had in mind.

"New technology is doing very old work," said Malkia Cyril, executive director of the Center for Media Justice. Communities such as black and Muslim Americans "live scanned, tracked, and traced today," she said. "It did not begin with NSA revelations." Reframing the argument as she does changes the focus entirely. In arguing against mass surveillance, many (mostly white) privacy advocates have accepted that targeted surveillance has its place. For Cyril, "It's not a useful distinction." Because: whom do you target? Disproportionately, the groups you always have. Similarly, network neutrality is usually presented as a question of business opportunity; for Cyril it's a matter of freedom and social justice.

The discussion led the author and former software engineer Ellen Ullman to ask this: "How do we reach venture capitalists to say, 'Why is disruption the goal?'" She was referring to the current mantra of Silicon Valley venture capitalists, who select investments based on their potential to up-end established industries and sectors: to disrupt. (I imagine the word has very a very different meaning for Cyril and Gosier.) Uber, for example, doesn't just disrupt the taxi trade, though of course it does that; it's also frequently seen as a threat to public transport, the delivery business, and possibly municipal government itself. Why else does a six-year-old company have a market cap in the tens of billions?

Anika Collier Navaroli noted that she spends a lot of time talking to technologists. "Lots of the time it's the first time they've heard the civil rights implications." Part of Ullman's answer is to encourage everyone to learn to code as the only route to self-determination in the digital world.

All these pieces taken together - especially when stood next to EPIC international policy fellow Fanny Hidvegi's account of Hungary's increasingly restrictive government and Gosier's references to "trickle-down techonomics" - is an important reminder. It's not just that technology reaches the streets where ordinary citizens get their hands on it - GPS, the internet, and mobile telephony are all technologies that began with the military and are now ubiquitous. It's that missions creep and before it gets to us that same technology will be deployed by much lower layers of law enforcement. We've seen this in the UK: half of local councils have been caught using the Regulation of Investigatory Powers Act to tackle...littering. Paging Arlo Guthrie.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.