" /> net.wars: June 2019 Archives

« May 2019 | Main | July 2019 »

June 28, 2019

Failure to cooperate

sweat-nottage.jpgIn her 2015 Pulitzer Prize-winning play, Sweat, on display nightly in London's West End until mid-July, Lynn Nottage explores class and racial tensions in the impoverished, post-industrial town of Reading, PA. In scenes alternating between 2000 and 2008, she explores the personal-level effects of twin economic crashes, corporate outsourcing decisions, and tribalism: friends become opposing disputants; small disagreements become violent; and the prize for "winning" shrinks to scraps. Them who has, gets; and from them who have little, it is taken.

Throughout, you wish the characters would recognize their real enemies: the company whose steel tubing factory has employed them for decades, their short-sighted union, and a system that structurally short-changes them. The pain of the workers when they are locked out is that of an unwilling divorce, abruptly imposed.

The play's older characters, who would be in their mid-60s today, are of the age to have been taught that jobs were for life. They were promised pensions and could look forward to wage increases at a steady and predictable pace. None are wealthy, but in 2000 they are financially stable enough to plan vacations, and their children see summer jobs as a viable means of paying for college and climbing into a better future. The future, however, lies in the Spanish-language leaflets the company is distributing to frustrated immigrants the union has refused to admit and who will work for a quarter the price. Come 2008, the local bar is run by one of those immigrants, who of necessity caters to incoming hipsters. Next time you read an angry piece attacking Baby Boomers for wrecking the world, remember that it's a big demographic and only some were the destructors. *Some* Baby Boomers were born wreckage, some achieved it, and some had it thrust upon them.

We leave the characters there in 2008: hopeless, angry, and alienated. Nottage, who has a history of researching working class lives and the loss of heavy industry, does not go on to explore the inner workings of the "digital poorhouse" they're moving into. The phrase comes from Virginia Eubanks' 2018 book, Automating Inequality, which we unfortunately missed reviewing before now. If Nottage had pursued that line, she might have found what Eubanks finds: a punitive, intrusive, judgmental, and hostile benefits system. Those devastated factory workers must surely have done something wrong to deserve their plight.

Eubanks presents three case studies. In the first, struggling Indiana families navigate the state's new automated welfare system, a $1.3 billion, ten-year privatization effort led by IBM. Soon after its 2006 launch, it began sending tens of thousands of families notices of refusal on this Kafkaesque basis: "Failure to cooperate". Indiana eventually canceled IBM's contract, and the two have been suing each other ever since. Not represented in court is, as Eubanks says, the incalculable price paid in the lives of the humans the system spat out.

In the second, "coordinated entry" matches homeless Los Angelenos to available resources in order of vulnerability. The idea was that standardizing the intake process across all possible entryways would help the city reduce waste and become more efficient while reducing the numbers on Skid Row. The result, Eubanks finds, is an unpredictable system that mysteriously helps some and not others, and that ultimately fails to solve the underlying structural problem: there isn't enough affordable housing.

In the third, a Pennsylvania predictive system is intended to identify children at risk of abuse. Such systems are proliferating widely and controversially for varying purposes, and all raise concerns about fairness and transparency: custody decisions (Durham, England), gang membership and gun crime (Chicago and London), and identifying children who might be at risk (British local councils). All these systems gather and retain, perhaps permanently, huge amounts of highly intimate data about each family. The result in Pennsylvania was to deter families from asking for the help they're actually entitled to, lest they become targets to be watched. Some future day, those same records may pop when a hostile neighbor files a minor complaint, or haunt their now-grown children when raising their own children.

All these systems, Eubanks writes, could be designed to optimize access to benefits instead of optimizing for efficiency or detecting fraud. I'm less sanguine. In prior art, Danielle Citron has written about the difficulties of translating human law accurately into programming code, and the essayist Ellen Ullman warned in 1996 that even those with the best intentions eventually surrender to computer system imperatives of improving data quality, linking databases, and cross-checking, the bedrock of surveillance.

Eubanks repeatedly writes that middle class people would never put up with this level of intrusion. They may have no choice. As Sweat highlights, many people's options are shrinking. Refusal is only possible for those who can afford to buy their help, an option increasingly reserved for a privileged few. Poor people, Eubanks is frequently told, are the experimental models for surveillance that will eventually be applied to all of us.

In 2017, Cathy O'Neil argued in Weapons of Math Destruction that algorithmic systems can be designed for fairness. Eubanks' analysis suggests that view is overly optimistic: the underlying morality dates back centuries. Digitization has, however, exacerbated its effects, as Eubanks concludes. County poorhouse inmates at least had the community of shared experience. Its digital successor squashes and separates, leaving each individual to drink alone in that Reading bar.


Illustrations: Sweat's London production poster.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

June 21, 2019

Party games

tory-candidates-2019-06-2.pngTo anyone not born in the UK, and to many who were, the ongoing British Conservative party leadership contest - which doubles as a contest to replace the in-office Prime Minister - is a weird mix of decaying feudalism and pointlessness. On Tuesday, the five then-remaining candidates perched awkwardly on BBC stools and answered questions submitted by the public. This, despite the fact that the public at large has no vote. After a series of elimination rounds in which only Conservative MPs vote, the final two will make their case to the estimated 124,000 members of the Conservative party. The nearest US analogue to this particular contest, which began with 11 candidates, is today's 24-Democrat nomination field - if the final choice were up to a group only modestly larger than the population of Vermont and the person selected were about to take over the presidency.

In one sense, the least democratic part of this is the MPs-only selection of the shortlist. Yet they are doing what the electoral college was supposed to do: represent their constituents' wishes based on their greater and more intimate knowledge of the candidates. Yet if you've seen the transition episode in which Yes, Minister's Jim Hacker is lifted to the top job, instead you imagine these MPs all elbowing each other to further their own interests, making deals, weaponizing that personal knowledge, and discovering their inner killer instincts.

My sense in reading the briefing produced by the House of Commons researchers (PDF) on the history of these contests is that they are gradually becoming more presidential over time, though not more democratic. Until 1965, the new party leader "emerged" from back room discussions. You can see the remnants of this method in that Yes, Minister episode ("Party Games") as senior civil servants mull the right choice. Their criteria: easily manipulated, no "silly notions about running the country", and won't split the party. Hacker finally locks down the job by convincing the press he has blocked an onerous EC plan to standardize Euro sausages and make British sausages illegal.

Europe: a scapegoat then, as now. In Tuesday evening's debate, the four not-Rory Stewart candidates competed on two things: tax cuts, which Stewart correctly pointed out the country can't afford, and which one was more likely to deliver Brexit, which Stewart correctly pointed out cannot be solved by any of their proposals. Meanwhile, weary MPs are speculating how soon the next contest will be, while journalists are mulling which outcome makes the best story and for how long. A YouGov poll this week found Conservative party members will sacrifice almost anything - their party, Scotland, Northern Ireland, for Brexit. Anything except a Labour government.

The reason I said "more presidential" is that slowly but surely over the the last 30 to 40 years the campaigns for party leadership have become more public-facing, personality-driven, and expensive. The library note says that in 2016 the spending limit was £135,000 per candidate. Granted, even this year's limit of £150,000 seems piddling to anyone in the US, but in a three-week contest in which only party members can vote, what on earth do they spend it on? Given the Electoral Commission spending limits for general elections, it's arguable that blanketing the country with hustings for this run-off is a cheater's way of getting ahead on campaigning for the general election that everyone thinks is inevitably coming soon.

Over the same time, government power has been concentrating toward the center, a trend helped by austerity, which has seen cuts of almost 60% to local authority budgets. While I've long deplored the fact that the British system is in effect an elected dictatorship - since a party with a big enough majority in the House of Commons can push through any legislation it likes - allowing a cult of Prime Ministerial personality to take hold in a country with no written constitution to guarantee the separation of powers seems dangerous. The one saving grace used to be that the government's legitimacy could be challenged at any time - and that was greatly watered down with the passage of the Fixed-term Parliaments Act (2011).

The contest for power between Parliament and the Prime Minister has been a notable feature of politics since the 2016 EU referendum. Theresa May's original plan was to give notice of withdrawal to the EU without Parliament's approval. It took activist Gina Miller to bring a legal case to challenge the government's authority to act unilaterally. She won in the High Court of Justice, and then again on appeal in the Supreme Court.

We last discussed Brexit here only three months ago, shortly before the original March 29 deadline. It seems like eternity. The new deadline, October 31, is eighteen weeks away in calendar time, but after you subtract four weeks of campaigning, another to vote, summer holidays, and three weeks of party conferences starting in mid-September, there's barely a handful of days of Parliamentary time. The Conservative party candidates are clearly rearranging the deck chairs on the Titanic. But is it their shrinking party, Brexit, or the country that's the ship?


Illustrations: Tuesday's BBC debate (left to right: Emily Maitlis, Boris Johnson, Jeremy Hunt, Michael Gove, Sajid Javid, Rory Stewart).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

June 13, 2019

Matrices of numbers

Wilcox, Dominic - Stained Glass car.jpgThe older man standing next to me was puzzled. "Can you drive it?"

He gestured at the VW Beetle-style car-like creation in front of us. Its exterior, except for the wheels and chassis, was stained glass. This car was conceived by the artist Desmond Wilcox, who surmised that by 2059 autonomous cars will be so safe that they will no longer need safety features such as bumpers and can be made of fragile materials. The sole interior furnishing, a bed, lets you sleep while in transit. In person, the car is lovely to look at. Utterly impractical today in 2019, and it always will be. The other cars may be safe, but come on: falling tree, extreme cold, hailstorm...kid with a baseball?

On being told no, it's an autonomous car that drives itself, my fellow visitor to the Science Museum's new exhibition, Driverless, looked dissatisfied. He appeared to prefer driving himself.

"It would look good with a light bulb inside it hanging at the back of the garden," he offered. It would. Bit big, though last week in San Francisco I saw a bigger superbloom.

"Driverless" is a modest exhibition by Science Museum standards, and unlike previous robot exhibitions, hardly any of these vehicles are ready for real-world use. Many are graded according to their project status: first version, early tests, real-world tests, in use. Only a couple were as far along as real-world tests.

Probably a third are underwater explorers. Among the exhibits: the (yellow submarine!) long-range Boaty McBoatface Autosub, which is meant to travel up to 2,000 km over several months, surfacing periodically to send information back to scientists. Both this and the underwater robot swarms are intended for previously unexplored hostile environments, such as underneath the Antarctic ice sheet.

Alongside these and Wilcox's Stained Glass Driverless Car of the Future was the Capri Mobility pod, the result of a project to develop on-demand vans that can shuttle up to four people along a defined route either through a pedestrian area or on public roads. Small Robot sent its Tom farm monitoring robot. And from Amsterdam came Roboat, a five-year research project to develop the first fleet of autonomous floating boats for deployment in Amsterdam's canals. These are the first autonomous vehicles I've seen that really show useful everyday potential for rethinking traditional shapes, forms, and functionality: their flat surfaces and side connectors allow them to be linked into temporary bridges a human can walk across.

There's also an app-controlled food delivery drone; the idea is you trigger it to drop your delivery from 20 meters up when you're ready to receive it. What could possibly go wrong?

On the fun side is Duckietown (again, sadly present only as an image), a project to teach robotics via a system of small, mobile robots that motor around a Lego-like "town" carrying small rubber ducks. It's compelling like model trains, and is seeking Kickstarter funding to make the hardware for wider distribution. This should have been the hands-on bit.

Previous robotics-related Science Museum exhibitions have asked as many questions as they answered. At that, this one is less successful. dont-cross.jpgDrive.ai's car-mounted warning signs, for example, are meant to tell surrounding pedestrians what its cars are doing. But are we really going to allow cars onto public roads (or even worse, pedestrian areas, like the Capri pods) to mow people down who don't see, don't understand, can't read, or willfully ignore the "GOING NOW; DON'T CROSS" sign? So we'll have to add sound: but do we want cars barking orders at us? Today, navigating the roads is a constant negotiation between human drivers, human pedestrians, and humans on other modes of transport (motorcycles, bicycles, escooters, skateboards...). Do we want a tomorrow where the cars have all the power?

In video clips researchers and commentators like Noel Sharkey, Kathy Nothstine, and Natasha Merat discuss some of these difficulties. Merat has an answer for the warning sign: humans and self-driving cars will have to learn each other's capabilities in order to peacefully coexist. This is work we don't really see happening today, and that lack is part of why I tend to think Christian Wolmar is right in predicting that these cars are not going to be filling our streets any time soon.

The placard for the Starship Bot (present only as a picture) advises that it cannot see above knee height, to protect privacy, but doesn't discuss the issues raised when Edward Hasbrouck encountered one in action. I was personally disappointed, after the recent We Robot discussion of the "monstrous" Moral Machine and its generalized sibling the trolley problem, to see it included here with less documentation than on the web. This matters, because the most significant questions about autonomous vehicles are going to be things like: what data do they collect about the people and things around them? To whom are they sending it? How long will it be retained? Who has the right to see it? Who has the right to command where these cars go?

More important, Sharkey says in a video clip, we must disentangle autonomous and remote-controlled vehicles, which present very different problems. Remote-controlled vehicles have a human in charge that we can directly challenge. By contrast, he said, we don't know why autonomous vehicles make the decisions they do: "They're just matrices of numbers."


Illustrations: Wilcox's stained glass car.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

June 7, 2019

The right to lie

Sand_Box-wikimedia.JPGPrivacy, pioneering activist Simon Davies writes in his new book, Privacy: A Personal Chronicle, "varies widely according to context and environment to the extent that even after decades of academic interest in the subject, the world's leading experts have been unable to agree on a single definition." In 2010, I suggested defining it as being able to eat sand without fear. The reference was to the prospect that detailed electronic school records present to small children and their parents of a permanently stored data on everything they do. It didn't occur to me at the time, but in a data-rich future when eating sand has been outlawed (because some pseudoscientist believes it leads to criminality) and someone asks, "Did you eat sand as a child?", saying no because you forgot the incident (because you were *three* and now you're 65) will make you a dangerous liar.

The fact that even innocent pastimes - like eating sand - look sinister when the beholder is already prejudiced - is the kind of reason why sometimes we need privacy even from the people we're supposed to be able to trust. This year's Privacy Law Scholars tossed up two examples, provided by Najarian Peters, whose project examines the reasons why black Americans adopt edu0cational alternatives - home-schooling, "un-schooling" (children follow their own interests, Summerhill-style), and self-directed education (children direct their own activities), and Carleen M. Zubrzycki, who has been studying privacy from doctors. Cue Greg House: Everybody lies. Judging from the responses Zubrzycki is getting from everyone she talks to about her projects, House is right, but, as he would not accept, we have our reasons.

Sometimes lying is essential to get a new opinion untainted by previous incorrect diagnoses or dismissals (women in pain, particularly). In some cases, the problem isn't the doctor but the electronic record and the wider health system that may see it. In some cases, lying may protect the doctor, too; under the new, restrictive Alabama law that makes performing an abortion after six weeks a felony, doctors would depend on their patients' silence. This last topic raised a question: given that women are asked the date of their last period at every medical appointment, will states with these restrictive laws (if they are allowed to stand) begin demanding to inspect women's menstrual apps?

The intriguing part of Peters' project is that most discussions of home-schooling and other alternative approaches to education focus on the stereotype of parents who don't want their kids to learn about evolution, climate change, or sex. But her interviewees have a different set of concerns: they want a solid education for their children, but they also want to protect them from prejudice, stigmatization, and the underachievement that comes with being treated as though you can't achieve much. The same infraction that is minor for a white kid may be noted and used to confirm teachers' prejudices against a black child. And so on. It's another reminder of how little growing up white in America may tell you about growing up black in America.

Zybrzycki and Peters were not alone in finding gaps in our thinking: Anne Toomey McKenna, Amy C. Gaudion, and Jenni L. Evans have discovered that existing laws do not cover the use of data collected by satellites and aggregated via apps - think last year's Strava incident, in which a heat map published by the company from aggregated data exposed the location of military bases and the identities of personnel - while PLSC co-founder Chris Hoofnagle began the initial spadework on the prospective privacy impacts of quantum computing.

Both of these are gaps in current law. GDPR covers processing data; it says little about how the predictions derived from that data may be used. GDPR also doesn't cover the commercial aggregation of satellite data, an intersectional issue requiring expertise in both privacy law and satellite technology. Yet all data may eventually be personal data, as 100,000 porn stars may soon find out. (Or they may not; the claim that a programmer has been able to use facial recognition to match porn performers to social media photographs is considered dubious, at least for now) For this reason, Margot Kaminski is proposing "binary governance", in which one prong governs the use of data and the other ensures due process.

Tl;dr: it's going to be rough. Quantum computing is expected to expose things that today can successfully be hidden- including stealth surveillance technologies. It's long been mooted, for example, that quantum computing will render all of today's encryption crackable, opening up all our historical encrypted data. PLSC's discussion suggests it will also vastly increase the speed of communications. More interesting was a comment from Pam Dixon, whose research shows that high-speech biometric analysis is already beginning to happen, as companies in China find new, much faster, search methods that are bringing "profound breakthroughs" in mass surveillance.

"The first disruption was the commodification of data and data breakers," she said. "What's happening now is the next phase, the commodification of prediction. It's getting really cheap." If the machine predicts that you fit the profile of people who ate sand, what will it matter if you say you didn't? Even if it's true.


Illustrations: Sand box (via Janez Novak at Wikimedia).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.