net.wars

Privacy advocacy has the effect of making you hyper-conscious of the exponentially increasing supply of data. All sorts of activities that used to leave little or no trace behind now create a giant stream of data exhaust, from interactions with friends (now captured by social media companies), to TV viewing (now captured by streaming services and cable companies), where and when we travel (now captured by credit card companies, municipal smart card systems, and facial recognition-equipped cameras), and everything we buy (unless you use cash). And then there are the vast amounts of new forms of data being gathered by the sensors attached to Internet of Things devices and, increasingly, more intimate devices, such as medical implants.

And yet. In a recent paper (PDF) that Tammy Xu summarizes at MIT Technology Review, the EPOCH AI research and forecasting unit argues that we are at risk of running out of a particular kind of data: the stuff we use to train large language models. More precisely, the stock of data deemed suitable for use in language training datasets is growing more slowly than the size of the datasets these increasingly large and powerful models require for training. The explosion of privacy-invasive, mechanically captured data mentioned above doesn't help with this problem; it can't help train what today passes for "artificial intelligence to improve its ability to generate content that reads like it could have been written by a sentient human.

So in this one sense the much-debunked saw that "data is the new oil" is truer than its proponents thought. Like drawing water from aquifers or depleting oil reserves, data miners have been relying on capital resources that have taken eras to build up and that can only be replenished over similar time scales. We professional writers produce new "high-quality" texts too slowly.

As Xu explains, "high-quality" in this context generally means things like books, news articles, scientific papers, and Wikipedia pages - that is, the kind of prose researchers want their models to copy. Wikipedia's English language section makes up only 0.6% of GPT-3 training data. "Low-quality" is all the other stuff we all churn out: social media postings, blog postings, web board comments, and so on. There is of course vastly more of this (and some of it is, we hope, high-quality)..

The paper's authors estimate that the high-quality text modelers prefer could be exhausted by 2026. Images, which are produced at higher rates, will take longer to exhaust - lasting to perhaps between 2030 and 2040. The paper considers three options for slowing exhaustion: broaden the standard for acceptable quality; find new sources; and develop more data-efficient solutions for training algorithms. Pursuing the fossil fuel analogy, I guess the equivalents might be: turning to techniques such as fracking to extract usable but less accessible fossil fuels, developing alternative sources such as renewables, and increasing energy efficiency. As in the energy sector, we may need to do all three.

I suppose paying the world's laid-off and struggling professional writers to produce text to feed the training models can't form part of the plan?

The first approach might have some good effects by increasing the diversity of training data. The same is true of the second, although using AI-generated text (synthetic data to train the model seems as recursive as using an algorithm to highlight trends to tempt users. Is there anything real in there?

Regarding the third... It's worth remembering the 2020 paper On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? (the paper over which Google apparently fired AI ethics team leader Timnit Gebru). In this paper (and a FaCCT talk), Gebru, Emily M. Bender, Angelina McMillan-Major, and Shmargaret Shmitchell outlined the escalating environmental and social costs of increasingly large language models and argued that datasets needed to be carefully curated and documented, and tailored to the circumstances and context in which the model was eventually going to be used.

As Bender writes at Medium, there's a significant danger that humans reading the language generated by systems like GPT-3 may *believe* it's the product of a sentient mind. At IAI News, she and Chirag Shah call text generators like GPT-3 dangerous because they have no understanding of meaning even as they spit out coherent answers to user questions in natural language. That is, these models can spew out plausible-sounding nonsense at scale; in 2020, Renée DiResta predicted at The Atlantic that generative text will provide an infinite supply of disinformation and propaganda.

This is humans finding patterns even where they don't exist: all the language model does is make a probabilistic guess about the next word based on statistics derived from the data it's been trained on. It has no understanding of its own results. As Ben Dickson puts it at TechTalks as part of an analysis of the workings of the language model BERT, "Coherence is in the eye of the beholder." On Twitter, Bender quipped that a good new name would be PSEUDOSCI (for Pattern-matching by Syndicate Entities of Uncurated Data Objects, through Superfluous (energy) Consumption and Incentives).

If running out of training data means a halt on improving the human-like quality of language generators' empty phrases, that may not be such a bad thing.

Illustrations: Drunk parrot (taken by Simon Bisson).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 5:05 AM | Permalink | Comments (0) | TrackBacks (0)

The early months of 2020 were a time of radical uncertainty - that is, decisions had to be made that affected the lives of whole populations where little guidance was available. As Leonard Smith and David Tuckett explained at their 2018 conference on the subject (and a recent Royal Society scientific meeting) decisions under radical uncertainty are often one-offs whose lessons can't inform the future. Tuckett's and Smith's goal was to understand the decision-making process itself in the hope that this part of the equation at least could be reused and improved.

Inevitably, the discussion landed on mathematical models, which attempt to provide tools to answer the question, "What if?" This question is the bedrock of science fiction, but science fiction writers' helpfulness has limits: they don't have to face bereaved people if they get it wrong; they can change reality to serve their sense of fictional truth; and they optimize for the best stories, rather than the best outcomes. Beware.

In the case of covid, humanity had experience in combating pandemics, but not covid, which turned out to be unlike the first known virus family people grabbed for: flu. Imperial College epidemiologist Neil Ferguson became a national figure when it became known that his 2006 influenza model suggesting that inaction could lead to 500,000 deaths had influenced the UK government's delayed decision to impose a national lockdown. Ferguson remains controversial; Scotland's The Ferrett offers a fact check that suggests that many critics failed to understand the difference between projection and prediction and the importance of the caveat "if nothing is done". Models offer possible futures, but not immutable ones.

As Erica Thompson writes in her new book, Escape From Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It, models also have limits that we ignore at our peril. Chief among them is the fact that the model is always an abstracted version of reality. If it weren't, our computers couldn't calculate them any more than they can calculate all the real world's variables. Thompson therefore asks: how can we use models effectively in decision making without becoming trapped inside the models' internal worlds, where their simplified assumptions are always true? More important, how can we use models to improve our decision making with respect to the many problems we face that are filled with uncertainties?

The science of covid - or of climate change - is only a small part of the factors a government must weigh in deciding how to respond; what science tells us must be balanced against the economic and social impacts of different approaches. In June 2020, Ferguson estimated that locking down a week earlier would have saved 20,000 lives. At the time, many people had already begun withdrawing from public life. And yet one reason the government delayed was the belief that the population would quickly give in to lockdown fatigue and resist restrictions, rendering an important tool unusable later, when it might be needed even more. This assumption turned out to be largely wrong, as was the assumption in Ferguson's 2006 model that 50% of the population would refuse to comply with voluntary quarantine. Thompson calls this misunderstanding of public reaction a "gigantic failure of the model".

What else is missing? she asks. Ferguson had to resign when he himself was caught breaking the lockdown rules. Would his misplaced belief that the population wouldn't comply have been corrected by a more diverse team?

Thompson began her career with a PhD in physics that led her to examine many models of North Atlantic storms. The work taught her more about the inferences we make from models than about storms, and it opened for her the question of how to use the information models provide without falling into the trap of failing to recognize the difference between the real world and Model Land - that is, the assumption-enclosed internal world of the models.

From that beginning, Thompson works through different aspects of how models work and where their flaws can be found. Like Cathy O'Neil's Weapons of Math Destruction, which illuminated the abuse of automated scoring systems, this is a clearly-written and well thought-out book that makes a complex mathematical subject and accessible to a general audience. Thompson's final chapter, which offers approaches to evaluating models and lists of questions to ask modelers, should be read by everyone in government.

Thompson's focus on the dangers of failing to appreciate the important factors models omit leads her to skepticism about today's "AI", which of course is trained on such models: "It seems to me that rather than AI developing towards the level of human intelligence, we are instead in danger of human intelligence descending to the level of AI by concreting inflexible decision criteria into institutional structures, leaving no room for the human strengths of empathy, compassion, a sense of fairness and so on." Later, she adds, "AI is fragile: it can work wonderfully in Model Land but, by definition, it does not have a relationship with the real world other than one mediated by the models that we endow it with."

In other words, AI works great if you can assume a spherical cow.

Illustrations: The spherical cow that mocks unrealistic scientific models drawn jumping over the moon by Ingrid Kallick for the 1996 meeting of the American Astronomical Association (via Wikimedia).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 4:12 PM | Permalink | Comments (0) | TrackBacks (0)

Robots have stopped being robots. This is a good thing.

This is my biggest impression of this year's We Robot conference: we have moved from the yay! robots! of the first year, 2012, through the depressed doldrums of "AI" systems that make the already-vulnerable more vulnerable circa 2018 to this year, when the phrase that kept twanging was "sociotechnical systems". For someone with my dilettantish conference-hopping habit, this seems like the necessary culmination of a long-running trend away from robots as autonomous mobile machines to robots/AI as human-machine partnerships. We Robot has never talked much about robot rights, instead focusing on considering the policy challenges that arise as robots and AI become embedded in our lives. This is realism; as We Robot co-founder Michael Froomkin writes, we're a long, long way from a self-aware and sentient machine.

The framing of sociotechnical systems is a good thing in part because so much of what passes for modern "artificial intelligence" is humans all the way down, as Mary L. Gray and Siddhart Suri documented in their book, Ghost Work. Even the companies that make self-driving cars, which a few years ago were supposed to be filling the streets by now, are admitting that full automation is a long way off. "Admitting" as in consolidating or being investigated for reckless hyping.

If this was the emerging theme, it started with the first discussion, of a paper on humans in the loop, by Margot Kaminski, Nicholson Price, and Rebecca Crootof. Too often, the proposed policy-making proposal for handling problems with decision making systems is to insert a human, a "solution" they called the "MABA-MABA trap", for "Machines Are Better At / Men Are Better At". While obviously humans and machines have differing capabilities - people are creative and flexible, machines don't get bored - just dropping in a human without considering what role that human is going to fill doesn't necessarily take advantage of the best capabilities of either. Hybrid systems are of necessity more complex - this is why cybersecurity keeps getting harder - but policy makers may not take this into account or think clearly about what the human's purpose is going to be.

At this conference in 2016, Madeleine Claire Elish foresaw that the human would become a moral crumple zone or liability sponge, absorbing blame without necessarily being at fault. No one will admit that this is the human's real role - but it seems an apt description of the "safety driver" watching the road, trying to stay alert in case the software driving the car needs backup or the poorly-paid human given a scoring system and tasked with awarding welfare benefits. What matters, as Andrew Selbst said in discussing this paper, is the *loop*, not the human - and that may include humans with invisible control, such as someone who can massage the data they enter into a benefits system in order to help a particularly vulnerable child, or who have wide discretion, such as a judge who is ultimately responsible for parole decisions no matter what the risk assessment system says.

This is not the moment to ask what constitutes a human.

It might be, however, the moment to note the commentator who said that a lot of the problems people are suggesting robots/AI can solve have other, less technological solutions. As they said, if you are putting a pipeline through a community without its consent, is the solution to deploy police drones to protect the pipeline and the people working on it - or is it to put the pipeline somewhere else (or to move to renewables and not have a pipeline at all)? Change the relationship with the community and maybe you can partly disarm the police.

One unwelcome forthcoming issue, discussed in a paper by Kate Darling and Daniella DiPaola is the threat merging automation and social marketing poses to consumer protection. A truly disturbing note came from DiPaola, who investigated manipulation and deception with personal robots and 75 children. The children had three options: no ads, ads allowed only if they are explicitly disclosed to be ads, or advertising through casual conversation. The kids chose casual conversation because they felt it showed the robot *knew* them. They chose this even though they knew the robot was intentionally designed to be a "friend". Oy. In a world where this attitude spreads widely and persists into adulthood, no amount of "media literacy" or learning to identify deception will save us; these programmed emotional relationships will overwhelm all that. As DiPaola said, "The whole premise of robots is building a social relationship. We see over and over again that it works better if it is more deceptive."

There was much more fun to be had - steamboat regulation as a source of lessons for regulating AI (Bhargavi Ganesh and Shannon Vallor), police use of canid robots (Carolin Kemper and Michael Kolain), and - a new topic - planning for the end of life of algorithmic and robot systems (Elin Björling and Laurel Riek). The robots won't care, but the humans will be devastated.

Illustrations: Hanging out at We Robot with Boston Dynamics' "Spot".

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 12:39 PM | Permalink | Comments (0) | TrackBacks (0)

Why is robotics hard?

This was Bill Smart's kickoff on the first (workshop) day of this year's We Robot. It makes sense: We Robot is 11 years old, and if robots were easy we'd have them by now. The basic engineering difficulties are things he's covered in previous such workshops: 2021, 2019, 2018, 2016.

More to the point for this cross-the-technicians-with-the-lawyers event: why is making robots "ethical" hard? Ultimately, because the policy has to be translated into computer code, and as Smart and others explain, the translation demands an order of precision humans don't often recognize. Wednesday's workshops explored the gap between what a policy says and what a computer can be programmed to do. For many years, Smart has liked to dramatize this gap by using four people to represent a "robot" and assigning a simple task. Just try picking up a ball with no direct visual input by asking yes/no questions of a voltage-measuring sensor.

This year, in a role-playing breakout group, we were asked to redesign a delivery robot to resolve complaints in a fictional city roughly the size of Seattle. Injuries to pedestrians have risen since delivery robots arrived; the residents of a retirement community are complaining that the robots' occupation of the sidewalks interferes with their daily walks; and one companysends its delivery robot down the street ;past a restaurant while playing ads for its across-the-street competitor.

It's not difficult to come up with ideas for ways to constrain these robots. Ban them from displaying ads. Limit them to human walking speed (which you'll need to specify precisely). Limit the time or space they're allowed to occupy. Eliminate cars and reallocate road space to create zones for pedestrians, cyclists, public tranport, and robots. Require lights and sound to warn people of the robots' movements. Let people ride on the robots. (Actually, not sure how that solves any of the problems presented, but it sounds like fun.)

As you can see from the sample, many of the solutions that the group eventually proposed were only marginally about robot design. Few could be implemented without collaboration with the city, which would have to agree and pay for infrastructure changes or develop policies and regins specifying robot functionality.

This reality was reinfoced in a later exercise, in which Cindy Grimm, Ruth West, and Kristen Thomasen broke us into robot design teams and tasked us to design a robot to solve these complaints reinforced this. Most of the proposals involved reorganizating public space (one group suggested sending package delivery robots through the sewer system rather than on public streets and sidewalks), sometimes at considerable expense. Our group, concerned about sustainability, wanted the eventual robot made out of 3D printed engineered wood, but hit physical constraints when Grimm pointed out that our comprehensive array of sensors wouldn't fit on the small form factor we'd picked - and would be energy-intensive. No battery life.

The deeper problem we raised: why use robots for this at all? Unless you're a package delivery company seeking to cut labor costs, what's the benefit over current delivery systems? We couldn't think of one. With Canadian journalist Paris Marx's recent book on autonomous vehicles , Road to Nowhere fresh in my mind, however, the threat to publc ownership of the sidewalk seemed real.

The same sort of real problem surfaced in discussions of a different problem, based on Paige Tutosi's winning entry in a recent roboethics competition. In this exercise, we were given three short lists: rooms in a house, people who live in the house, and objects around the house. The idea was to come up with rules for sending the objects to individuals that could be implemented in computer code for its robot servant. In an example ruleset, no one can order the robot to send a beer to the baby or chocolate to the dog.

My breakout group quickly got stuck in contemplating the possible power dynamics and relationships in the house. Was the "mother" the superuser who operated in God mode? Or was she an elderly dementia patient who lived with her superuser daughter, her daughter's boyfriend, and their baby? Then someone asked the killer question: "Who is paying for the robot?" People whose benefits payments arrive on prepay credit cards with government-designed constraints on their use could relate.

The summary reports from the other groups revealed a significant split between those who sought to build a set of rules that specified what was forbidden (comparable to English or American law) and those who sought to build a set of rules that specified what was permitted (more like German law).

For the English approach, you have to think ahead of time of all the things that could go wrong and create rules to prevent them. This is by far the easier approach - easier to code, and safer for robot manufacturers seeking to limit their liability. Robots' capabilities will default to strictly limited to "known-safe".

The fact of this split suggested that at heart developing "robot ethics" is recapitulating all of legal history back to first principles. Viewed that way, robots are dangerous. Not because they are likely to attack us - but because they can be the vector for making moot, in stealth, by inches, and to benefit their empowered commissioners, our entire framework of human rights and freedoms.

Illustrations: Boston Dynamics' canine robot visits We Robot.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 3:51 AM | Permalink | Comments (0) | TrackBacks (0)

It's easy to say the future is hybrid. Much harder to say how to make it - and for "it" read "conferences and events" - hybrid and yet still a good experience for all involved.

I should preface this by stating the obvious: writers don't go to events the same way as other people. For one thing, our work is writing about what we find. So where a "native" (say, a lawyer at a conference on robots, policy, and law will be looking to connect their work to the work others at the conference are doing, the writer is always thinking, "Is that worth writing about?" or "Why is everyone excited about this paper?" You're also always looking around: who would be interesting to schmooze over the next lunch break?

For writers, then - or at least, *this* writer - attending remotely can be unsatisfying, more like reviewing a TV show. After one remote event last year, I approached a fellow attendee on Twitter and suggested a Zoom lunch to hash over what we'd just witnessed. She thought it was weird. In person, wandering up to join her lunchtime conversation would have been unremarkable. The need to ask makes people self-conscious.

And yet, there is a big advantage in being able to access many more events than you could ever afford to fly to. So I want hybrid events to *work*.

In a recent editorial note, a group of academic researchers set out guidelines and considerations for hybrid conferences, the result of discussions held in July 2021 at the Dagstuhl seminar on Climate Friendly Internet Research. They divide hybrid conferences into four types: passive (in which the in-person conference is broadcast to the remote audience, who cannot present or comment); semi-passive (in which remote participants can ask questions but not present or act as panelists); true (in which both local and remote participants have full access and capabilities); and distributed (in which local groups form clusters or nodes, which link together to form the main event).

I have encountered the first three of these (although I think the fourth holds a lot of promise). My general rule: the more I can participate as a remote attendee the better I like it and the more I feel like the conference must be joined in real time. A particular bugaboo is organizers who disable the chat window. At one in-person-only event this year, several panels were composed solely of remote speakers, who needed a technician's help to get audience feedback.

As the Dagstuhl authors write, hybrid events are not new. One of the organizations I'm involved with has enabled remote participation in council meetings for more than 15 years. At pre-pandemic meetings a telephone hookup and conference speaker provided dial-in access. Alongside, two of us typed into a live chat channel updates that both became the meeting's minutes and helped clarify what was being said and who was speaking. Those two also monitored the chat for remote participants who needed help being heard.

Folk music gatherings have developed practices that might be more broadly useful. For one thing, they set up many more breakout "rooms" than seems needed at first glance. One becomes the "parking lot" - a room where participants can leave their computer logged in, mic and camera off, so they can resume the session at any time without having to log in again. There's usually a "kitchen" or some such where people can chat with new and old friends. Every music session has both a music host and a technical assistant who keeps things running smoothly. And there is always an empty period following each session, so people can linger and the next session has ample set-up time. A lobby is continuously staffed by a host who helps incomers find the sessions they want and provides a point of contact if something is going wrong.

As both these examples suggest, enabling remote attendees to be full participants requires a lot of on-site support. In a discussion about this on Twitter, Jon Crowcroft, one of the note's authors, said, for example, that each in-person participant should also have a Zoom (or whatever) login so they could interact fully with remote participants, including accessing the chat window. I would second this. At a multi-track workshop earlier this year, some of the event's tracks were inaccessible because the room's only camera and microphone were poorly placed, making it impossible to see or understand commenters. At the end of each session the conference split in two; those of us on Zoom chatted to each other, while the in-person attendees wandered off to the room where the refreshments were. Crowcroft's recommendation would have helped a lot.

It's a lot of effort, but there is a big reason to do it, which the Dagstuhl authors also discuss: embracing diversity. The last two years have enabled all of us to gain contact with people who could never muster the funding or logistics to travel to distant events. Treating remote participants as an add-on sends the message that we're back to exclusionary business as previous normal. In locking us down, the pandemic also opened up much more of the world to participation. It would be wrong to close it back down again.

Illustrations: The second shot of the final episode of Better Call Saul (because I couldn't think of anything).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:37 PM | Permalink | Comments (0) | TrackBacks (0)

This week provided two examples of the dangers of believing too much hype about modern-day automated systems and therefore overestimating what they can do.

The first is relatively minor: Google employee Blake Lemoine published his chats with a bot called LaMDA and concluded it was sentient "basd on my religious beliefs". Google put Lemoine on leave and the press ran numerous (many silly) stories. Veterans shrugged and muttered, "ELIZA, 1966".

The second, however...

On Wednesday, the US National Highway Traffic Safety Administration released a report (PDF) studying crashes involving cars under the control of "driver-assist" technologies. Out of 367 such crashes in the nine months after NHTSA began collecting data in July 2021, 273 involved Teslas being piloted by either "full self-driving software" or its precursor, "Tesla Autopilot".

There are important caveats, which NTHSA clearly states. Many contextual details are missing, such as how many of each manufacturer's cars are on the road and the number of miles they've traveled. Some reports may be duplicates; others may be incomplete (private vehicle owners may not file a report) or unverified. Circumstances such as surface and weather conditions, or whether passengers were wearing seat belts, are missing. Manufacturers differ in the type and quantity of crash data they collect. Reports may be unclear about whether the car was equipped with SAE Level 2 Advanced Driver Assistance Systems (ADAS) or SAE Levels 3-5 Automated Driving Systems (ADS). Therefore, NTHSA says, "The Summary Incident Report Data should not be assumed to be statistically representative of all crashes." Still, the Tesla number stands out, far ahead of Honda's 90, which itself is far ahead of the other manufacturers listed.

SAE, ADAS, and ADS refer to the system of levels devised by the Society of Automotive Engineers (now SAE International) in 2016. Level 0 is no automation at all; Level 1 is today's modest semi-automated assistance such as cruise control, lane-keeping, and automatic emergency braking. Level 2, "partial automation", is now: semi-automated steering and speed systems, road edge detection, and emergency braking.

Tesla's Autopilot is SAE Level 2. Level 3 - which may someday include Tesla's Full Self Drive Capability - is where drivers may legitimately begin to focus on things other than the road. In Level 4, most primary driving functions will be automated, and the driver will be off-duty most of the time. Level 5 will be full automation, and the car will likely not even have human-manipulable controls.

Right now, in 2022, we don't even have Level 3, though Tesla CEO Elon Musk keeps promising we're on the verge of it with his company's Full Self-Drive Capability, its arrival always seems to be one to two years away. As long ago as 2015, Musk was promising Teslas would be able to drive themselves while you slept "within three years"; in 2020 he estimated "next year" - and he said it again a month ago. In reality, it's long been clear that cars autonomous enough for humans to check out while on the road are further away than they seemed five years ago, as British transport commentator Christian Wolmar accurately predicted in 2018.

Many warned that Levels 2 and 3 are would be dangerous. The main issue, pointed out by psychologists and behavorial scientists, is that humans get bored watching a computer do stuff. In an emergency, where the car needs the human to take over quickly, said human, whose attention has been elsewhere, will not be ready. In this context it's hard to know how to interpret the weird detail in the NTHSA report that in 16 cases Autopilot disengaged less than a second before the crash.

The NHTSA news comes just a few weeks after a New York Times TV documentary investigation examining a series of Tesla crashes. Some it links to the difficulty of designing software that can distinguish objects across the road - that is, the difference between a truck crossing the road and a bridge. In others, such as the 2018 crash in Mountain View, California, the NTSB found a number of contributing factors, including driver distraction and overconfidence in the technology - "automation complacence", as Robert L. Sumwalt calls it politely.

This should be no surprise. In his 2019 book, Ludicrous, auto industry analyst Edward Niedermeyer mercilessly lays out the gap between the rigorous discipline embraced by the motor industry so it can turn out millions of cars at relatively low margins with very few defects and the manufacturing conditions Niedermeyer observes at Tesla. The high-end, high-performance niche sports cars Tesla began with were, in Niedermeyer's view, perfectly suited to the company's disdain for established industry practice - but not to meeting the demands of a mass market, where affordability and reliability are crucial. In line with Nidermeyer's observations, Bloomberg Intelligence predicts that Volkswagen will take over the lead in electric vehicles by 2024. Niedermeyer argues that because it's not suited to the discipline required to serve the mass market, Tesla's survival as a company depends on these repeated promises of full autonomy. Musk himself even said recently that the company is "worth basically zero" if it can't solve self-driving.

So: financial self-interest meets the danger zone of Level 2 with perceptions of Level 4. I can't imagine anything more dangerous.

Illustrations: One of the Tesla crashes investigated in New York Times Presents.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:57 PM | Permalink | Comments (0) | TrackBacks (0)

In 1989, at 5-6 in the third and final set of the French Open women's singles final, 20-year-old Steffi Graf abruptly ran off-court. Soon afterwards, her opponent, Arantxa Sanchez-Vicario, completed one of the biggest upsets in the history of women's tennis.

Why did Graf do it? the press demanded to know in the post-march interview. When Graf finally (and slightly crankily) explained that she had her period. some journalists - Michael Mewshaw cites Italian Hall of Fame journalist Gianni Clerici for one - followed up by printing her (presumably imagined) menstrual cycle in the newspapers.

Mewshaw recounted this incident in June 2021 to illustrate the unpleasantness that can attend sports press conferences, in sympathy with Naomi Osaka. However, he could as easily have been writing about the commodification of athletes and their personal information. Graf got no benefit from journalists' prurient curiosity. But bettors, obsessive fans, and commentators could imagine they were being sold insight into her on-court performance. Ick.

This week, the Australian Science Academy launched a discussion paper on the use of athlete data in professional sport, chaired by Julia Powles and Toby Walsh. Powles and Walsh have also provided a summary at The Conversation.

The gist: the amount and variety of data collected about athletes has exploded using the justification of improving athletic performance and reducing injury risk. It's being collected and saved with little oversight and no clarity about how it's being used or who gets access to it; the overriding approach is to collect everything possible and save it in case a use is found. "It's rare for sports scientists and support staff to be able to account for it, and rarer still for sports governing bodies and athletes themselves," they write.

In the ASA's launch panel, Powles commented that athletes are "at the forefront of data gathering and monitoring", adding that such monitoring will eventually be extended to the rest of us as it filters from professional sports to junior sports, and onward from there.

Like Britain's intensively monitored children, athletes have little power to object: they have already poured years of their own and their family's resources into their obsession. Who would risk the chance of big wins to argue when their coach or team manager fits them with sensors tracking their sleep, heart rate, blood oxygenation, temperature, and muscle twitches and says it will help them? The field, Kathryn Henne observed is just an athlete's workplace.

In at least one case - the concussion in American football - data analysis has proved the risk to athletes. But, Powles noted, the report finds that it's really the aggregate counts that matter: how many meters you ran, not what your muscles were doing while you ran them. Much of the data being collected lies fallow, and no theory exists for testing its value.

Powles' particular concern is twofold. First, the report finds that the data is not flowing to sports scientists and others who really understand athletes (and therefore does not actually further the goal of helping them) but toward data scientists and other dedicated data-crunchers who have no expertise in sports science. Second, she deplores the resulting opportunity costs.

"What else aren't we spending money on?" she asked. Healthier environments and providing support are things we know work; why not pursue them instead of "technology dreams"? Her biggest surprise, she said, was discovering how cash-strapped most sports are. Even tennis: the stars make millions, but the lower ranks starve.

Professional athletes have always had to surrender aspects of their privacy in order to play their sport, beginning with the long, unpleasant history of gender testing, which began with men-only games in which competitors appeared nude, and continued in 1968 with requiring athletes wishing to compete in women's sports to prove they qualify. Then came anti-doping, which presumes everyone is guilty except when testing finds them innocent: urine tests under observation and blood tests for more sophisticated doping agents like EPO. In 2004, the anti-doping authorities initiated the "Whereabouts rule", which requires athletes to provide their location every day to facilitate no-notice out-of-competition testing. More recently, sporting authorities have begun collecting and storing blood and other parameters to populate the "athlete biological passport" with the idea that longitudinal profiling will highlight changes indicative of doping. An athlete who objects to any of this is likely to be publicly accused of cheating; sympathy is in short supply.

The report adds to those obvious invasions the ongoing blurring of the line between health data - which apparently is determined by the involvement of a doctor - and what the authors call "performance data". This was raised as an issue at the Privacy Health Summit back in 2014, where panelists noted that the range of sensitive data being collected by then-new Fitbits, sleep apps, and period trackers wasn't covered by the US health information law, HIPAA.

Athletes are the commodities in all this. It's not a big stretch to imagine the use of this data turning hostile, particularly as it extends to junior sports, where it can be notoriously difficult to pic future winners. Sports hold our interest because they provide the unexpected. Data-crunching by its nature tries to eliminate it. As Powles put it, "The story of sport is not just the runs and the goals." But that's what data can count.

Illustrations: Arantxa Sanchez-Vicario holding the 1989 French Open women's singles trophy.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:03 PM | Permalink | Comments (0) | TrackBacks (0)

One of the first things the magician and paranormal investigator James Randi taught all of us in the skeptical movement was the importance of consulting the right kind of expert.

Randi made this point with respect to tests of paranormal phenomena such as telekinesis and ESP. At the time - the 1970s and 1980s - there was a vogue for sending psychic claimants to physicists for testing. A fair amount of embarrassment ensued. As Randi liked to say, physicists, like many other scientists, are not experienced in the art of deception. Instead, they are trained to assume that things in their lab do not lie to them.

Not a safe assumption when they're trying to figure out how a former magician has moved an empty plastic film can a few millimeters, apparently with just the power of their mind. Put in a magician who knows how to set up the experiment so the claimant can't cheat, and *then* if the effect still occurs you know something genuinely weird is going on.

I was reminded of this reading this quote from Fabio Urbina, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins, writing in Nature: "When we think of drug discovery, we normally do not consider technology misuse potential. We are not trained to consider it, and it is not even required for machine learning research,"

The article itself is scary enough for one friend to react to it with, "This is the apocalypse". The researchers undertook a "thought experiment" after the Swiss Federal Institute for NBC Protection (Spiez Laboratory), asked theiir company, Collaborations Pharmaceuticals Inc, to provide a presentation on how their AI technology could be misused in drug discovery to its biennial conference on new technologies and their implications for the Chemical and Biological Weapons conventions. They work, they write, in an entirely virtual world; their molecules exist only in their computer. It had never previously occurred to them to wonder if the machine learning models they were building to help design new molecules that could be developed into new, life-saving drugs could be turned to generating toxins instead. Asked to consider it, they quickly discovered that it was disturbingly easy to generate prospective lethal neurotoxins. Because: generating potentially helpful molecules required creating models to *avoid* toxicity - which meant being able to predict its appearance.

As they go on to say, our general discussions of the potential harms AI can enable are really very limited. The biggest headlines go to putting people out of work; the rest is privacy, discrimination, fairness, and so on. Partly, that's because those are the ways AI has generally been most visible: automation that deskills or displaces humans, or algorithms that make decisions about government benefits, employment, education, content recommendations, or criminal justice outcomes. But also it's because the researchers working on this technology blinker their imagination to how they want their new idea to work.

The demands of marketing don't help. Anyone pursuing any form of research, whether funded by industry or government grant, has to make the case for why they should be given the money. So of course in describing their work they focus on the benefits. Those working on self-driving cars are all about how they'll be safer than human drivers, not scary possibilities like widespread hundred-car pileups if hackers were to find a way to exploit unexpected software bugs to make them all go haywire at the same time.

Sadly, many technology journalists pick up only the happy side. On Wednesday, as one tiny example, the Washington Post published a cheery article about EliiQ, an Alexa-like AI device "designed for empathy" meant to keep lonely older people company. The commenters saw more of the dark side than the writer did: ongoing $30 subscription, data collection and potential privacy invasion, and, especially, potential for emotional manipulation as the robot tells its renter what it (not she, as per writer Steven Zeitchik) calculates they want to hear.

It's not like this is the first such discovery. Malicious Generative Adversarial Networks (GANs) are the basis of DeepFakes. If you can use some new technology for good, why *wouldn't* you be able to use it for evil? Cars drive sick kids to hospitals and help thieves escape. Computer programmers write word processors and viruses, the Internet connects us directly to medical experts and sends us misinformation, cryptography protects both good and bad secrets, robots help us and collect our data. Why should AI be different?

I'd like to think that this paper will succeed where decades of prior experience have failed, and make future researchers think more imaginatively about how their work can be abused. Sadly, it seems a forlorn hope.

In Gemma Milne's 2020 book examining how hype interferes with our ability to make good decisions about new technology, Smoke and Mirrors, she warns that hype keeps us from asking the crucial question: Is this new technology worth its cost? Potential abuse is part of that cost-benefit assessment. We need researchers to think about what can go wrong a lot earlier in the development cycle - and we need them to add experts in the art of forecasting trouble (science fiction writers, perhaps?) to their teams. Even technology that looks like magic...isn't.

Illustrations: EliiQ (company PR photo).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 2:16 PM | Permalink | Comments (0) | TrackBacks (0)

The mythology goes like this. In the beginning, the Internet was decentralized. Then came money and Web 2.0, and they warped the best dreams of Web 2.0 into corporate giants. Now, web3 is going to restore the status ante?

Initial reaction: why will it be different this time?

Maybe it won't. Does that mean people shouldn't try? Ah. No. No, it does not.

One reason it's so difficult to write about web3 is that under scrutiny it dissolves into a jumble of decentralized web, cryptocurrencies, blockchain, and NFTs, though the Economist has an excellent explanatory podcast. Decentralizing the web I get: ever since Edward Snowden decentralization has been seen as a way to raise the costs of passive surveillance. The question has been: how? Blockchain and bitcoin sound nothing like the web - or a useful answer.

But even if you drop all the crypto stuff and just say "decentralized web to counter surveillance and censorship, it conveys little to the man on the Clapham omnibus. Try to explain, and you rapidly end up in a soup of acronyms that are meaningful only to technologists. In November, on first encountering web3, I suggested there are five hard problems. The first of those, ease of use, is crucial. Most people will always flock to whatever requires least effort; the kind of people who want to build a decentralized Internet are emphatically unusual. The biggest missed financial opportunity of my lifetime will likely have been ignoring the advice to buy some bitcoin in 2009 because it was just too much trouble. Most of today's big Internet companies got that way because whatever they were offering was better - more convenient, saved time, provided better results.

This week, David Rosenthal, developer of core Nvidia technologies, published a widely-discussed dissection of cryptocurrencies and blockchain, which Cory Doctorow followed quickly with a recap/critique. Tl;dr: web3 is already centralized, and blockchain and cryptocurrencies only pay off if their owners can ignore the external costs they impose on the rest of the world. Rosenthal argues that ignoring externalities is inherent in theSilicon Valley-type libertarianism from which they sprang.

Rosenthal also makes an appearance in the Economist podcast to explain that if you ask most people what the problems are with the current state of the Web, they don't talk centralization. They talk about overwhelming amounts of advertising, harassment, scams, ransomware, and expensive bandwidth. In his view, changing the technical infrastructure won't change the underlying economics - scale and network effects - that drive centralization, which, as all of these commentators note, has been the eventual result of every Internet phase since the beginning.

It's especially easy to be suspicious about this because of the venture capital money flooding in seeking returns.

"Get ready for the crash," Tim O'Reilly told CBS News. In a blog posting last December, he suggestshow to find the good stuff in web3: look for the parts that aren't about cashing out and getting rich fast but *are* about solving hard problems that matter in the real world.

This is all helpful in understanding the broader picture, but doesn't answer the question of whether there's presently meat inside web3. Once bitten, twice shy, three times don't be ridiculous.

What gave me pause was discovering that Danny O'Brien has gone to work for the Filecoin Foundation and the Filecoin Foundation for the Distributed Web - aka, "doing something in web3". O'Brien has a 30-year history of finding the interesting places to be. In the UK, he was one-half of the 1990s must-read newsletter NTK, whose slogan was "They stole our revolution. Now we're stealing it back." Filecoin - a project to develop blockchain-based distributed storage, which he describes as "the next generation of something like Bittorrent" - appears to be the next stage of that project. The mention of Bittorrent reminded how technologically dull the last few years have been.

O'Brien's explanation of Filecoin and distributed storage repeatedly evoked prior underused art that only old-timers remember. For example, in 1997 Cambridge security engineer Ross Anderson proposed the Eternity Service, an idea for distributing copies of data around the world so its removal from the Internet would be extremely difficult. There was Ian Clarke's 1999 effort to build such a thing, Freenet, a peer-to-peer platform for distributing data that briefly caused a major moral panic in the UK. Freenet failed to gain much adoption - although it's still alive today - because no one wanted to risk hosting unknown caches of data. Filecoin intends to add financial economic incentives: think a distributed cloud service.

O'Brien's mention of the need to ensure that content remains addressable evokes Ted Nelson's Project Xanadu, a pre-web set of ideas about sharing information. Finally, zero-knowledge proofs make it possible to show a proof that you have run a particular program and gotten back a specific result without revealing the input. The mathematics involved is arcane, but the consequence is far-reaching: you can prove results *and* protect privacy.

If this marriage of old and new research is "web3", suddenly it sounds much more like something that matters. And it's being built, at least partly, by people who remember the lessons of the past well enough not to repeat them. So: cautious signs that some part of "web3" will do something.

Illustrations: Diagram of centralized vs decentralized (IPFS) systems (from zK Capital at Medium).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:53 PM | Permalink | Comments (0) | TrackBacks (0)

I have a friend I would like to lend anyone who thinks computers have gotten easier in the last 30 years.

The other evening, he asked how to host a Zoom conference. At the time, we were *in* a Zoom call, and I've seen him on many others, so he seemed competent enough.

"Di you have a Zoom account?" I said.

"How do I get that?"

I directed him to the website. No, not the window with our faces; that's the client. "Open up - what web browser do you use?"

"Er...Windows 10?"

"That's the computer's operating system. What do you use to go to a website?"

"Google?"

Did he know how to press ALT-TAB to see the open windows on his system? He did not. Not even after instruction.

But eventually he found the browser, Zoom's website, and the "Join" menu item. He created a password. The password didn't work. (No idea.) He tried to reset the password. More trouble. He decided to finish it later...

To be fair, computers *have* gotten easier. On a 1992 computer, I would have had to write my friend a list of commands to install the software, and he'd have had to type them perfectly every time and learn new commands for each program's individual interface. But the comparative ease of use of today's machines is more than offset by the increased complexity of what we're doing with them. It would never have occurred to my friend even two years ago that he could garnish his computer with a webcam and host video chats around the world.

I was reminded of this during a talk on new threats to privacy that touched on ubiquitous computing and referenced the 1991 paper The Computer for the 21st Century, by Marc Weiser, then head of the famed Xerox PARC research lab.

Weiser imagined the computer would become invisible, a theme also picked up by Donald Norman in his 1998 book, The Invisible Computer. "Invisible" here means we stop seeing it, even though it's everywhere around us. Both Weiser and Norman cited electric motors, which began as large power devices to which you attached things, and then disappeared inside thousands of small and large appliances. When computers are everywhere, they will stop commanding our attention (except when they go wrong, of course). Out of sight, out of mind - but in constant sight also means out of mind because our brains filter out normal background conditions to focus on the exceptional.

Weiser's group built three examples, which they called tabs (inch-scale), pads (foot-scale), and boards (yard-scale). His tabs sound rather like today's tracking tags. Like the Active Badges at Olivetti Research in Cambridge they copied (the privacy implications of which horrified the press at the time), they could be used to track people and things, direct calls, automate diary-keeping, and make presentations and research portable throughout the networked area. In 2013, when British journalist Simon Bisson revisited this same paper, he read them more broadly as sensors and effectuators. Pads, in Weiser's conception, were computerized sheets of "scrap" paper to be grabbed and used anywhere and left behind for the next person. Weiser called them an "antidote to windows", in that instead of cramming all programs into a window you could spread dozens of pads across a full-sized desk (or floor) to work with. Boards were displays, more like bulletin boards, that could be written on with electronic "chalk" and shared across rooms.

"The real power of the concept comes not from any one of these devices; it emerges from the interaction of all of them," Weiser wrote.

In 2013, Bisson suggested Weiser's "embodied virtuality" was taking shape around us as sensors began enabling the Internet of Things and smartphones became the dominant interface to the Internet. But I like Weiser's imagined 21st century computing better than what we actually have. While cloud services can make our devices more or less interchangeable as long as we have the right credentials, that only works if broadband is uninterruptedly reliable. But even then, has anyone lost awareness of the computer - phone - in their hand or the laptop on their desk? Compare today to what Weiser thought would be the case 20 years later - which would have been 2011:

Most important, ubiquitous computers will help overcome the problem of information overload. There is more information available at our fingertips during a walk in the woods than in any computer system, yet people find a walk among trees relaxing and computers frustrating. Machines that fit the human environment, instead of forcing humans to enter theirs, will make using a computer as refreshing as taking a walk in the woods.

Who feels like that? Certainly not the friend we began with. Even my computer expert friends seem one and all convinced that their computers hate them. People in search of relaxation watch TV (granted, maybe on a computer), play guitar (even if badly), have a drink, hang with friends and family, play a game (again, maybe on a computer), work out, tale a bath. In fact, the first thing people do when they want to relax is flee their computers and the prying interests that use them to spy on us. Worse, we no longer aspire to anything better. Those aspirations have all been lost to A/B testing to identify the most profitable design.

Illustrations: Windows XP's hillside wallpaper (via Wikimedia).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 4:02 PM | Permalink | Comments (0) | TrackBacks (0)

Web3, the push to decentralize the net, got a lot more attention this week after the venture capital firm Andreesen Horowitz published guidance for policy makers - while British software engineer Stephen Diehl to blogged calling web3 "bullshit", a "vapid marketing campaign", and a "rhetorical trick" (thanks to Mike Nelson for the pointer).

Here, a month ago, we tried to tease out some of the hard problems web3 is up against. Diehl attacks the technical basis, citing the costs of the computation and bandwidth necessary to run a censorship-proof blockchain network, plus the difficulty of storage, as in "who owns the data?". In other words, web3, as he understands it, won't scale.

Meanwhile, on Twitter, commenters have highlighted Andreesen Horowitz's introductory words, "We are radically optimistic about the potential of web3 to restore trust in institutions and expand access to opportunity." If, the argument goes, venture capitalists are excited about web3 that's a clear indicator that they expect to reap the spoils. Which implies an eventual outcome favoring giant corporate interests.

The thing that modern venture capitalists always seek with (due) diligence is scale. Scale means you can make more of something without incurring (much) additional cost. Scale meant Instagram could build a business Facebook would buy for $1 billion with only 13 employees. Venture capitalists want the hockey stick.

Unsurprisingly, given the venture capital appeal, the Internet is full of things that scale - social media sites, streaming services, software, other forms of digital content distribution, and so on. Yet many of the hard problems we struggle to solve are conflicts between scale and all the things on the Internet that either *don't* scale. Easy non-Internet example: viruses scale, nurses don't. Or, more nettishly, facial recognition scales; makeup artists don't. And so on.

An obvious and contentious Internet example: content moderation. Even after AI has automatically removed the obvious abuses, edge cases rapidly escalate beyond the resources most companies are willing to throw at it. In his book Social Warming, Charles Arthur suggests capping the size of social networks, an idea echoed recently by Lawfare editor Ben Wittes in an episode of In Lieu of Fun, who commented that sites shouldn't be allowed to grow larger than they can "moderate well". It's hard to think of a social media site that hasn't. It's also hard to understand how such a cap would work without frustrating everyone. If you're user number cap+1, do you have to persuade all your friends to join a less-populated network so you can be together?

More broadly - a recurrent theme - community on the Internet does not scale. In every form of online community back to bulletin board systems and Usenet, increasing size always brings abuse. In addition, over and over online forums show the power law distribution of posters: a small handful do most of the talking, followed by a long tail of occasional contributors and a vast majority of lurkers. The loudest and most persistent voices set the tone, get the attention, and reap the profits, if there are any to be had.

The problem of scaling content moderation applies more generally to online governance. As societies grow, become more complex, and struggle with abuse, turning governance over to paid professionals seems to be the near-universal solution.

Another thing that doesn't scale: discovery, as Benedict Evans recently pointed out in a discussion of email newsletters and Substack.

One of the marvels of 2021 has been the reinvention of emailed newsletters as a paying proposition. Of course, plenty of people were making *some* money from such things way back even before email. But this year has taken it to a new level. People are signing six-figure deals with Substack and giving up ordinary journalism gigs and book deals to do it.

Evans points out that in newsletters, as in previous Internet phenomena - podcasts, web pages (hence search engines, and ecommerce (hence aggregation) - the first people who show up in an empty space with good stuff people want do really well. We don't hear so much any more about first-mover advantage, but it often still applies.

Non-fungible tokens (NFTs) may be the latest example. A few very big paydays are drawing all sorts of people into the field. Some will profit, but many more will not. Meanwhile, scams and copyright and other issues are proliferating. Even if regulation eventually makes participation safer, the problem will remain: people have limited resources to spend on such things, and the field will be increasingly crowded.

So, too, Substacks and newsletters: there are not only limits to how many subscriptions people can afford, but also to how many things they have time to read. In a crowded field, discovery is everything.

Individuals' attention spans and financial resources do not scale. The latter is one reason the pay-with-data model has been so successful on the web; the former is part of why people will sacrifice privacy and participatory governance in favor of convenience.

So, our partial list of things that do not scale: content moderation, community, discovery, governance. Maybe also security to some extent. In general: anything that requires human labor to be added proportionately to its expansion. Incorporating solving problems of scale will matter if we're going to have a different outcome from web3 than from previous iterations.

Illustrations: A hockey stick.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 4:12 PM | Permalink | Comments (0) | TrackBacks (0)

How do democracy and algorithmic governance live together? This was the central question of a workshop this week on computational governance. This is only partly about the Internet; many new tools for governance are appearing all the time: smart contracts, for example, and AI-powered predictive systems. Many of these are being built with little idea of how they can go wrong.

The workshop asked three questions:

- What can technologists learn from other systems of governance?
- What advances in computer science would be required for computational systems to be useful in important affairs like human governance?
- Conversely, are there technologies that policy makers can use to improve existing systems?

Implied is this: who gets to decide? On the early Internet, for example, decisions were reached by consensus among engineers, funded by hopeful governments, who all knew each other. Mass adoption, not legal mandate, helped the Internet's TCP/IP protocols dominate over many other 1990s networking systems: it was free, it worked well enough, and it was *there*. The same factors applied to other familiar protocols and applications: the web, email, communications between routers and other pieces of infrastructure. Proposals circulated as Requests for Comments, and those that found the greatest acceptance were adopted. In those early days, as I was told in a nostalgic moment at a conference in 1998, anyone pushing a proposal because it was good for their company would have been booed off the stage. It couldn't last; incoming new stakeholders demanded a voice.

If you're designing an automated governance system, the fundamental question is this: how do you deal with dissenting minorities? In some contexts - most obviously the US Supreme Court - dissenting views stay on the record alongside the majority opinion. In the long run of legal reasoning, it's important to know how judgments were reached and what issues were considered. You must show your work. In other contexts where only the consensus is recorded, minority dissent is disappeared - AI systems, for example, where the labelling that's adopted is the result of human votes we never see.

In one intriguing example, a panel of judges may rule a defendant is guilty or not guilty depending on whether you add up votes by premise - the defendant must have both committed the crime and possessed criminal intent - or by conclusion, in which each judge casts a final vote and only these are counted. In a small-scale human system the discrepancy is obvious. In a large-scale automated system, which type of aggregation do you choose, and what are the consequences, and for whom?

Decentralization poses a similarly knotty conundrum. We talk about the Internet's decentralized origins, but its design fundamentally does not prevent consolidation. Centralized layers such as the domain name system and anti-spam blocking lists are single points of control and potential failure. If decentralization is your goal, the Internet's design has proven to be fundamentally flawed. Lots of us have argued that we should redecentralize the Internet, but if you adopt a truly decentralized system, where do you seek redress? In a financial system running on blockchains and smart contracts, this is a crucial point.

Yet this fundamental flaw in the Internet's design means that over time we have increasingly become second-class citizens on the Internet, all without ever agreeing to any of it. Some US newspapers are still, three and a half years on, ghosting Europeans for fear of GDPR; videos posted to web forums may be geoblocked from playing in other regions. Deeper down the stack, design decisions have enabled surveillance and control by exposing routing metadata - who connects to whom. Efforts to superimpose security have led to a dysfunctional system of digital certificates that average users either don't know is there or don't know how to use to protec themselves. Efforts to cut down on attacks and network abuse have spawned a handful of gatekeepers like Google, Akamai, Cloudflare, and SORBS that get to decide what traffic gets to go where. Few realize how much Internet citizenship we've lost over the last 25 years; in many of our heads, the old cooperative Internet is just a few steps back. As if.

As Jon Crowcroft and I concluded in our paper on leaky networks for this year's this year's Gikii, "leaky" designs can be useful to speed development early on even though they pose problems later, when issues like security become important. The Internet was built by people who trusted each other and did not sufficiently imagine it being used by people who didn't, shouldn't, and couldn't. You could say it this way: in the technology world, everything starts as an experiment and by the time there are problems it's lawless.

So this the main point of the workshop: how do you structure automated governance to protect the rights of minorities? Opting to slow decision making to consider the minority report impedes decision making in emergencies. If you limit Internet metadata exposure, security people lose some ability to debug problems and trace attacks.

We considered possible role models: British corporate governance; smart contracts;and, presented by Miranda Mowbray, the wacky system by which Venice elected a new Doge. It could not work today: it's crazily complex, and impossible to scale. But you could certainly code it.

Illustrations: Monument to the Doge Giovanni Pesaro (via Didier Descouens at Wikimedia).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 2:26 PM | Permalink | Comments (0) | TrackBacks (0)

Last week, Gikii speakers pondered whether regulating magic could suggest how to regulate AI. This week, Woody Hartzog led a session at We Robot pondering how to regulate robots, and my best analogy was...old cars.

Bill Smart was explaining that "robot kits" wouldn't become a thing because of the complexity. Even the hassock-sized Starship delivery robot spotted on a Caltrain platform and deliver groceries in Milton Keynes are far too complex for a home build. "Like a car. There are no car kits."

Oh, yes, there are: old cars, made before electronics, that can be taken to pieces and rebuilt; you just need the motor vehicle people to pass it as roadworthy. See also: Cuba.

Smart's main point stands, though: the Starship robots have ten cameras, eight ultrasonic sensors, GPS, and radar, and that's just the hardware (which one could imagine someone plugging together). The software includes neural nets, 3D mapping, and a system for curb climbing, plus facilities to allow remote human operation. And yet, even with all that one drove straight into a canal last year.

"There's a tendency seen in We Robot to think about a robot as a *thing* and to write around that thing," Cindy Grimm observed. Instead, it's important to consider what task you want the robot to accomplish, what it's capable of, what it *can't* do, and what happens when someone decides to use it differently. Starship warns not to disturb its robots if they're sitting doing nothing. "It may just be having a rest."

A rest? To do what? Reorder the diodes all down its left side?

The discussion was part of a larger exercise in creating a law to govern delivery robots and trying to understand tradeoffs. A physical device that interacts with thea real world is, as Smart and Grimm have been saying all the way back to the first We Robot, in 2012, dramatically different from the devices we've sought to regulate so far. We tend, like the Starship people above, to attribute intentionality to things that can move, I believe as a matter of ancestral safety: things that can move autonomously can attack you. Your washing machine is more intelligent than your Roomba, but which one gets treated like a pet?

Really, though, Grimm said, "They're just a box of 1s and 0s."

So Hartzog began with a piece of proposed legislation. Posit: small delivery robots that uses sidewalks, roads, and bike lanes. Hypothetical city council doesn't want to ban outright. But the things can disrupt daily lives and impede humans' use of public space. So, they propose a law: delivery robots must have a permit, Must respect all city ordinances and physical safety of all people. Speed limited to 15 miles an hour. No contact with humans except the designated recipient. Must remain 12 feet apart and prioritize human mobility by moving away from assistive devices and making its presence known via audio signals. Only allowed to collect data for core functions; may not collect data from inside homes without consent; may not use facial recognition, only face detection for safety. What's missing?

Well, for one thing, 15 miles an hour is *dangerous* on a crowded sidewalk, and even in some bike lanes. For another, what capabilities does the robot need to recognize the intended recipient? Facial recognition? Fingerprint scanner? How much do permits cost and who can (and can't) afford them? Is it better to limit robot density rather than set a specific number? How does it recognize assistive devices? How much noise should we tolerate? Who has right of way if there's only one narrow path? If every robot's location must be known at all times, what are the implications of all that tracking? How and when do permits get revoked?

Hartzog left us with a final question: "Is the juice worth the squeeze?" Are there opportunity costs inherent in accepting the robots in the first place?

As Grimm said, nothing is for free; every new robot capability brings tradeoffs. Adding awareness, so the robot "knows" to move out of the way of strollers and wheelchairs, means adding data-gathering sensors, adding privacy risk? Grimm's work with apple-picking robots has taught her their success depends on pruning apple trees to make their task simpler. The job is a lot harder in her backyard, where this hasn't been done. So legal considerations must include how and whether we change the environment so it's safer for robots to be around people. Grimm calls this making a "tunnel" for the robot; narrow and simplify the task rather than making the robot "smarter".

Personally, I like the idea of barring the robots from weighing more than an average human can lift, so you can always pick the thing up and move it out of the way.

No such issues mar the cheery Starship promotional video linked above. This seems impossible; why should delivery robots be less of a nuisance than abandoned dockless scooters and bikes? In the more realistic view to be found in Anywhere But Westminster's 2019 visit to Milton Keynes, the robots still seem mostly inoffensive as they roll through an unpopulated park and wait to cross the empty street. Then the filmmakers encounter one broadcasting Boris Johnson speeches. Suddenly, ad-spewing sidewalk robots seem inescapable. Maybe instead hire the unemployed people the filmmakers find at the food bank?

Illustrations: Screenshot from Anywhere But Westminster, "We must deliver: Brexit, Johnson, and the robots of Milton Keynes".

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 12:30 PM | Permalink | Comments (0) | TrackBacks (0)

One part of our brains knows that software can be fragile. Another part of our brains, when faced with the choice of trusting the human or trusting the machine...trusts the machine. It may have been easier to pry trust away from the machine twenty years ago, when systems crashed more often, sometimes ruining months of work and the mantra, "Have you tried turning it off and back on again?" didn't yet work as a reliable way of restoring function. Perhaps more important, we didn't *have* to trust software because we had canonical hard copies. Then, as predicted, the copies became "backups". Now, often, they don't exist at all, with the result that much of what we think we know is becoming less well-attested. How many of us even print out our bank statements any more? Three recent stories highlight this.

First is the biggest UK computer-related scandal for many years, the outrageous Post Office prosecution of hundreds of subpostmasters of theft and accounting fraud, all while insisting that their protests of innocence must all be lies because its software, sourced from Fujitsu, could not possibly be wrong. Eventually, the Court of Appeal quashed 39 convictions and excoriated both the Post Office and Fujitsu for denying the existence of two known bugs that led to accounting discrepancies. They should never have been able to get away with their claim of infallibility - first, because generations of software engineers could have told the court that all software has bugs, and second, because Ross Anderson's work proving that software vulnerabilities were the cause of phantom ATM withdrawals, overriding the UK banking industry's insistence that its software, too, was infallible.

At Lawfare, Susan Landau, discussing work she did in collaboration with Steve Bellovin, Matt Blaze, and Brian Owsley. uses the Post Office fiasco as a jumping-off point to discuss the increasing problem of bugs in software used to produce evidence presented in court. Much of what we think of as "truth" - Breathalyzer readings, forensic tools, Hawkeye line calls in tennis matches - are not direct measurements but software-derived interpretations of measurements. Hawkeye at least publishes its margin for error even though tennis has decided to pretend it doesn't exist. Manufacturers of evidence-producing software, however, claim commercial protection, leaving defendants unable to challenge the claims being made about them. Landau and her co-authors conclude that courts must recognize that they can't assume the reliability of evidence produced bysoftware and that defendants must be able to conduct "adversarial audits".

Second story. At The Atlantic, Jonathan Zittrain complains that the Internet is "rotting". Link rot - broken links when pages get deleted or reorganized - and content drift, which sees the contents of a linked page change over time, are familiar problems for anyone who posts anything online. Gabriel Weinberg, the founder of search engine DuckDuckGo, has has talked about API rot, which breaks dependent functionality. Zittrain's particular concern is legal judgments, which increasingly may incorporate disappeared or changed online references like TikTok videos and ebooks. Ebooks in particular can be altered on the fly, leaving no trace of that thing you distinctly remember seeing.

Zittrain's response has been to help create sites to track these alterations and provide permanent links. It probably doesn't matter much that the net.wars archive has (probably) thousands of broken links. As long as the Internet Archive's Wayback Machine continues to exist as a source for vaped web pages, most of the ends of those links can be recovered. The Archive is inevitably incomplete, and only covers the open web. But it *does* matter if the basis for a nation's legal reasoning and precedents - what Zittrain calls "long-term writing" - can't be established with any certainty. Hence the enormous effort put in by the UK's National Archives to convert millions of pages of EU legislation so all could understand the legitimacy of post-Brexit UK law.

Third story. It turns out the same is true for the brick-by-brick enterprise we call science. In the 2020 study Open is not forever, authors Mikael Laakso, Lisa Matthias, and Najko Jahn find journal rot. Print publications are carefully curated and preserved by librarians and archivists, as well as the (admittedly well-funded) companies that publish them. Open access journals, however, have had a patchy record of success, and the study finds that between 2000 and 2019 174 open access journals from all major research disciplines and from all geographical regions vanished from the web. In science, as in law, it's not enough to retain the end result; you must be able to show your work and replicate your reasoning.

It's more than 20 years since I heard experts begin to fret about the uncertain durability of digital media; the Foundation for Information Research included the need for reliable archives in its 1998 founding statement. The authors of the journal study note that the journals themselves are responsible for maintaining their archives and preserving their portion of the scholarly record; they conclude that solving this problem will require the participation of the entire scholarly community.

What isn't clear, at least to me, is how we assure the durability of the solutions. It seemed a lot easier when it was all on paper in a reassuringly solid building.

Illustrations: The UK National Archives, in Kew (photo by Erian Evans via Wikimedia)..

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 12:20 PM | Permalink | Comments (0) | TrackBacks (0)

It is shameful that the UK government has apparently refused to learn anything over decades of these discussions, and is now ordering GPs in England to send their patient data to NHSx beginning on July 1 and continuing daily thereafter. GPs are unhappy about this. Patients - that is, the English population - have until June 23 to opt out. Government information has been so absent that if it were not for medConfidential we might not even know it was happening. The opt-out process is a dark pattern; here's how.

The pandemic has taught us a lot about both upsides and downsides of sharing information. The downside is the spread of covid conspiracy theories, refusal to accept public health measures, and death threats to public health experts.

But there's so much more upside. The unprecedented speed with which we got safe and effective vaccinations was enormously boosted by the Internet. The original ("ancestral") virus was genome-sequenced and shared across the world within days, enabling everyone to get cracking. While the heavy reliance on preprint servers meant some errors have propagated, rapid publication and direct access to experts has done far more good than harm overall.

Crowdsourcing is also proving its worth: by collecting voluntary symptom and test/vaccination status reports from 4.6 million people around the UK, the Covid Symptom Study, to which I've contributed daily for more than a year, has identified additional symptoms, offered early warning of developing outbreaks, and assessed the likelihood of post-vaccination breakthrough covid infections. The project is based on an app built by the startup Joinzoe in collaboration with 15 charities and academic research organizations. From the beginning it has seemed an obviously valuable effort worth the daily five seconds it takes to report - and worth giving up a modest amount of data privacy for - because the society-wide benefit is so obvious. The key points: the data they collect is specific, they show their work and how my contribution fits in, I can review what I've sent them, and I can stop at any time. In the blog, the project publishes ongoing findings, many of which have generated journal papers for peer review.

The government plans meet none of these criteria. The data grab is comprehensive, no feedback loop is proposed, and the subject access rights enshrined in data protection law are not available. How could it be more wrong?

Established in 2019, NHSx is the "digital arm" of the National Health Service. It's the branch that commissioned last year's failed data-collecting contact tracing app ("failed", as in many people correctly warned that their centralized design was risky and wouldn't work,). NHSx is all data and contracts. It has no direct relationship with patients, and many people don't know it exists. This is the organization that is demanding the patient records of 56 million people, a policy Ross Anderson dates to 1992.

If Britain has a national religion it's the NHS. Yes, it's not perfect, and yes, there are complaints - but it's a lot like democracy: the alternatives are worse. The US, the only developed country that has refused a national health system, is near-universally pitied by those outside it. For those reasons, no politician is ever going to admit to privatizing the NHS, and most citizens are suspicious, particularly of conservatives, that this is what they secretly want to do.

Brexit has heightened these fears, especially among those of us who remember 2014, when NHS England announced care.data, a plan to collect and potentially sell NHS patient data to private companies. Reconstructing the UK's economy post-EU membership has always been seen as involving a trade deal with the US, which is likely to demand free data flows and, most people believe, access to the NHS for its private medical companies. Already, more than 50 GPs' practices (1%) are managed by Operose, a subsidiary of US health insurer Centene. The care.data plan was rapidly canceled with a promise to retreat and rethink.

Seven years later, the new plan is the old plan, dusted off, renamed, and expanded. The story here is the same: it's not that people aren't willing to share data; it's that we're not willing to hand over full control. The Joinzoe app has worked because every day each contributor remakes the decision to participate and because the researchers provide a direct feedback loop that shows how the data is being used and the results. NHSx isn't offering any of that. It is assuming the right to put our most sensitive personal data into a black box it owns and controls and keep doing so without granting us any feedback or recourse. This is worse than advertisers pretending that we make free choices to accept tracking. No one in this country has asked for their relationship with their doctor to be intermediated by a bunch of unknown data managers, however well-meaning. If their case for the medical and economic benefits is so strong (and really, it is, *when done right*), why not be transparent and open about it?

The pandemic has made the case for the value of pooling medical data. But it has also been a perfect demonstration of what happens when trust seeps out of a health system - as it does when governments feudally treat citizens as data serfs. *Both* lessons should be learned.

Illustrations: Asklepios, Greek god of medicine.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:19 PM | Permalink | Comments (0) | TrackBacks (0)

The spring of 2021 is all about "last year at this time..."

Last year at this time I was frantically helping Privacy International track covid responses as every country passed laws, mandated curfews and other restrictions, and awarded police enforcement powers. It transpires that lots of others were thinking similarly about the need to track the different policies countries were pursuing to contain the pandemic. Tracking surveillance is important, if only to ensure that we can identify new intrusions inwhatever "new normal" emerges post-pandemic, but there are many other lessons to learn from comparisons across the world in this global, uncontrolled experiment in public health we call a pandemic.

One group doing this sort of study recently reported initial findings based on collected contributions from 600 data collectors in 186 countries comparing policies, vaccine rollouts, and so on. Most of it is negative, listing what *didn't* make a difference: the wealth of nations hasn't counted for much, nor did a nation's scientific and health care capacity by itself, and neither did the democratic or autocratic nature of a country's government.

The instinctive answer to this sort of question is that the countries that had recent experience of epidemics - HN1, SARS, MERS - understood the seriousness of the danger and how quickly exponential - viral! - growth can get out of hand and responded quickly and decisively. In some cases, such as New Zealand (26 deaths in total to April 8, 2021 for a rate of five per million population), Taiwan (10. 0.4), Vietnam (35, 0.4), China (4,636, 3), and Mongolia (18) that response has been effective over the last year (all figures from Worldometers. Other countries, such as the Czech Republic, Hungary, and Bulgaria, responded quickly to the initial threat, but then failed to use that experience to reimpose the measures that had worked for them when new waves of the pandemic arose. Some of the countries that were slow and/or chaotic about imposing lockdowns and other restrictions are at the front of vaccine rollouts. Mongolia, which has done fantastically well at curbing both case numbers and deaths, especially given its limited resources, by starting as early as January 2020 to implement low-tech public health measures, has abruptly seen its daily case load spike from 65 cases a day to 500 in the last month. The prime minister has announced a lockdown.

Eventually those doing this kind of work will be able to look across the entire experience of the pandemic to assess what worked and what didn't and make recommendations for more effective responses in future. That won't, however, be possible for at least another two years. In the meantime, some thoughts.

Even with this many countries to compare, the number of variables is staggering. Are you aiming to minimize deaths, contain costs, ensure your health care system doesn't collapse, or minimize spread? In one sense those are all the same: limit infection and the rest follows. But take vaccines, still patchily available, which goal you prioritize changes who gets protected first: minimizing deaths and hospital admissions means starting with the oldest, as in the US and UK, while minimizing spread might mean first targeting the most mobile, 20- and 30-somethings. If you're a scientist you develop models you hope resemble reality that let you see the results of different strategies. If you're a politician in the US or UK, you might be tempted to remember that old people vote.

One thing I think will become (even more) clear in hindsight is the tension in many countries between the magnetic attraction of new information technology approaches such as AI and risk modeling and the plodding effectiveness of low-tech public health approaches. It's very tempting to view this as the difference between "male" and "female" approaches, particularly because washing your hands, avoiding people when you're sick, and opening a window are the kinds of things your mother might have told you to do, and you're a grown-up now. It's more likely, though, that the last 50 years of high-speed computer industry developments have left us too inclined to think newer must be better.

In some cases this is true; certainly, the new mRNA vaccines seem set to achieve numerous breakthroughs against diseases that have proved recalcitrant until now, and without the Internet we would have been far more isolated and scientists could never have responded so quickly and so collaboratively. But in others our modern gizmos have been largely a distraction. Remember last year's mad rush to develop contact tracing apps? That whole controversial effort appears to have been largely pointless. The UK's app has been downloaded 21 million times...and so what? The country still has one of the highest covid death rates in the world (fifth in number of deaths, 13th in deaths per million population). As Jonny Ball and Michael Goodier write at New Statesman, contact tracing apps help provide manual contact tracers with leads to work with; they are no substitute for robust local laborious effort.

So my guess is that when the dust settles key advice will be that you can't automate your way out of a pandemic. Last year around this time, US Speaker of the House Nancy Pelosi said, "...And for those who choose prayer over science, I say that science is the answer to our prayers." Science, yes, but not just *new* science.

Illustrations: A visualization of the Stringency Index from the Covid-19 Government Response Tracker project.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 10:25 AM | Permalink | Comments (0) | TrackBacks (0)

One of the longest-running conflicts on the Internet surrounds whether and what restrictions should be applied to the content people post. These days, those rules are known as "platform governance", and this week saw the first conference by that name. In the background, three of the big four CEOs returned to Congress for more questioning, the EU is planning the Digital Services Act; the US looks serious about antitrust action, and debate about revising Section 230 of the Communications Decency Act continues even though few understandwhat it does; and the UK continues to push "online harms.

The most interesting thing about the Platform Governance conference is how narrow it makes those debates look. The second-most interesting thing: it was not a law conference!

For one thing, which platforms? Twitter may be the most-studied, partly because journalists and academics use it themselves and data is more available; YouTube, Facebook, and subsidiaries WhatsApp and Instagram are the most complained-about. The discussion here included not only those three but less "platformy" things like Reddit, Tumblr, Amazon's livestreaming subsidiary Twitch, games, Roblox, India's ShareChat, labor platforms UpWork and Fiverr, edX, and even VPN apps. It's unlikely that the problems of Facebook, YouTube, and Twitter that governments obsess over are limited to them; they're just the most visible and, especially, the most *here*. Granting differences in local culture, business model, purpose, and platform design, human behavior doesn't vary that much.

For example, Jenny Domino reminded - again - that the behaviors now sparking debates in the West are not new or unique to this part of the world. What most agree *almost* happened in the US on January 6 *actually* happened in Myanmar with far less scrutiny despite a 2018 UN fact-finding mission that highlighted Facebook's role in spreading hate. We've heard this sort of story before, regarding Cambridge Analytica. In Myanmar and, as Sandeep Mertia said, India, the Internet of the 1990s never existed. Facebook is the only "Internet". Mertia's "next billion users" won't use email or the web; they'll go straight to WhatsApp or a local or newer equivalent, and stay there.

Mehitabel Glenhaber, whose focus was Twitch, used it to illustrate another way our usual discussions are too limited: "Moderation can escape all up and down the stack," she said. Near the bottom of the "stack" of layers of service, after the January 6 Capitol invasion Amazon denied hosting services to the right-wing chat app Parler; higher up the stack, Apple and Google removed Parler's app from their app stores. On Twitch, Glenhaber found a conflict between the site's moderatorial decision the handling of that decision by two browser extensions that replace text with graphics, one of which honored the site's ruling and one of which overturned it. I had never thought of ad blockers as content moderators before, but of course they are, and few of us examine them in detail.

Separately, in a recent lecture on the impact of low-cost technical infrastructure, Cambridge security engineer Ross Anderson also brought up the importance of the power to exclude. Most often, he said, social exclusion matters more than technical; taking out a scammer's email address and disrupting all their social network is more effective than taking down their more easily-replaced website. If we look at misinformation as a form of cybersecurity challenge - as we should, that's an important principle.

One recurring frustration is our general lack of access to the insider view of what's actually happening. Alice Marwick is finding from interviews that members of Trust and Safety teams at various companies have a better and broader view of online abuse than even those who experience it. Their data suggests that rather than being gender-specific harassment affects all groups of people; in niche groups the forms disagreements take can be obscure to outsiders. Most important, each platform's affordances are different; you cannot generalize from a peer-to-peer site like Facebook or Twitter to Twitch or YouTube, where the site's relationships are less equal and more creator-fan.

A final limitation in how we think about platforms and abuse is that the options are so limited: a user is banned or not, content stays up or is taken down. We never think, Sarita Schoenebeck said, about other mechanisms or alternatives to criminal justice such as reparative or restorative justice. "Who has been harmed?" she asked. "What do they need? Whose obligation is it to meet that need?" And, she added later, who is in power in platform governance, and what harms have they overlooked and how?

In considering that sort of issue, Bharath Ganesh found three separate logics in his tour through platform racism and the governance of extremism: platform, social media, and free speech. Mark Zuckerberg offers a prime example of the latter, the Silicon Valley libertarian insistence that the marketplace of ideas will solve any problems and that sees the First Amendment freedom of expression as an absolute right, not one that must be balanced against others - such as "freedom from fear". Following the end of the conference by watching the end of yesterday's Congressional hearings, you couldn't help thinking about that as Mark Zuckerberg embarked on yet another pile of self-serving "Congressman..." rather than the simple "yes or no" he was asked to deliver.

Illustrations: Mark Zuckerberg, testifying in Congress on March 25, 2021.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:23 PM | Permalink | Comments (0) | TrackBacks (0)

How dumb do you have to be to spend decades watching movies and reading books about science fiction dystopias with perfect surveillance and then go on and build one anyway?

*This* dumb, apparently, because that what Shalini Kantayya discovers in her documentary Coded Bias, which premiered at the 2020 Sundance Film Festival. I had missed it until European Digital Rights (EDRi) arranged a streaming this week.

The movie deserves the attention paid to The Social Dilemma. Consider the cast Kantayya has assembled: "math babe" Cathy O'Neil, data journalism professor Meredith Broussard, sociologist Zeynep Tufekci, Big Brother Watch executive director Silkie Carlo, human rights lawyer Ravi Naik, Virginia Eubanks, futurist Amy Webb, and "code poet" Joy Buolamwini, who is the film's main protagonist and provides its storyline, such as it is. This film wastes no time on technology industry mea non-culpas, opting instead to hear from people who together have written a year's worth of reading on how modern AI disassembles people into piles of data.

The movie is framed by Buoalmwini's journey, which begins in her office at MIT. At nine, she saw a presentation on TV from MIT's Media Lab, and, entranced by Cynthia Breazeal's Kismet robot, she instantly decided: she was going to be a robotics engineer and she was going to MIT.

At her eventual arrival, she says, she imagined that coding was detached from the world - until she started building the Aspire Mirror and had to get a facial detection system working. At that point, she discovered that none of the computer vision tracking worked very well...until she put on a white mask. She started examining the datasets used to train the facial algorithms and found that every system she tried showed the same results: top marks for light-skinned men, inferior results for everyone else, especially the "highly melanated".

Teaming up with Deborah Raji, in 2018 Buolamwini published a study (PDF) of racial and gender bias in Amazon's Rekognition system, then being trialed with law enforcement. The company's response leads to a cameo, in which Buolamwini chats with Timnit Gebru about the methods technology companies use to discredit critics. Poignantly, today's viewers know that Gebru, then still at Google was only months away from becoming the target of exactly that behavior, fired over her own critical research on the state of AI.

Buolamwini's work leads Kantayya into an exploration of both algorithmic bias generally, and the uncontrolled spread of facial recognition in particular. For the first, Kantayya surveys scoring in recruitment, mortgage lending, and health care, and visits the history of discrimination in South Africa. Useful background is provided by O'Neil, whose Weapons of Math Destruction is a must-read on opaque scoring, and Broussard, whose Artificial Unintelligence deplores the math-based narrow conception of "intelligence" that began at Dartmouth in 1956, an arrogance she discusses with Kantayya on YouTube.

For the second, a US unit visits Brooklyn's Atlantic Plaza Towers complex, where the facial recognition access control system issues warnings for tiny infractions. A London unit films the Oxford Circus pilot of live facial recognition that led Carlo, with Naik's assistance, to issue a legal challenge in 2018. Here again the known future intervenes: after the pandemic stopped such deployments, BBW ended the challenge and shifted to campaigning for a legislative ban.

Inevitably, HAL appears to remind us of what evil computers look like, along with a red "I'm an algorithm" blob with a British female voice that tries to sound chilling.

But HAL's goals were straightforward: it wanted its humans dead. The motives behind today's algorithms are opaque. Amy Webb, whose book The Big Nine profiles the nine companies - six American, three Chinese - who are driving today's AI, highlights the comparison with China, where the government transparently tells citizens that social credit is always watching and bad behavior will attract penalties for your friends and family as well as for you personally. In the US, by contrast, everyone is being scored all the time by both government and corporations, but no one is remotely transparent about it.

For Buolamwini, the movie ends in triumph. She founds the Algorithmic Justice League and testifies in Congress, where she is quizzed by Alexandria Ocasio-Cortez(D-NY) and Jamie Raskin (D-MD), who looks shocked to learn that Facebook has patented a system for recognizing and scoring individuals in retail stores. Then she watches as facial recognition is banned in San Francisco, Somerville, Massachusetts, and Oakland, and the electronic system is removed from the Brooklyn apartment block - for now.

Earlier, however, Eubanks, author of Automating Inequality, issued a warning that seems prescient now, when the coronavirus has exposed all our inequities and social fractures. When people cite William Gibson's "The future is already here - it's just not evenly distributed", she says, they typically mean that new tools spread from rich to poor. "But what I've found is the absolute reverse, which is that the most punitive, most invasive, most surveillance-focused tools that we have, they go into poor and working communities first." Then they get ported out, if they work, to those of us with higher expectations that we have rights. By then, it may be too late to fight back.

See this movie!

Illustrations: Joy Buolamwini, in Coded Bias.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:38 PM | Permalink | Comments (0) | TrackBacks (0)

I wonder a lot about how the baby downstairs will develop differently because of his September 2020 birth date. In his first five months, the only humans who have been in close contact are his two parents, a smattering of doctors and nurses, and a stray neighbor who occasionally takes him for walks. Walks, I might add, in which he never gets out of his stroller but in which he exhibits real talent for staring contests (though less for intelligent conversation). His grandparents he only knows through video calls. His parents think he's grasped that they're real, though not present, people. But it's hard to be sure.

The effects of the pandemic are likely to be clear a lot sooner for the older children and young people whose lives and education have been disrupted over the past year. This week, as part of the LSE Post-Covid World Festival, Sonia Livingstone (for whose project I wrote some book reviews a few years ago) led a panel to discuss those effects.

Few researchers in the UK - Livingstone, along with Andy Phippen, is one of the exceptions, as is, less formally, filmmaker and House of Lords member Beeban Kidron, whose 2013 film InRealLife explores teens' use of the Internet - ever bother to consult children to find out what their online experiences and concerns really are. Instead, the agenda shaped by politicians and policy makers centers on adults' fears, particularly those that can be parlayed into electoral success. The same people who fret that social media is posing entirely new problems today's adults never encountered as children refuse to find out what those problems look like to the people actually experiencing them. Worse, the focus is narrow: protecting children from pornography, grooming, and radicalization is everywhere, but protecting them from data exploitation is barely discussed. In the UK, as Jen Persson, founder of DefendDigitlMe, keeps reminding us, collecting children's data is endemic in education.

This was why the panel was interesting: all four speakers are involved in projects aimed to understand and amplify children's and young people's own concerns. From that experience, all four - Konstantinos Papachristou, the youth lead for the #CovidUnder19 project, Maya Götz, who researches children, youth, and television, Patricio Cuevas-Parra, who is part of a survey of 10,000 children and young people, and Laurie Day - highlighted similar issues of lack of access and inequality - not just to the Internet but also to vaccines and good information.

In all countries, the shift to remote leaning has been abrupt, exposing infrastructure issues that were always urgent, but never quite urgent enough to fix. Götz noted that in some Asian countries and Chile she's seeing older technologies being pressed into service to remedy some of this - technologies like broadcast TV and radio; even in the UK, after the first lockdown showed how many low-income families could not afford sufficient data plans, the the BBC began broadcasting curriculum-based programming.

"Going back to normal," Day said, "needs a rethink of what support is needed." Yet for some students the move to online learning has been liberating, lightening social and academic pressures and giving space to think about their values and the opportunity to be creative. We don't hear so much about that; British media focus on depression and loss.

By the time the baby downstairs reaches school age, the pandemic will be over, but its footprint will be all over how his education proceeds.

Persson, who focuses on the state's use of data in education, says that one consequence of the pandemic is that Microsoft and Google have entrenched themselves much more deeply into the UK's education infrastructure.

"With or without covid, schools are dependent on them for their core infrastructure now, and that's through platforms joining up their core personal data about students and staff - email addresses, phone numbers, names, organizational data - and joining all that up," she says. Parents are encouraged to link to their children's accounts, and there is, for the children concerned, effectively, "no privacy". The software, she adds, was really designed for business and incompletely adapted for education. For example, while there are controls schools can use for privacy protection, the defaults, as always, are towards open sharing. In her own children's school, which has 2,000 students, the software was set up so every user could see everyone else's email address.

"It's a huge contrast to [the concern about] online harms, child safety, and the protection mantra that we have to watch everything because the world is so unsafe," she says. Partly, this is also a matter of perception: policy makers tend to focus on "stranger danger" and limiting online content rather than ID theft, privacy, and how all this collected data may be used in the future. The European Digital Rights Initiative (EDRi) highlights the similar thinking behind European Commission proposals to require the platforms to scan private communications as part of combating child sexual abuse online.

All this awaits the baby downstairs. The other day, an 18-month-old girl ran up to him, entranced. Her mother pulled her back before she could touch him or the toys tied to his stroller. For now, he, like other pandemic babies, is surrounded by an invisible barrier. We won't know for several decades what the long-term effect will be.

Illustrations: Illustrations: Sonia Livingstone's LSE panel.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 12:30 PM | Permalink | Comments (0) | TrackBacks (0)

Years ago, a by-then-retired former teenaged hacker listened to my account of what mid-1990s hackers were saying, and sighed at how little things had changed since his 1980s heyday. "It's still that thing of doing it day after day," he said, or more or less, meaning tedious daily hours doggedly poking around sites, trying logins, keeping meticulous records, and matching a user name collected one year with a password spotted the next. Grueling, painstaking work for which you could get arrested. Which he eventually was, leading to said retirement.

Today's young hackers, brought up on video games, are used to such lengthy keyboard stints to grind out points. Does that make them better suited for the kind of work my former hacker, brought up on pinball machines, described? Not necessarily.

In a paper presented this week at the 2020 Workshop on Economics of Information Security, by Cambridge postdoc Ben Collier, he and co-authors Richard Clayton, Alice Hutchings, and Daniel R. Thomas lay out the lives of the vast majority of today's hackers. Attracted by the idea of being part of a "cool cybercrime", they find themselves doing low-level tech support, customer service, and 24/7 server maintenance for well-worn exploits, all while under the threat of disruption from intermediaries, law enforcement, and bugs left by incompetent software coders while impatient, distrustful customers fume at them. Worse, this kind of work doesn't attract the admiration of other hackers and these workers don't get to make creative leaps. It's just routine, boring office work, nothing like the hacker ethic they embraced or the hacker culture's self-image, which hasn't changed in any real sense since the 1990s, when it was described to me with evangelical fervor as thrilling.

The disappointment "fundamentally changes the experience of being in this world," Collier said. Isn't that always the way when your hobby becomes your day job?

These guys are little different from the "ghost workers", Mary L. Gray and Siddharth Suri profile in their 2019 book. However, this group don't expect these conditions, unlike the millions of invisible fixers and maintainers for companies like Uber, Amazon, and every other company that boasts of its special "AI" sauce. In the legitimate economy, these workers occupy the low-status bottom of the hierarchy and have little prospect of attaining the respect and perks of the engineers, research scientists, and top-level management who get all the visibility. The illegitimate economy is no different.

The authors got their idea from a leap of logic that seems obvious in retrospect: the gradual transition from the exploits of lone bedroom hackers to organized cybercrime-as-a- service. What was high-impact, low-volume crime is now high-volume crime, which requires a large, built infrastructure. "True scaling up needs lots of invisible supportive labor to enable true scale." Think the electrical or water grid in a large city.

Based on their forays onto cybercrime forums and numerous interviews, the authors find that neither the public at large nor the hackers themselves have adapted their mental models. "The heart of the subculture is still based on this idea of the mythic, lone, high-skilled hacker," Collier said. "It looks nothing like this invisible maintenance work." Or, of course, like this week's discovery that nation-state hackers have penetrated numerous US federal agencies.

In other words, the work these hackers are doing is exactly the same as life as a sysadmin for a legitimate business - with the same "deep, deep boredom" but with the added difficulty of how to spend their earnings. One of their many interviewees was able to monetize his efforts unusually well. "He ran out of stuff to buy himself and his friends, and finally quit because he was piling up Amazon gift cards in shoeboxes under his bed and it stressed him out." At one point, he even cut up thousands of dollars' worth of the cards "just for something to do". Closed to him: using the money to buy a house or education and improve his life.

WEIS began in 2002 as a unique effort to apply familiar concepts of economics - incentives, externalities, asymmetric information, and moral hazard - to information security, understanding that despite the growing threats no organizations has infinite resources. Over the years, economists have increasingly taken an interest. The result is a cross-the-streams event where a study like this one may be followed by a math-heavy analysis of the relationship between pricing and security-related business strategies, each offering possibilities for new approaches.

Collier concluded that arresting, charging, and convicting these guys is counter-productive because, "It's important not to block their escape routes. They often get in because the main routes in society are blocked." He added, "The systems of value and capital and social status that exist in the world are not working for loads of people, or they don't have access so they make their own alternatives." Cracking down and conducting mass arrests also blocks those routes back into mainstream society.

Would today's teens choose the hacking life if they really understood what the job was going to be like? As someone commented, at the next big arrest perhaps the press release should stress the number of hours the miscreants worked, the sub-McDonalds hourly pay they eventually earned, and the permanent anomie induced by their disappointment, disillusionment, and alienation.

Illustrations: Ben Collier, presenting "Cybercrime is (often) boring: maintaining the infrastructure of cybercrime economies" at WEIS 2020.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 11:43 AM | Permalink | Comments (0) | TrackBacks (0)

Last week's review of 30 years of writing about the Internet and copyright focused on rightsholders' efforts to protect a business model developed for physical media and geographical restrictions in the face of new, global, digital media. Of the counter-efforts, mainstream attention has focused on the illegal ones; I squeezed in links to most of my past writing on "pirate" sites, although I missed pieces on The Pirate Bay, BitTorrent, and new business models. I also missed out discussing large-scale appropriation by companies that are apparently too big to sue, such as Google books and the more recent fuss over the Internet Archive's Controlled Digital Lending and National Emergency Library.

More interesting, however, are the new modes of access the Internet clearly could open up to niche material and frustrated artists, creators, and collaborators. At the MIT Media Lab's 1994 open day (TXT), a remarkable collection of Hollywood producers, and creative artists predicted that the Internet would unlock a flood of (American) creativity that previously had no outlet (although Penn Jillette doubted the appeal of interactive storytelling).

Lots of this has actually happened. Writers have developed mainstream audiences through self-publishing; web-based publishing enabled generations of cartoonists; and YouTube and TikTok offer options that would never fit into a TV schedule. Mass collaboration has also flourished: Wikipedia, much despised in some quarters 15 years ago, has ripened into an invaluable resource (despite its flaws that need fixing), as has OpenStreetMap, which was outed this week as a crucial piece of infrastructure for Facebook, Apple, Amazon, and Microsoft.

Developing new forms of copyright law has been a critical element in all this, beginning with the idea of copyleft, first used in 1976 and fleshed out in more detail by Richard Stallman in 1985. Traditionally, either you copyrighted the work and claimed all rights or you put the work into the public domain for everyone to use for free, as the satirist Tom Lehrer has recently done.

Stallman, however, wanted to ensure that corporate interests couldn't appropriate the work of volunteers, and realized that he could write a copyright license that dictates those terms, paving the way for today's open source community. In 2001, Lawrence Lessig, Hal Abelson, and Eric Eldred founded Creative Commons to make it easy for people posting new material to the web to specify whether and how others can use it. It's easy to forget now how big an undertaking it was to create licenses that comply with so many legal systems. I would argue that it's this, rather than digital rights management that has enabled widespread Internet creative publishing.

The third piece of this story has played a crucial role in this pandemic year of A.D. 2020. In the halls of a mid-1990s Amsterdam conference on copyright, a guy named Christopher Zielinski made this pitch: a serious problem was brewing around early paywall experiments. How were people in poorer countries going to gain access to essential scientific and medical information? He had worked for the WHO, I think; in a later email I remember a phrase about information moving through disadvantaged countries in "armored trucks".

Zielinski was prescient. In 2015, the Ebola virus killed 10,000 people in Liberia, Sierra Leone, and Guinea, in part because received wisdom held that Ebola was not present in West Africa, slowing the initial response. It was only later that three members of a team drafting Liberia's Ebola recovery plan discover that scientific researchers had written articles establishing its presence as long ago as 1982. None of the papers were co-written with Liberian scientists, and they were published in European journals, which African researchers cannot afford. In this case, as writers Bernice Dahn, Vera Mussah, and Cameron Nutt laid out, closed access cost lives: "Equity must be an indispensable goal in protecting from threats like Ebola, and in the quality of care delivered when prevention fails."

Meanwhile, in another part of the forest...as early as 1991 others saw the potential of using the Internet to speed up scientific publishing and peer review, leading Paul Ginsparg to respond by creating the arXiv repository to share preprints of physics journal articles. Numerous copies for other fields followed. In 2003, leading research, scientific, and cultural institutions created and signed the Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities laying out steps to promote the Internet as a medium for disseminating global knowledge. By 2006, the six-year-old Public Library of Science had set up PLOS ONE, the first peer-reviewed open access scientific journal for primary research in science and medicine.

While there are certainly issues to be solved, such as the proliferation of fake journals, improving peer review, and countering enduring prejudice that ties promotions and prestige to traditional proprietary journals, open access continues to grow. Those who believe that the Internet is going to destroy science are likely to be wrong, and publishers who don't plan for this future are likely to crater.

The global distribution accessible to artists and creators is valuable, but openness is critical to the scientific method of building knowledge. The open approach has been critical during the pandemic. As vaccine candidates prepare for takeoff, we can thank the Internet and the open access movement that it's taken a year, not decades.

Illustrations: Edward Jenner, who created the first vaccine, for smallpox (from the Wellcome images collection, via Wikimedia).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 12:26 PM | Permalink | TrackBacks (0)

There's a TV ad currently running on MSNBC that touts the services of a company that makes custom T-shirts to help campaigns raise funds for causes such as climate change.

Pause. It takes 2,700 liters of water to make a cotton T-shirt - water that, the Virtual Water project would argue, is virtually exported from cotton-growing nations to those earnest climate change activists. Plus other environmental damage relating to cotton; see also the recent paper tracking the pollution impact of denim microfibers. So the person buying the T-shirt may be doing a good thing on the local level by supporting climate change activism while simultaneously exacerbating the climate change they're trying to oppose.

The same sort of issue arose this week at the UK Internet Governance Forum with respect to what the MP and engineer Chi Onwurah (Labour-Newcastle upon Tyne Central) elegantly called "data chaos" - that is, the confusing array of choices and manipulations we're living in. Modern technology design has done a very good job of isolating each of us into a tiny silo, in which we attempt to make the best decisions for ourselves and our data without any real understanding of the wider impact on wider society.

UCL researcher Michael Veale expanded on this idea: "We have amazing privacy technologies, but what we want to control is the use of technologies to program and change entire populations." Veale was participating in a panel on building a "digital identity layer" - that is, a digital identity infrastructure to enable securely authenticated interactions on the Internet. So if we focus on confidentiality we miss the danger we're creating in allowing an entire country to rely on intermediaries whose interests are not ours but whose actions could - for example - cause huge populations to self-isolate during a pandemic. It is incredibly hard just to get a half-dozen club tennis players to move from WhatsApp to something independent of Facebook. At the population level, lock-in is far worse.

Third and most telling example. Last weekend, at the 52nd annual conference of the Cybernetics Society, Kate Cooper, from the Birmingham Food Council, made a similar point when, after her really quite scary talk, she was asked whether we could help improve food security if those of us who have space started growing vegetables in our gardens. The short answer: no. "It's subsistence farming," she said, going on to add that although growing your own food helps you understand your own relationship with food and where it comes from and can be very satisfying to do, it does nothing at all to help you gain a greater understanding of the food system and the challenges of keeping it secure. This is - or could be - another of Yes, Minister's irregular verbs: I choose not to eat potato chips; you very occasionally eat responsibly-sourced organic potato chips; potato chips account for 6% of Britain's annual crop of potatoes. This was Cooper's question: is that a good use of the land, water, and other resources? Growing potatoes in your front garden will not lead you to this question.

Cybernetics was new to me two years ago, when I was invited to speak at the 50th anniversary conference. I had a vague idea it had something to do with Isaac Asimov's robots. In its definition, Wikipedia cites MIT scientific Norbert Weiner in 1948: "the scientific study of control and communication in the animal and the machine". So it *could* be a robot. Trust Asimov.

Attending the 2018 event, followed by this year's, which was shared with the American Society for Cybernetics, showed cybernetics up as a slippery transdiscipline. The joint 2020 event veered from a case study of IBM to choreography, taking in subjects like the NHS Digital Academy, design, family therapy, social change, and the climate emergency along the way. Cooper, who seemed as uncertain as I was two years ago whether her work really had anything to do with cybernetics, fit right in.

The experience has led me to think of cybernetics as a little like Bayes' Theorem as portrayed in Sharon Bertsch McGrayne's book The Theory That Would Not Die. As she tells the story, for two and a half centuries after its invention, select mathematicians kept the idea alive but rarely dared to endorse it publicly - and today it's everywhere. The cybernetics community feels like this, too: a group who are nurturing an overlooked, poorly understood-by-the-wider-world, but essential field waiting for the rest of us to understand its power.

For a newcomer, getting oriented is hard; some of the discussion seems abstract enough to belong in a philosophy department. Other aspects - such as Ray Ison's description of his new book, The Hidden Power of Systems Thinking, smacks of self-help, especially when he describes it: "The contention of the book is that systems thinking in practice provides the means to understand and fundamentally alter the systems governing our lives."

At this stage, however, with the rolling waves of crises hitting our societies (which Ison helpfully summed up in an apt cartoon), if this is cybernetics, it sounds like exactly what we need. "Why," asked the artist Vanilla Beer, whose father was the cybernetics pioneer Stafford Beer, "is something so useful unused?" Beats me.

Illustrations: Chi Onwurah (official portrait, via Wikimedia).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 11:03 AM | Permalink | Comments (0) | TrackBacks (0)

The point is not whether it's ludicrous but whether it breaks the law.

Until Hannah Smethurst began speaking at this week's gikii event - the year's chance to mix law, digital rights, and popular culture - I had not realized just how many invisible vehicles there are in our books and films. A brief trawl turns up: Wonder Woman's invisible jet, Harry Potter's invisibility cloak and other invisibility devices, and James Bond's invisible Aston Martin. Do not trouble me with your petty complaints about physics. This is about the law.

Every gikii (see here for links to writeups of previous years) - ranges from deeply serious-with-a-twist to silly-with-an-insightful undercurrent. This year's papers included the need for a fundamental rethink of how we regulate power (Michael Veale), the English* "bubble" law that effectively granted flatmates permanent veto power over each other's choice of sex partner (gikii founder Lilian Edwards), and the mistaken-identity frustrations of having early on used your very common name as your Gmail address (Jat Singh).

In this context, Smethurst's paper is therefore business as usual. As she explained, there is nothing in highway legislation that requires your car to be visible. The same is not true of number plates, which the law says must be visible at all times. But can you enforce it? If you can't see the car, how do you know you can't see the number plate? More uncertain is the highway code's requirement to indicate braking and turns when people don't know you're there; Smethurst suggested that a good lawyer could argue successfully that turning on the lights unexpectedly would dazzle someone. No, she said, the main difficulty is the dangerous driving laws. Well, that and the difficulty of getting insurance to cover the many accidents when people - pedestrians, cyclists, other cars - collide with it.

This raised the possibly of "invisibility lanes", an idea that seems like it should be the premise for a sequel to Death Race 2000. My overall conclusion: invisibility is like online anonymity. People want it for themselves, but not for other people - at least, not for other people they don't trust to behave well. If you want an invisible car so you can drive 100 miles an hour with impunity, I suggest a) you probably aren't safe to have one, and b) try driving across Kansas.

We then segued into the really important question: if you're riding an invisible bike, are *you* visible? (General consensus: yes, because you're not enclosed.)

On a more serious note, people have a tendency to laugh nervously when you mention that numerous jurisdictions are beginning to analyze sewage for traces of coronavirus. Actually, wastewater epidemiology, as this particular public health measure is known, is not a new surveillance idea born of just this pandemic, though it does not go all the way back to John Snow and the Broadwick Street pump. Instead, Snow plotted known cases on a map, and spotted the pump as the source of contagion when they formed a circle around it. Still, epidemiology did start with sewage.

In the decades since wastewater epidemiology was developed, some of its uses have definitely had an adversarial edge, such asestablishing the level of abuse of various drugs and doping agents or particular diseases in a given area. The goal, however, is not to supposed to be trapping individuals; instead it's to provide population-wide data. Because samples are processed at the treatment plant along with everyone else's, there's a reasonable case to be made the system is privacy-preserving; even though you could analyze samples for an individual's DNA and exact microbiome, matching any particular sample to its own seems unlikely.

However, Reuben Binns argued, that doesn't mean there are no privacy implications. Like anything segmented by postcode, the catchment areas defined for such systems are likely to vary substantially in the number of households and individuals they contain, and a lot may depend on where you put the collection points. This isn't so much an issue for the present purpose, which is providing an early-warning system for coronavirus outbreaks, but will be later, when the system is in place and people want to use it for other things. A small neighborhood with a noticeable concentration of illegal drugs - or a small section of an Olympic athletes village with traces of doping agents above a particular threshold - could easily find itself a frequent target of more invasive searches and investigations. Also, unless you have your own septic field, there is no opt-out.

Binns added this unpleasant prospect: even if this system is well-intentioned and mostly harmless, it becomes part of a larger "surveillant assemblage" whose purpose is fundamentally discriminatory: "to create distinctions and hierarchies in populations to treat them differently," as he put it. The direction we're going, eventually every part of our infrastructure will be a data source, for our own good.

This was also the point of Veale's paper: we need to stop focusing primarily on protecting privacy by regulating the use and collection of data, and start paying attention to the infrastructure. A large platform can throw away the data and still have the models and insights that data created - and the exceptional computational power to make use of it. All that infrastructure - there's your invisible car.

Illustrations: James Bond's invisible car (from Live and Let Die).

*Correction: I had incorrectly identified this law as Scottish.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:15 PM | Permalink | Comments (0) | TrackBacks (0)

I realize I cheated. The question was complex: assume a pre-defined character ("Rural Rain") and decide how she would have handled an ethical dilemma involving a portfolio and a bank that appeared to be profiting from shady Middle East arms deals. I said she'd do nothing for now and do more research.

In other words, *I* didn't want to decide. I plead that I've never LARPed before, and my character could be a procrastinator, but that, too, is a cheat. Any fiction writer has to make decisions like these all the time. If you want to write that novel, this is practice. And I failed.

The LARP in this case was a fictional focus group organized by Ruth Catlow as part of real UCL research studying how attitudes to data transparency and consumer ethics are shifting. Recursion 'r' us. The UCL project, Glass Houses, has already produced an umber of papers on subjects like banks and cryptocurrencies, and privacy, transparency, and the blockchain, which is often being mentioned as a method for ensuring privacy and transparency.

The thought process that led to our fictional focus group began with Sarah Meiklejohn, who specializes in cryptocurrencies, and observed that even though Zcash supports anonymity, most users don't take advantage of it (PDF), even though there's no particular social pressure to deter them. Lacking the ability to talk to Zcash users themselves, the researchers developed this exercise to explore why and how people care about transparency and how they might think about changing their behavior based on life experiences or arguments presented to them.

So: the fictitious company True Insight, founded 2013 to use data-based emerging technology and design methodologies to find novel solutions, presented us each with a dilemma involving either finance or food and asked us to make a decision. In breakout groups (by topic), we discussed those decisions. Then we were asked to imagine our lives in 2030, taking into account the consequences of those decisions.

My character's dilemma was whether to move her savings account, which was held by a new online bank, chosen for its flexibility and competitive interest rate. Unfortunately, she had now discovered that 90% of the bank's investments were linked with major arms and military contractors operating in the Middle East and the Gulf. Should she move her account? This is where I felt someone who had just lost her earning power - my character was described as a newly retired care home worker who had finished secondary school - would be slow and cautious. What are her alternatives?

I have to applaud the creativity of the others in the group. Mr Fintech, who in 2020 was the bank's head and was trying to control the PR fallout, had abandoned his wife and children, moved to Thailand, and remarried. Now, he said, he had left the industry and was leading a group "hacking the blockchain". Another's assigned 2020 character was a fellow customer who decided with her partner to move their account for ethical reasons even though it meant denting their aspirations to have children and buy a house. By 2030, she said, the new radical transparency had exposed things her partner had hidden, and they'd split up. "I should have known when he wanted to name our child 'Elon'," she said sadly. Her job had disappeared, and with it her dreams. She was just trying to get by.

My character's description said she liked to read the news. I decided she would conveniently also like to read, now she had time, and would continue to educate herself, including reading books about banking, investment, the Middle East, and the arms trade. I thought she'd be more shocked at the bank's incompetence in failing to spot that it was investing in front for an arms dealer than by its ethical failure. Her 2030, in my imagining, was not much different from her 2020: she'd remain in her small town apartment, carefully managing her resources. A cell of the radical transparency movement that another character mentioned arrived early in her town, and what began as a movement to force ethics on companies and their supply chains had been turned on individuals. In her case, the local group had discovered that a workman replacing her toilet had eaten lunch at a disapproved pub and blamed her for not having prevented this.

Mr Fintech suggested my character should VPN herself thoroughly. Instead, I thought she'd opt for physical world interactions as much as possible because people behave differently when they actually know you. Interestingly, the now-single struggler reported a similar approach. She no longer had "the luxury" to embrace ethical choices, but her area's inability to depend on government was leading them to use barter, trade off the books, and create local currencies.

In 1998, privacy advocates were outraged by David Brin's book The Transparent Society, which argued for radical openness (an idea whose time is apparently trying to come). At the Computers, Freedom, and Privacy conference, I remember him saying that privacy laws protected only the rich and powerful. I never believed that. This exercise showed me, to my surprise, that I apparently do believe that transparency laws could be abused the same way and for the same reason: we live in a society that is judgmental and unforgiving about small infractions. Like so much else, transparency is a tool, not a solution.

Illustrations: The Palm House at Kew (via Kew Gardens).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 11:53 AM | Permalink | Comments (0) | TrackBacks (0)

What I - and I suspect a lot of other people - would love to have right now is an online calculator into which you could put where you were going, the time of day, the length of time you expect to spend there, and the type of activity and get back out a risk estimate of acquiring coronavirus infection given various mitigations. I write this as the UK government announces that the "threat level" is dropping from "4" to "3", which tells me more or less precisely nothing useful.

Throughout the pandemic, the British government has explained every decision by saying it's led by the science. I'm all for following the advice of scientists - particularly, in our present situation, public health experts, virologists, and epidemiologists - but "the science" implies there's a single received monolithic truth even while failing to identify any particular source for it. Which science? Whose research? Based on what evidence? Funded by whom? How does it fit in with what we were told before?

Boris Johnson's government spent much of the early months avoiding answering those questions, which has led, as the biologist Ian Boyd complains to the characterization of the Scientific Advisory Group for Emergencies (SAGE) as "secretive". As the public trusts this government less and less, showing their work has become increasingly important, especially when those results represent a change of plan.

The last four months have seen two major U-turns in "the science" that's governing our current lives, and a third may be in progress: masks, contact tracing apps, and the two-meter rule. Meanwhile, the pieces that are supposed to be in place for reopening - a robust contact tracing system, for example - aren't.

We'll start with masks. Before this thing started, the received wisdom was that masks protected other people from you, but not you from them. This appears to still be the generally accepted case. But tied in with that was the attitude that wearing masks while ill was something only Asians did; Westerners...well, what? Knew better? Were less considerate? Were made of tougher stuff and didn't care if they got sick? In mid-March, Zeynep Tufecki got a certain amount of stick on Twitter for impassioned plea in the New York Times that public health authorities should promote wearing masks and teach people how to do it properly. "Of course masks work," she wrote, "maybe not perfectly, and not all to the same degree, but they provide some protection."

But we had to go on arguing about it back and forth. There is says Snopes, no real consensus on how effective they are. Nonetheless, it seems logical they ought to help, and both WHO and CDC now recommend them while mayors of crowded cities are increasingly requiring them. In this case, there's no obvious opportunity for profiteering and for most people the inconvenience is modest. The worst you can suspect is that the government is recommending them so we'll feel more confident about resuming normal activity.

Then, for the last four months we've been told to stay two meters from everyone else except fellow household members. During the closures, elves - that is, people who took on the risks of going to work - have been busy painting distancing indicators on underground platforms, sidewalks, and park benches and sticking decals to train windows. They've set up hand sanitizer stations in London's stations, and created new bike lanes and pedestrian areas. Now, the daily news includes a drumbeat of pressure on government to reduce that recommended distance to one meter. Is this science or economics? The BBC has found a study that says that standing one meter apart carries ten times the risk of two meters. But how significant is that?

I'm all for "the science", but there's so much visible vested interest that I want details. What are the tradeoffs? How does the drop in distance change R0, the reproduction number? The WHO recommends one meter - but it assumes that people are wearing masks - which, in London, on public transport they will be but in restaurants they can't be.

Finally, when last seen, the UK's contact tracing app was being trialed on the Isle of Wight and was built in-house using a centralized design despite the best efforts of privacy advocates and digital rights activists to convince NHSx it was a bad idea. Yesterday, this app was officially discarded.

The relevant scientific aspect, however, is how much apps matter. In April, an an Oxford study suggested that 60% of the population would have use the app for it to be effective.

We should have read the study, as MIT Technology Review did this week to find that it actually says contact tracing apps can be helpful at much lower levels of takeup. It is still clear that human tracers with local knowledge are more effective and there are many failings in the tracing system, as the kibitzing scientific group Independent SAGE says, but *some* help is better than no help.

"The science" unfortunately can't offer us what we really want: certainty. Instead, we have many imperfect but complementary tools and must hope they add up to something like enough. The science will only become fully clear much later.

Illustrations: London's Paddington station on June 13.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:09 PM | Permalink | Comments (0) | TrackBacks (0)

In 2011, Netscape creator-turned-venture capitalists Marc Andreesen argued that software is eating the world. Andreesen focused on a rather narrow meaning of "world" - financial value. Amazon ate Borders' lunch; software fuels the success of Wal-Mart, Fedex, airlines, and financial services. Like that.

There is, however, a more interesting sense in which software is eating the world, and that's its takeover of what we think of as "hardware". A friend tells me, for example, that part of the pleasure he gets from driving a Tesla is that its periodic software updates keep the car feeling new, so he never looks enviously at the features on later models. Still, these updates do at least sound like traditional software. The last update of 2019, for example, included improved driver visualization, a "Camp Mode" to make the car more comfortable to spend the night in, and other interface improvements. I assume something as ordinarily useful as map updates is too trivial to mention.

Even though this means a car is now really a fancy interconnected series of dozens of computer networks whose output happens to be making a large, heavy object move on wheels. I don't have trouble grasping the whole thing, not really. It's a control system.

Much more confounding was the time, in late 1993. when I visited Demon Internet, then a startup founded to offer Internet access to UK consumers. Like quite a few others, I was having trouble getting connected via the Demon's adapted version of KA9Q, connection software written for packet radio. This was my first puzzlement: how could software for "packet radio" (whatever that was) do anything on a computer? That was nothing to my confusion when Demon staffer Mark Turner explained to me that the computer could parse the stream of information coming into it and direct the results to different applications simultaneously. At that point, I'd only ever used online services where you could only do one thing at a time, just as you could only make one phone call at a time. I remember finding the idea of one data stream servicing many applications at once really difficult to grasp. How did it know what went where?

That is software, and it's what happened in the shift from legacy phone networks' circuit switching to Internet-style packet switching.

I had a similar moment of surreality when first told about software-defined radio. A radio was a *thing*. How could it be software? By then I knew about spread spectrum, invented by the actress Hedy Lamarr and pianist George Antheil to protect wartime military conversations from eavesdropping, so it shouldn't have seemed as weird as it did.

And so to this week, when, at the first PhD Cyber Security Winter School, I discovered programmable - - that is, software-defined - networks. Of course networks are controlled by software already, but at the physical layer it's cables, switches, and routers. If one of those specialized devices needs to be reconfigured you have to do it locally, device by device. Now, the idea is more generic hardware that can be reprogrammed on the fly, enabling remote - and more centralized and larger-scale - control. Security people like the idea that a network can both spot and harden itself against malicious traffic much faster. I can't help being suspicious that this new world will help attackers, too, first by providing a central target to attack, and second because it will be vastly more complex. Authentication and encryption will be crucial in an environment where a malformed or malicious data packet doesn't just pose a threat to the end user who receives it but can reprogram the network. Helpfully, the NSA has thought about this in more depth and greater detail. They do see centralization as a risk, and recommend a series of measures for protecting the controller; they also highlight the problems increased complexity brings.

As the workshop leader said, this is enough of a trend for Cisco, and Intel to embrace it; six months ago, Intel paid $5 billion for Barefoot Networks, the creator of P4, the language I saw demonstrated for programming these things.

At this point I began wondering if this doesn't up-end the entire design philosophy of the Internet, which was to push all the intelligence out to the edges, The beginnings of this new paradigm, active networking, appeared around the early 2000s. The computer science literature - for example, Activating Networks (PDF), by Jonathan M. Smith, Kenneth L. Calvert, Sandra L. Murphy, Hilarie K. Orman, and Larry L. Peterson, and Active Networking: One View of the Past, Present, and Future (PDF), by Smith and Scott M. Nettles - plots out the problems of security and complexity in detail, and considers the Internet and interoperability issues. The Road to SDN: An Intellectual History of Programmable Networks, by Nick Feamster, Jennifer Rexford, and Ellen Zegura, recapitulates the history to date.

My real question, however, is one I suspect has received less consideration: will these software-defined networks make surveillance and censorship easier or harder? Will they have an effect on the accessibility of Internet freedoms? Are there design considerations we should know about? These seem like reasonable questions to ask as this future hurtles toward us.

Illustrations: Hedy Lamarr, in The Conspirators, 1944..

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 12:59 PM | Permalink | Comments (0) | TrackBacks (0)

For me, the scariest presentation of 2019 was a talk given by Cornell University professor Vitaly Shmatikov about computer models. It's partly a matter of reframing the familiar picture; for years, Bill Smart and Cindy Grimm have explained to attendees at We Robot that we don't necessarily really know what it is that neural nets are learning when they're deep learning.

In Smart's example, changing a few pixels in an image can change the machine learning algorithm's perception of it from "Abraham Lincoln" to "zebrafish". Misunderstanding what's important to an algorithm is the kind of thing research scientist Janelle Shane exploits when she pranks neural networks and asks them to generate new recipes or Christmas carols from a pile of known examples. In her book, You Look Like a Thing and I Love You, she presents the inner workings of many more examples.

All of this explains why researchers Kate Crawford and Trevor Paglen's ImageNet Roulette experiment tagged my my Twitter avatar as "the Dalai Lama". I didn't dare rerun it, because how can you beat that? The experiment over, would-be visitors are now redirected to Crawford's and Paglen's thoughtful examination of the problems they found in the tagging and classification system that's being used in training these algorithms.

Crawford and Paglen write persuasively about the world view captured by the inclusion of categories such as "Bad Person" and "Jezebel" - real categories in the Person classification subsystem. The aspect has gone largely unnoticed until now because conference papers focused on the non-human images in ten-year-old ImageNet and its fellow training databases. Then there is the *other* problem, that the people's pictures used to train the algorithm were appropriated from search engines, photo-sharing sites such as Flickr, and video of students walking their university campuses. Even if you would have approved the use of your forgotten Flickr feed to train image recognition algorithms, I'm betting you wouldn't have agreed to be literally tagged "loser" so the algorithm can apply that tag later to a child wearing sunglasses. Why is "gal" even a Person subcategory, still less the most-populated one? Crawford and Paglen conclude that datasets are "a political intervention". I'll take "Dalai Lama", gladly.

Again, though, all of this fits with and builds upon an already known problem: we don't really know which patterns machine learning algorithms identify as significant. In his recent talk to a group of security researchers at UCL, however, Shmatikov, whose previous work includes training an algorithm to recognize faces despite obfuscation, outlined a deeper problem: these algorithms "overlearn". How do we stop them from "learning" (and then applying) unwanted lessons? He says we can't.

"Organically, the model learns to recognize all sorts of things about the original data that were not intended." In his example, in training an algorithm to recognize gender using a dataset of facial images, alongside it will learn to infer race, including races not represented in the training dataset, and even identities. In another example, you can train a text classifier to infer sentiment - and the model also learns to infer authorship.

Options for counteraction are limited. Censoring unwanted features doesn't work because a) you don't know what to censor; b) you can't censor something that isn't represented in the training data; and c) that type of censoring damages the algorithm's accuracy on the original task. "Either you're doing face analysis or you're not." Shmatikov and Congzheng Song explain their work more formally in their paper Overlearning Reveals Sensitive Attributes.

"We can't really constrain what the model is learning," Shmatikov told a group of security researchers at UCL recently, "only how it is used. It is going to be very hard to prevent the model from learning things you don't want it to learn." This drives a huge hole through GDPR, which relies on a model of meaningful consent. How do you consent to something no one knows is going to happen?

What Shmatikov was saying, therefore, is that from a security and privacy point of view, the typical question we ask, "Did the model learn its task well?", is too limited. "Security and privacy people should also be asking: what else did the model learn?" Some possibilities: it could have memorized the training data; discovered orthogonal features; performed privacy-violating tasks; or incorporated a backdoor. None of these are captured in assessing the model's accuracy in performing the assigned task.

My first reaction was to wonder whether a data-mining company like Facebook could use Shmatikov's explanation as an excuse when it's accused of allowing its system to discriminate against people - for example, in digital redlinining. Shmatikov thought not, at least, not more than their work helps people find out what their models are really doing.

"How to force the model to discover the simplest possible representation is a separate problem worth invdstigating," he concluded.

So: we can't easily predict what computer models learn when we set them a task involving complex representations, and we can't easily get rid of these unexpected lessons while retaining the usefulness of the models. I was not the only person who found this scary. We are turning these things loose on the world and incorporating them into decision making without the slightest idea of what they're doing. Seriously?

Illustrations: Vitaly Shmatikov (via Cornell).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 12:27 AM | Permalink | Comments (0) | TrackBacks (0)

The years between 1995 and 1999 were a time when predicting the future was not a spectator sport. The lucky prognosticators gained luster from having their best predictions quoted and recirculated. The unlucky ones were often still lucky enough to have their worst ideas forgotten. I wince, personally, to recall (I don't dare actually reread) how profoundly I underestimated the impact of electronic commerce, although I can more happily point to predicting that new intermediaries would be the rule, not the disintermediation that everyone else seemed obsessed with.. Two things sparked this outburst: the uncertainty of fast-arriving technological change, and the onrushing new millennium.

Those early books fell into several categories. First was travelogues: the Internet for people who never expected to go there (the joke would be on them except that the Old Net these books explored mostly doesn't exist any more, nor the Middle Net after it). These included John Seabrook's Deeper, Melanie McGrath's Hard, Soft, and Wet, and JC Herz's Surfing on the Internet. Second was futurology and techno-utopianism: Nicholas Negroponte's Being Digital, and Tips for Time Travellers, by Peter Cochrane, then head of BT Research. There were also well-filled categories of now-forgotten how-to books and, as now, computer crime. What interested me, then as now, was the conflict between old and new: hence net.wars-the-book and its sequel, From Anarchy to Power. The conflicts those books cover - cryptography, copyright, privacy, censorship, crime, pornography, bandwidth, money, and consumer protection - are ones were are still wrangling over.

A few were simply contrarian: in 1998, David Brin scandalized privacy advocates with The Transparent Society, in which he proposed that we should embrace surveillance, but ensure that it's fully universal. Privacy, I remember him saying at that year's Computers, Freedom, and Privacy, favors the rich and powerful. Today, instead, privacy is as unequally distributed as money.

Among all these, one book had its own class: Frances Cairncross's The Death of Distance. For one thing, at that time writing about the Internet was almost entirely an American pastime (exceptions above: Cochrane and McGrath). For another, unlike almost everyone else, she didn't seem to have written her book by hanging around either social spaces on the Internet itself or in a technology lab or boardroom where next steps were being plotted out and invented. Most of us wrote about the Internet because we were personally fascinated by it. Cairncross, a journalist with The Economist studied it like a bug pinned to cardboard under a microscope. What was this bug? And what could it *do*? What did it mean for businesses and industries?

To answer those questions she did - oh, think of it - *research*. Not the kind that involves reading Usenet for hours on end, either: real stuff on industries and business models.

"I was interested in the economic impact it was going to have," she said the other day. Cairncross's current interest is the future of local news; early this year she donated her name to the government-commissioned review of that industry. Ironically, both because of her present interest and because of her book's title, she says the key thing she missed in considering the impact of collapsing communications costs and therefore distance was the important of closeness and the complexity of local supply chains. It may seem obvious in hindsight, now that three of the globe's top six largest companies by market capitalization are technology giants located within 15 miles of each other in Silicon Valley (the other two are 800 miles north, in Seattle).

The person who got that right was Michael Porter, who argued in 1998 that clusters mattered. Clusters allow ecosystems to develop to provide services and supplies, as well as attract skills and talent.

Still, Cairncross was right about quite a few things. She correctly predicted that the inequality of wages would grow within countries (and, she thought, narrow between countries); she was certainly right about the ongoing difficulty of enforcing laws restricting the flow of information - copyright, libel, bans on child abuse imagery; the increased value of brands; and the concentration that would occur in industries where networks matter. On the other hand, she suggested people would accept increased levels of surveillance in return for reduced crime; when she was writing, the studies showing cameras were not effective were not well-known. Certainly, we've got the increased surveillance either way.

More important, she wrote about the Internet in a way that those of us entranced with it did not, offering a dispassionate view even where she saw - and missed - the same trends everyone else did. Almost everyone missed how much mobile would take over. It wasn't exactly an age thing; more that if you came onto the Internet with big monitors and real keyboards it was hard to give them up -and if you remember having to wait to do things until you were in the right location your expectations are lower.

I think Cairncross's secret, insofar as she had one, was that she didn't see the Internet, as so many of us did, as a green field she could remake in her own desired image. There's a lesson there for would-be futurologists: don't fall in love with the thing whose future you're predicting, just like they tell journalists not to sleep with the rock stars.

Illustrations: Late 1990s books.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 11:58 AM | Permalink | Comments (0) | TrackBacks (0)

So can we stop now with the fantasy that data can be anonymized?

Two things sparked this train of thought. The first was seeing that researchers at the Mayo Clinic have shown that commercial facial recognition software accurately identified 70 of a sample set of 84 (that's 83%) MRI brain scans. For ten additional subjects, the software placed the correct identification in its top five choices. Yes, on reflection, it's obvious that you can't scan a brain without including its container, and that bone structure defines a face. It's still a fine example of data that is far more revealing than you expect.

The second was when Phil Booth, the executive director of medConfidential, on Twitter called out the National Health Service for weakening the legal definition of "anonymous" in its report on artificial intelligence (PDF).

In writing the MRI story for the Wall Street Journal (paywall), Melanie Evans notes that people have also been reidentified from activity patterns captured by wearables, a cautionary tale now that Google's owner, Alphabet, seeks to buy Fitbit. Cautionary, because the biggest contributor to reidentifying any particular dataset is other datasets to which it can be matched.

The earliest scientific research on reidentification I know of was Latanya Sweeney's 1997 success in identifying then-governor William Weld's medical record by matching the "anonymized" dataset of records of visits to Massachusetts hospitals against the voter database for Cambridge, which anyone could buy for $20. Sweeney has since found that 87% of Americans can be matched from just their gender, date of birth, and zip code. More recently, scientists at Louvain and Imperial College found that just 15 attributes can identify 99.8% of Americans. Scientists have reidentified individuals from anonymized shopping data, and by matching mobile phone logs against transit trips. Combining those two datasets identified 95% of the Singaporean population in 11 weeks; add GPS records and you can do it in under a week.

This sort of thing shouldn't be surprising any more.

The legal definition that Booth cited is Recital 26 of the General Data Protection Regulation, which specifies in a lot more detail about how to assess the odds ("all the means likely to be used", "account should be taken of all objective factors") of successful reidentification.

Instead, here's the passage he highlighted from the NHS report as defining "anonymized" data (page 23 of the PDF, 44 of the report): "Data in a form that does not identify individuals and where identification through its combination with other data is not likely to take place."

I love the "not likely". It sounds like one of the excuses that's so standard that Matt Blaze put them on a bingo card. If you asked someone in 2004 whether it was likely that their children's photos would be used to train AI facial recognition systems that in 2019 would be used to surveil Chinese Muslims and out pornography actors in Russia. And yet here we are. You can never reliably predict what data will be of what value or to whom.

At this point, until proven otherwise it is safer to assume that that there really is no way to anonymize personal data and make it stick for any length of time. It's certainly true that in some cases the sensitivity of any individual piece of data - say your location on Friday at 11:48 - vanishes quickly, but the same is not true of those data points when aggregated over time. More important, patient data is not among those types and never will be. Health data and patient information are sensitive and personal not just for the life of the patient but for the lives of their close relatives on into the indefinite future. Many illnesses, both mental and physical, have genetic factors; many others may be traceable to conditions prevailing where you live or grew up. Either way, your medical record is highly revealing - particularly to insurance companies interested in minimizing their risk of payouts or an employer wishing to hire only robustly healthy people - about the rest of your family members.

Thirty years ago, when I was first encountering large databases and what happens when you match them together, I came up with a simple privacy-protecting rule: if you do not want the data to leak, do not put it in the database. This still seems to me definitive - but much of the time we have no choice.

I suggest the following principles and assumptions.

One: Databases that can be linked, will be. The product manager's comment Ellen Ullman reported in 1997 still pertains: "I've never seen anyone with two systems who didn't want us to hook them together."

Two: Data that can be matched, will be.

Three: Data that can be exploited for a purpose you never thought of, will be.

Four: Stop calling it "sharing" when the entities "sharing" your personal data are organizations, especially governments or commercial companies, not your personal friends. What they're doing is *disclosing* your information.

Five: Think collectively. The worst privacy damage may not be to *you*.

The bottom line: we have now seen so many examples of "anonymized" data that can be reidentified that the claim that any dataset is anonymized should be considered as extraordinary a claim as saying you've solved Brexit. Extraordinary claims require extraordinary proof, as the skeptics say.

Addendum: if you're wondering why net.wars skipped the 50th anniversary of the first ARPAnet connection: first of all, we noted it last week; second of all, whatever headline writers think, it's not the 50th anniversary of the Internet, whose beginnings, as we wrote in 2004, are multiple. If you feel inadequately served, I recommend this from 2013, in which some of the Internet's fathers talk about all the rules they broke to get the network started.

Illustrations: Monty Python performing the Spanish Inquisition sketch in 2014 (via Eduardo Unda-Sanzana at Wikimedia.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:22 PM | Permalink | Comments (0) | TrackBacks (0)

The older man standing next to me was puzzled. "Can you drive it?"

He gestured at the VW Beetle-style car-like creation in front of us. Its exterior, except for the wheels and chassis, was stained glass. This car was conceived by the artist Desmond Wilcox, who surmised that by 2059 autonomous cars will be so safe that they will no longer need safety features such as bumpers and can be made of fragile materials. The sole interior furnishing, a bed, lets you sleep while in transit. In person, the car is lovely to look at. Utterly impractical today in 2019, and it always will be. The other cars may be safe, but come on: falling tree, extreme cold, hailstorm...kid with a baseball?

On being told no, it's an autonomous car that drives itself, my fellow visitor to the Science Museum's new exhibition, Driverless, looked dissatisfied. He appeared to prefer driving himself.

"It would look good with a light bulb inside it hanging at the back of the garden," he offered. It would. Bit big, though last week in San Francisco I saw a bigger superbloom.

"Driverless" is a modest exhibition by Science Museum standards, and unlike previous robot exhibitions, hardly any of these vehicles are ready for real-world use. Many are graded according to their project status: first version, early tests, real-world tests, in use. Only a couple were as far along as real-world tests.

Probably a third are underwater explorers. Among the exhibits: the (yellow submarine!) long-range Boaty McBoatface Autosub, which is meant to travel up to 2,000 km over several months, surfacing periodically to send information back to scientists. Both this and the underwater robot swarms are intended for previously unexplored hostile environments, such as underneath the Antarctic ice sheet.

Alongside these and Wilcox's Stained Glass Driverless Car of the Future was the Capri Mobility pod, the result of a project to develop on-demand vans that can shuttle up to four people along a defined route either through a pedestrian area or on public roads. Small Robot sent its Tom farm monitoring robot. And from Amsterdam came Roboat, a five-year research project to develop the first fleet of autonomous floating boats for deployment in Amsterdam's canals. These are the first autonomous vehicles I've seen that really show useful everyday potential for rethinking traditional shapes, forms, and functionality: their flat surfaces and side connectors allow them to be linked into temporary bridges a human can walk across.

There's also an app-controlled food delivery drone; the idea is you trigger it to drop your delivery from 20 meters up when you're ready to receive it. What could possibly go wrong?

On the fun side is Duckietown (again, sadly present only as an image), a project to teach robotics via a system of small, mobile robots that motor around a Lego-like "town" carrying small rubber ducks. It's compelling like model trains, and is seeking Kickstarter funding to make the hardware for wider distribution. This should have been the hands-on bit.

Previous robotics-related Science Museum exhibitions have asked as many questions as they answered. At that, this one is less successful. Drive.ai's car-mounted warning signs, for example, are meant to tell surrounding pedestrians what its cars are doing. But are we really going to allow cars onto public roads (or even worse, pedestrian areas, like the Capri pods) to mow people down who don't see, don't understand, can't read, or willfully ignore the "GOING NOW; DON'T CROSS" sign? So we'll have to add sound: but do we want cars barking orders at us? Today, navigating the roads is a constant negotiation between human drivers, human pedestrians, and humans on other modes of transport (motorcycles, bicycles, escooters, skateboards...). Do we want a tomorrow where the cars have all the power?

In video clips researchers and commentators like Noel Sharkey, Kathy Nothstine, and Natasha Merat discuss some of these difficulties. Merat has an answer for the warning sign: humans and self-driving cars will have to learn each other's capabilities in order to peacefully coexist. This is work we don't really see happening today, and that lack is part of why I tend to think Christian Wolmar is right in predicting that these cars are not going to be filling our streets any time soon.

The placard for the Starship Bot (present only as a picture) advises that it cannot see above knee height, to protect privacy, but doesn't discuss the issues raised when Edward Hasbrouck encountered one in action. I was personally disappointed, after the recent We Robot discussion of the "monstrous" Moral Machine and its generalized sibling the trolley problem, to see it included here with less documentation than on the web. This matters, because the most significant questions about autonomous vehicles are going to be things like: what data do they collect about the people and things around them? To whom are they sending it? How long will it be retained? Who has the right to see it? Who has the right to command where these cars go?

More important, Sharkey says in a video clip, we must disentangle autonomous and remote-controlled vehicles, which present very different problems. Remote-controlled vehicles have a human in charge that we can directly challenge. By contrast, he said, we don't know why autonomous vehicles make the decisions they do: "They're just matrices of numbers."

Illustrations: Wilcox's stained glass car.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 9:19 PM | Permalink | TrackBacks (0)

"There is a disruption coming." Words of doom?

Several months back we discussed Michael Salmony's fear that the Internet is about to destroy science. Salmony reminded that his comments came in a talk on the virtues of the open economy, and then noted the following dangers:

- Current quality-assurance methods (peer-review, quality editing, fact checking etc) are being undermined. Thus potentially leading to an avalanche of attention-seeking open garbage drowning out the quality research;
- The excellent high-minded ideals (breaking the hold of the big controllers, making all knowledge freely accessible etc) of OA are now being subverted by models that actually ask authors (or their funders) to spend thousands of dollars per article to get it "openly accessible". Thus again privileging the rich and well connected.

The University of Bath associate professor Joanna Bryson rather agreed with Salmony, also citing the importance of peer review. So I stipulate: yes, peer review is crucial for doing good science.

In a posting deploring the death of the monograph, Bryson notes that, like other forms of publishing, many academic publishers are small and struggle for sustainability. She also points to a Dutch presentation arguing that open access costs more.

Since she, as an academic researcher, has skin in this game, we have to give weight to her thoughts. However, many researchers dissent, arguing that academic publishers like Elsevier, Axel Springer profit from an unfair and unsustainable business model. Either way, an existential crisis is rolling toward academic publishers like a giant spherical concrete cow.

So to yesterday's session on the ten-year future of research, hosted by European Health Forum Gastein and sponsored by Elsevier. The quote of doom we began with was voiced there.

The focal point was a report (PDF), the result of a study by Elsevier and Ipsos MORI. Their efforts eventually generated three scenarios: 1) "brave open world", in which open access publishing, collaboration, and extensive data sharing rule; 2) "tech titans", in which technology companies dominate research; 3) "Eastern ascendance", in which China leads. The most likely is a mix of the three. This is where several of us agreed that the mix is already our present. We surmised, cattily, that this was more an event looking for a solution to Elsevier's future. That remains cloudy.

The rest does not. For the last year I've been listening to discussions about how academic work can find greater and more meaningful impact. While journal publication remains essential for promotions and tenure within academia, funders increasingly demand that research produce new government policies, change public conversations, and provide fundamentally more effective practice.

Similarly, is there any doubt that China is leading innovation in areas like AI? The country is rising fast. As for "tech titans", while there's no doubt that these companies lead in some fields, it's not clear that they are following the lead of the great 1960s and 1970s corporate labs like Bell Labs, Xerox PARC and IBM Watson, which invested in fundamental research with no connection to products. While Google, Facebook, and Microsoft researchers do impressive work, Google is the only one publicly showing off research, that seems unrelated to its core business">.

So how long is ten years? A long time in technology, sure: in 2009: Twitter, Android, and "there's an app for that" were new(ish), the iPad was a year from release, smartphones got GPS, netbooks were rising, and 3D was poised to change the world of cinema. "The academic world is very conservative," someone at my table said. "Not much can change in ten years."

Despite Sci-Hub, the push to open access is not just another Internet plot to make everything free. Much of it is coming from academics, funders, librarians, and administrators. In the last year, the University of California dropped Elsevier rather than modify its open access policy or pay extra for the privilege of keeping it. Research consortia in Sweden, Germany, and Hungary have had similar disputes; a group of Norwegian institutions recently agreed to pay €9 million a year to cover access to Elsevier's journals and the publishing costs of its expected 2,000 articles.

What is slow to change is incentives within academia. Rising scholars are judged much as they were 50 years ago: how much have they published, and where? The conflict means that younger researchers whose work has immediate consequences find themselves forced to choose between prioritizing career management - via journal publication - or more immediately effective efforts such as training workshops and newspaper coverage to alert practitioners in the field of new problems and solutions. Choosing the latter may help tens of thousands of people - at a cost of a "You haven't published" stall to their careers. Equally difficult, today's structure of departments and journals is poorly suited for the increasing range of multi-, inter-, and trans-disciplinary research. Where such projects can find publication remains a conundrum.

All of that is without considering other misplaced or perverse incensitives in the present system: novel ideas struggle to emerge; replication largely does not happen or fails, and journal impact factors are overvalued. The Internet has opened up beneficial change: Ben Goldacre's COMPare project to identify dubious practices such as outcome switching and misreported findings, and the push to publish data sets; and preprint servers give much wider access to new work. It may not be all good; but it certainly isn't all bad.

Illustrations: A spherical cow jumping over the moon (via Wikimedia.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:52 PM | Permalink | Comments (0) | TrackBacks (0)

"My iPhone won't stab me in my bed," Bill Smart said at the first We Robot, attempting to explain what was different about robots - but eight years on, We Robot seems less worried about that than about the brains of the operation. That is, AI, which conference participant Aaron Mannes described as, "A pile of math that can do some stuff".

But the math needs data to work on, and so a lot of the discussion goes toward possible consequences: delivery drones displaying personalized ads (Ryan Calo and Stephanie Ballard); the wrongness of researchers who defend their habit of scraping publicly posted data by saying it's "the norm" when their unwitting experimental subjects have never given permission; the unexpected consequences of creating new data sources in farming (Solon Barocas, Karen Levy, and Alexandra Mateescu); and how to incorporate public values (Alicia Solow-Neiderman) into the control of...well, AI, but what is AI without data? It's that pile of math. "It's just software," Bill Smart (again) said last week. Should we be scared?

The answer seems to be "sometimes". Two types of robots were cited for "robotic space colonialism" (Kristen Thomasen), because they are here enough and now enough for legal cases to be emerging. These are 1) drones, and 2) delivery robots. Mostly. Mason Marks pointed out Amazon's amazing Kiva robots, but they're working in warehouses where their impact is more a result of the workings of capitalism that that of AI. They don't scare people in their homes at night or appropriate sidewalk space like delivery robots, which Paul Colhoun described as "unattended property in motion carrying another person's property". Which sounds like they might be sort of cute and vulnerable, until he continues: "What actions may they take to defend themselves?" Is this a new meaning for move fast and break things?

Colhoun's comment came during a discussion of using various forecasting methods - futures planning, design fiction, the futures wheel (which someone suggested might provide a usefully visual alternative to privacy policies) - that led Cindy Grimm to pinpoint the problem of when you regulate. Too soon, and you risk constraining valuable technology. Too late, and you're constantly scrambling to revise your laws while being mocked by technical experts calling you an idiot (see 25 years of Internet regulation). Still, I'd be happy to pass a law right now barring drones from advertising and data collection and damn the consequences. And then be embarrassed; as Levy pointed out, other populations have a lot more to fear from drones than being bothered by some ads...

The question remains: what, exactly do you regulate? The Algorithmic Accountability Act recently proposed by Senators Cory Booker (D-NJ) and Ron Wyden (D-OR) would require large companies to audit machine learning systems to eliminate bias. Discrimination is much bigger than AI, said conference co-founder Michael Froomkin in discussing Alicia Solow-Neiderman's paper on regulating AI, but special to AI is unequal access to data.

Grimm also pointed out that there are three different aspects: writing code (referring back to Petros Terzis's paper proposing to apply the regime of negligence laws to coders); collecting data; and using data. While this is true, it doesn't really capture the experience Abby Jacques suggested could be a logical consequence of following the results collected by MIT's Moral Machine: save the young, fit, and wealthy, but splat the old, poor, and infirm. If, she argued, you followed the mandate of the popular vote, old people would be scrambling to save themselves in parking lots while kids ran wild knowing the cars would never hit them. An entertaining fantasy spectacle, to be sure, but not quite how most of us want to live. As Jacques tells it, the trolley problem the Moral Machine represents is basically a metaphor that has eaten its young. Get rid of it! This was a rare moment of near-universal agreement. "I've been longing for the trolley problem to die," robotics pioneerRobin Murphy said. Jacques herself was more measured: "Philosophers need to take responsibility for what happens when we leave our tools lying around."

The biggest thing I've learned in all the law conferences I go to is that law proceeds by analogy and metaphor. You see this everywhere: Kate Darling is trying to understand how we might integrate robots into our lives by studying the history of domesticating animals; Ian Kerr and Carys Craig are trying to deromanticize "the author" in discussions of AI and copyright law; the "property" in "intellectual property" draws an uncomfortable analogy to physical objects; and Hideyuki Matsumi is trying to think through robot registration by analogy to Japan's Koseki family registration law.

Getting the metaphors right is therefore crucial, which explains, in turn, why it's important to spend so much effort understanding what the technology can really do and what it can't. You have to stop buying the images of driverless cars to produce something like the "handoff model" proposed by Jake Goldenfein, Deirdre Mulligan, and Helen Nissenbaum to explore the permeable boundaries between humans and the autonomous or connected systems driving their cars. Similarly, it's easy to forget, as Mulligan said in introducing her paper with Daniel N. Kluttz, that in "machine learning" algorithms learn only from the judgments at the end; they never see the intermediary reasoning stages.

So metaphor matters. At this point I had a blinding flash of realization. This is why no one can agree about Brexit. *Brexit* is a trolley problem. Small wonder Jacques called the Moral Machine a "monster".

Previous We Robot events as seen by net.wars: 2018 workshop and conference; 2017; 2016 workshop and conference, 2015; 2013, and 2012. We missed 2014.

Illustrations: The Moral Labyrinth art installation, by Sarah Newman and Jessica Fjeld, at We Robot 2019; Google driverless car.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 10:27 PM | Permalink | TrackBacks (0)

The futurist Anders Sandberg has often observed that we call it "artificial intelligence" only as long as it doesn't work; after that it's simply "automation". This week, Rich Walker, the managing director of Shadow Robot, said the same thing about robotics. No one calls a self-driving car or a washing machine a robot, for example. Then again, a friend does indeed call the automated tea maker that reliably wakes up every morning before he does "the robot", which suggests we only call things "robots" when we can mock their limitations.

Walker's larger point was robotics, like AI, suffers from confusion between the things people think it can do and the things it can actually do. The gap in AI is so large, that effectively the term now has two meanings, a technological one revolving around the traditional definition of AI, and a political one, which includes the many emerging new technologies - machine learning, computer vision, and so on - that we need to grapple with.

When, last year, we found that Shadow Robot was collaborating on research into care robots it seemed time for a revisit: the band of volunteers I met in 1997 and the tiny business it had grown into in 2009 had clearly reached a new level.

Social care is just one of many areas Shadow is exploring; others include agritech and manufacturing. "Lots are either depending on other pieces of technology that are not ready or available yet or dependent on economics that are not working in our favor yet," Walker says. Social care is an example of the latter; using robots outside of production lines in manufacturing is an example of the former. "It's still effectively a machine vision problem." That is, machine vision is not accurate enough with high enough reliability. A 99.9% level of accuracy means a failure per shift in a car manufacturing facility.

Getting to Shadow Robot's present state involved narrowing down the dream founder Richard Greenhill conceived after reading a 1980s computer programming manual: to build a robot that could bring him a cup of tea. The project, then struggling to be taken seriously as it had no funding and Greenhill had no relevant degrees, built the first robot outside Japan that could stand upright and take a step; the Science Museum included it in its 2017 robot exhibition.

Greenhill himself began the winnowing process, focusing on developing a physical robot that could function in human spaces rather than AI and computer vision, reasoning that there were many others who would do that. Greenhill recognized the importance of the hand, but it was Walker who recognized its commercial potential: "To engage with real-world, human-scale tasks you need hands."

The result, Walker says, is, "We build the best robot hand in the world." And, he adds, because several employees have worked on all the hands Shadow has ever built, "We understand all the compromises we've made in the designs, why they're there, and how they could be changed. If someone asks for an extra thumb, we can say why it's difficult but how we could do it."

Meanwhile, the world around Shadow has changed to include specialists in everything else. Computer vision, for example: "It's outside of the set of things we think we should be good at doing, so we want others to do it who are passionate about it," Walker says. "I have no interest in building robot arms, for example. Lots of people do that." And anyway, "It's incredibly hard to do it better than Universal Robots" - which itself became the nucleus of a world-class robotics cluster in the small Danish city of Odense.

Specialization may be the clearest sign that robotics is growing up. Shadow's current model, mounted on a UR arm, sports fingertips developed by SynTouch. With SynTouch and HaptX, Shadow collaborated to create a remote teleoperation system using HaptX gloves in San Francisco to control a robot hand in London following instructions from a businessman in Japan. The reason sounds briefly weird: All Nippon Airways is seeking new markets by moving into avatars and telepresence. It sounds less weird when Walker says ANA first thought of teleportation...and then concluded that telepresence might be more realistic.

Shadow's complement of employees is nearing 40, and they've moved from the undifferentiated north London house they'd worked in since the 1990s, dictated, Walker says, by buying a new milling machine. Getting the previous one in, circa 2007, required taking out the front window and the stairs and building a crane. Walker's increasing business focus reflects the fact that the company's customers are now as often commercial companies as the academic and research institutions that used to form their entire clientele.

For the future, "We want to improve tactile sensing," Walker says. "Touch is really hard to get robots to do well." One aspect they're particularly interested in for teleoperation is understanding intent: when grasping something, does the controlling human want to pinch, twist, hold, or twist it? At the moment, to answer that he imagines "the robot equivalent" of Clippy that asks, "It looks like you're trying to twist the wire. Do you mean to roll it or twist it?" Or even: "It looks like you're trying to defuse a bomb. Do you want to cut the red wire or the black wire?" Well, do ya, punk?

Illustrations: Rich Walker, showing off the latest model, which includes fingertips from HaptX and a robot arm from Universal Robotics; the original humanoid biped, on display at the Science Museum.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 12:45 PM | Permalink | TrackBacks (0)

"As a citizen, how will I know I live in a smarter city, and how will life be different?" This question was probably the smartest question asked at yesterday's Westminster Forum seminar on smart cities (PDF); it was asked by Tony Sceales, acting as moderator.

"If I feel safe and there's less disruption," said Peter van Manen. "You won't necessarily know. Thins will happen as they should. You won't wake up and say, 'I'm in the city of the future'," said Sam Ibbott. "Services become more personalized but less visible," said Theo Blackwell the Chief Digital Office for London.

"Frictionless" said Jacqui Taylor, offering it as the one common factor she sees in the wildly different smart city projects she has encountered. I am dubious that this can ever be achieved: one person's frictionless is another's desperate frustration: streets cannot be frictionless for *both* cars and cyclists, just as a city that is predicted to add 2 million people over the next ten years can't simultaneously eliminate congestion. "Working as intended" was also heard. Isn't that what we all wish computers would do?

Blackwell had earlier mentioned the "legacy" of contactless payments for public transport. To Londoners smushed into stuffed Victoria Line carriages in rush hour, the city seems no smarter than it ever was. No amount of technological intelligence can change the fact that millions of people all want to go home at the same time or the housing prices that force them to travel away from the center to do so. We do get through the ticket barriers faster.

"It's just another set of tools," said Jennifer Schooling. "It should feel no different."

The notion of not knowing as the city you live in smartens up should sound alarm bells. The fair reason for that hiddenness is the reality that, as Sara Degli Esposti pointed out at this year's Computers, Privacy, and Data Protection, this whole area is a business-to-business market. "People forget that, especially at the European level. Users are not part of the picture, and that's why we don't see citizens engaged in smart city projects. Citizens are not the market. This isn't social media."

She was speaking at CPDP's panel on smart cities and governance, convened by the University of Stirling's William Webster, who has been leading a research project, CRISP, to study these technologies. CRISP asked a helpfully different question: how can we use smart city technologies to foster citizen engagement, coproduction of services, development of urban infrastructure, and governance structures?

The interesting connection is this: it's no surprise when CPDP's activists, regulators, and academics talk about citizen engagement and participation, or deplore a model in which smart cities are a business-led excuse for corporate and government, surveillance. The surprise comes when two weeks later the same themes arise among Westminster Forum's more private and public sector speakers and audience. These are the people who are going to build these new programs and services, and they, too, are saying they're less interested in technology and more interested in solving the problems that keep citizens awake at night: health, especially.

There appears to be a paradigm shift beginning to happen as municipalities begin to seriously consider where and on what to spend their funds.

However, the shift may be solely European. At CPDP, Canadian surveillance studies researcher David Murakami Wood told the story of Toronto, where (Google owner) Alphabet subsidiary Sidewalk Labs swooped in circa 2014 with proposals to redevelop the Quayside area of Toronto in partnership with Waterfront Toronto. The project has been hugely controversial - there were hearings this week in Ottawa, the provincial capital.

As Murakami Wood's tells it, for Sidewalk Labs the area is a real-world experiment using real people's lives as input to create products the company can later sell elsewhere. The company has made clear it intends to keep all the data the infrastructure generates on its servers in the US as well as all the intellectual property rights. This, Murakami Wood argued, is the real cost of the "free" infrastructure. It is also, as we're beginning to see elsewhere, the extension of online tracking or, as Murakami Wood put it, surveillance capitalism into the physical world: cultural appropriation at municipal scale from a company that has no track record in building buildings, or even publishing detailed development plans. Small wonder that Murakami Wood laughed when he heard Sidewalk Labs CEO Dan Doctoroff impress a group of enthusiastic young Canadian bankers with the news that the company had been studying cities for *two years*.

Putting these things together, we have, as Andrew Adams suggested, three paradigms, which we might call US corporate, Chinese authoritarian, and, emerging, European participatory and cooperative. Is this the choice?

Yes and no. Companies obviously want to develop systems once, sell them everywhere. Yet the biggest markets are one-off outliers. "Croydon," said Blackwell, "is the size of New Orleans." In addition, approaches vary widely. Some places - Webster mentioned Glasgow - are centralized command and control; others - Brazil - are more bottom-up. Rick Robinson finds that these do not meet in the middle.

The clear takeaway overall is that local context is crucial in shaping smart city projects and despite some common factors each one is different. We should built on that.

Illustrations: Fritz Lang's Metropolis (1927).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 2:18 PM | Permalink | Comments (0) | TrackBacks (0)

Recently, Michael Salmony, who has appeared here before appeared horrified to discover open access, the movement for publishing scientific research so it's freely accessible to the public (who usually paid for it) instead of closed to subscribers. In an email, he wrote, "...looks like the Internet is now going to destroy science as well".

This is not my view.

The idea about science that I grew up with was that scientists building on and reviewing each other's work is necessary for good science, a self-correcting process that depends on being able to critique and replicate each other's work. So the question we should ask is: does the business model of traditional publishing support that process? Are there other models that would support that process better? Science spawns businesses, serves businesses, and may even be a business itself, but good-quality science first serves the public interest.

There are three separate issues here. The first is the process of science itself: how best to fund, support, and nurture it. The second is the business model of scientific *publishing*. The third, which relates to both of those, is how to combat abuse. Obviously, they're interlinked.

The second of these is the one that resonates with copyright battles past. Salmony: "OA reminds me warmly of Napster disrupting music publishing, but in the end iTunes (another commercial, quality controlled) model has won."

iTunes and the music industry are not the right models. No one dies of lack of access to Lady Gaga's latest hit. People *have* died through being unable to afford access to published research.

Plus, the push is coming from an entirely different direction. Napster specifically and file-sharing generally were created by young, anti-establishment independents who coded copyright bypasses because they could. The open access movement began with a statement of principles codified by university research types - mavericks, sure, but representing the Public Library of Science, Open Society Institute, BioMed Central, and universities in Montreal, London, and Southampton. My first contact with the concept was circa 1993, when World Health Organization staffer Christopher Zielinski raised the deep injustice of pricing research access out of developing countries' reach.

Sci-Hub is a symptom, not a cause. Another symptom: several months ago, 60 German universities canceled their subscriptions to Elsevier journals to protest the high fees and restricted access. Many scientists are offended at the journals' expectation that they will write papers for free and donate their time for peer review while then charging them to read the published results. One way we know this is that Sci-Hub builds its giant cache via educational institution proxies that bypass the paywalls. At least some of these are donated by frustrated people inside those institutions. Many scientists use it.

As I understand it, publication costs are incorporated into research grants; there seems no reason why open access should impede peer review or indexing. Why shouldn't this become financially sustainable and assure assure quality control as before?

A more difficult issue is that one reason traditional journals still matter is that academic culture has internalized their importance in determining promotions and tenure. Building credibility takes time, and many universities have been slow to adapt. However, governments and research councils in Germany, the UK, and South Africa are all pushing open access policies via their grant-making conditions.

Plus, the old model is no longer logistically viable in many fields as the pace of change accelerates. Computer scientists were first to ignore it, relying instead on conference proceedings and trading papers and research online.

Back to Salmony: "Just replacing one bad model with another one that only allows authors who can afford to pay thousands of dollars (or is based on theft, like Sci Hub) and that threatens the quality (edited, peer review, indexed etc) sounds less than convincing." In this he's at odds with scientists such as Ben Goldacre, who in 2007 called open access "self-evidently right and good".

This is the first issue. In 1992, Marcel C. LaFollette's Stealing into Print: Fraud, Plagiarism, and Misconduct in Scientific Publishing documented many failures of traditional peer review. In 2010, the Greek researcher John Ioannidis established how often medical research is retracted. At Retraction Watch, science journalist Ivan Oransky finds remarkable endemic sloppiness and outright fraud. Admire the self-correction, but the reality is that journals have little interest in replication, preferring newsworthy new material - though not *too* new.

Ralph Merkle, the "third man", alongside Whit Diffie and Martin Hellman, inventing public key cryptography, has complained that journals favor safe, incremental steps. Merkle's cryptography idea was dismissed with: "There is nothing like this in the established literature." True. But it was crucial for enabling ecommerce.

Salmony's third point: "[Garbage] is the plague of the open Internet", adding a link to a Defon 26 talk. Sarah Jeong's Internet of Garbage applies.

Abuse and fakery are indeed rampant, but a lot is due to academic incentives. For several years, my 2014 article for IEEE Security & Privacy explaining the Data Retention and Investigatory Powers Act (2014) attracted invitations to speak at (probably) fake conferences and publish papers in (probably) fake journals. Real researchers tell me this is par for the course. But this is a problem of human predators, not "the open Internet", and certainly not open access.

Illustrations: Participants in drafting the Budapest principles (via Wikimedia).

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 3:02 AM | Permalink | Comments (0) | TrackBacks (0)

The robot in the video is actually a giant hoist attached to the ceiling. It has big grab bars down at the level of the person sitting on the edge of the bed, waiting. When the bars approach, she grabs them, and lets the robot slowly help her up into a standing position, and then begins to move forward.

This is not how any of us imagines a care robot, but I am persuaded this is more like our future than the Synths in 2015's Humans, which are incredibly humanoid (helpfully for casting) but so, so far from anything ready for deployment. This thing, which Praminda Caleb-Solly showed at work in a demonstration video at Tuesday's The Shape of Things conference, is a work in progress. There are still problems, most notably that your average modern-build English home has neither high enough ceilings nor enough lateral space to accommodate it. My bedroom is about the size of the stateroom in the Marx Brothers movie A Night at the Opera; you'd have to put it in the hall and hope the grab bar assembly could reach through the doorway. But still.

As the news keeps reminding us, the the Baby Boomer bulge will soon reach frailty. In industrialized nations, where mobility, social change, and changed expectations have broken up extended families, need will explode. In the next 12 years, Caleb-Solly said, a fifth of people over 80 - 4.8 million people in the UK - will require regular care. Today, the National Health Service is short almost 250,000 staff (a problem Brexit exacerbates wholesale). Somehow, we'll have to find 110,000 people to work in social care in England alone. Technology is one way to help fill that gap. Today, though, 30% of users abandon their assistive technologies; they're difficult to adapt to changing needs, difficult to personalize, and difficult to interact with.

Personally, I am not enthusiastic about having a robot live in my house and report on what I do to social care workers. But I take Caleb-Solly's point when she says, "We need smart solutions that can deal with supporting a healthy lifestyle of quality". That ceiling-hoist robot is part of a modular system that can add functions and facilities as people's needs and capacity change over time.

In movies and TV shows, robot assistants are humanoids, but that future is too far away to help the onrushing 4.8 million. Today's care-oriented robots have biological, but not human, inspirations: the PARO seal, or Pepper, which Caleb-Solly's lab likes because it's flexible and certified for experiments in people's homes. You may wonder what intelligence, artificial or otherwise, a walker needs, but given sensors and computational power the walker can detect how its user is holding it, how much weight it's bearing, whether the person's balance is changing, and help them navigate. I begin to relax: this sounds reasonable. And then she says, "Information can be conveyed to the carer team to assess whether something changed and they need more help," and I close down with suspicion again. That robot wants to rat me out.

There's a simple fix for that: assume the person being cared for has priorities and agency of their own, and have the robot alert them to the changes and let them decide what they want to do about it. That approach won't work in all situations; there are real issues surrounding cognitive decline, fear, misplaced pride, and increasing multiple frailties that make self-care a heavy burden. But user-centered design can't merely mean testing the device with real people with actual functional needs; the concept must extend to ownership of data and decision-making. Still, the robot walker in Caleb-Solly's lab taught her how to waltz. That has to count for something.

The project - CHIRON, for Care at Home using Intelligent Robotic Omni-functional Nodes - is a joint effort between Three Sisters Care, Caleb-Solly's lab, and Shadow Robot, and funded with £2 million over two years by Innovate UK.

Shadow Robot was the magnet that brought me here. One of the strangest and most eccentric stories in an already strange and eccentric field, Shadow began circa 1986, when the photographer Richard Greenhill was becalmed on a ship with nothing to do for several weeks but read the manual for the Sinclair ZX 81. His immediate thought: you could control a robot with one of those! His second thought: I will build one.

By 1997, Greenhill's operation was a band of volunteers meeting every week in a north London house filled with bits of old wire and electronics scrounged from junkyards. By then, Greenhill had most of a hominid with deceptively powerful braided-cloth "air muscles". By my next visit, in 2009, former volunteer Rich Walker had turned Shadow into a company selling a widely respected robot hand, whose customers include NASA, MIT, and Carnegie-Mellon. Improbably, the project begun by the man with no degrees, no funding, and no university affiliation has outlasted numerous more famous efforts filled with degree-bearing researchers who used up their funding, published, and disbanded. And now it's contributing robotics research expertise to CHIRON.

Seen Tuesday, Greenhill was eagerly outlining a future in which we can all build what we need and everyone can live for free. Well, why not?

Illustrations: Praminda Caleb-Solly presenting on Tuesday (Kois Miah); Pepper; Richard Greenhill demonstrating his personally improved scooter.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 9:34 PM | Permalink | Comments (0) | TrackBacks (0)

"I think we would distinguish between the Internet and Facebook. They're not the same thing." With this, the MP Damian Collins (Conservative, Folkstone and Hythe) closed Tuesday's hearing on fake news, in which representatives of nine countries, combined population 400 million, posed questions to Facebook VP for policy Richard Allan, proxying for non-appearing CEO Mark Zuckerberg.

Collins was correct when you're talking about the countries present: UK, Ireland, France, Belgium, Latvia, Canada, Argentina, Brazil, and Singapore. However, the distinction is without a difference in numerous countries where poverty and no-cost access to Facebook or its WhatsApp subsidiary keeps the population within their boundaries. Foreseeing this probable outcome, India's regulator banned Facebook's Free Basics on network neutrality grounds.

Much less noticed, the nine also signed a set of principles for governing the Internet. Probably the most salient point is the last one, which says technology companies "must demonstrate their accountability to users by making themselves fully answerable to national legislatures and other organs of representative democracy". They could just as well have phrased it, "Hey, Zuckerberg: start showing up."

This was, they said, the first time multiple parliaments have joined together in the House of Commons since 1933, and the first time ever that so many nations assembled - and even that wasn't enough to get Zuckerberg on a plane. Even if Allan was the person best-placed to answer the committee's questions, it looks bad, like you think your company is above governments.

The difficulty that has faced would-be Internet regulators from the beginning is this: how do you get 200-odd disparate cultures to agree? China would openly argue for censorship; many other countries would openly embrace freedom of expression while happening to continue expanding web blocking, filtering, and other restrictions. We've seen the national disparities in cultural sensitivities played out for decades in movie ratings and TV broadcasting rules. So what's striking about this declaration is that nine countries from three continents have found some things they can agree on - and that is that libertarian billionaires running the largest and most influential technology companies should accept the authority of national governments. Hence, the group's first stated principle: "The internet is global and law relating to it must derive from globally agreed principles". It took 22 years, but at last governments are responding to John Perry Barlow's 1996 Declaration of the Independence of Cyberspace: "Not bloody likely."

Even Allan, a member of the House of Lords and a former MP (LibDem, Sheffield Hallam), admitted, when Collins asked how he thought it looked that Zuckerberg had sent a proxy to testify, "Not great!"

The governments' principles, however, are a statement of authority, not a bill of rights for *us*, a tougher proposition that many have tried to meet. In 2010-2012, there was a flurry of attempts. Then-US president Barack Obama published a list of privacy principles; the 2010 Computers, Freedom, and Privacy conference, led by co-chair Jon Pincus, brainstormed a bill of rights mostly aimed at social media; UK deputy Labour leader Tom Watson ran for his seat on a platform of digital rights (now gone from his website); and US Congressman Darrell Issa (R-OH) had a try.

Then a couple of years ago, Cybersalon began an effort to build on all these attempts to draft a bill of rights hoping it would become a bill in Parliament. Labour drew on it for its Digital Democracy Manifesto (PDF) in 2016 - though this hasn't stopped the party from supporting the Investigatory Powers Act.

The latest attempt came a few weeks ago, when Tim Berners-Lee launched a contract for the web, which has been signed by numerous organizations and individuals. There is little to object to: universal access, respect for privacy, free expression, and human rights, civil discourse. Granted, the contract is, like the Bishop of Oxford's ten commandments for artificial intelligence, aspirational more than practically prescriptive. The civil discourse element is reminiscent of Tim O'Reilly's 2007 Code of Conduct, which many, net.wars included, felt was unworkable.

The reality is that it's unlikely that O'Reilly's code of conduct or any of its antecedents and successors will ever work without rigorous human moderatorial intervention. There's a similar problem with the government pledges: is China likely to abandon censorship? Next year half the world will be online - but alongside the Contract a Web Foundation study finds that the rate at which people are getting online has fallen sharply since 2015. Particularly excluded are women and the rural poor, and getting them online will require significant investment in not only broadband but education - in other words, commitments from both companies and governments.

Popular Mechanics calls the proposal 30 years too late; a writer on Medium calls it communist; and Bloomberg, among others, argues that the only entities that can rein in the big technology companies is governments. Yet the need for them to do this appears nowhere in the manifesto. "...The web is long past attempts at self-regulation and voluntary ethics codes," Bloomberg concludes.

Sadly, this is true. The big design error in creating both the Internet and the web was omitting human psychology and business behavior. Changing today's situation requires very big gorillas. As we've seen this week, even nine governments together need more weight.

Illustrations: Zuckerberg's empty chair in the House of Commons.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:02 PM | Permalink | Comments (0) | TrackBacks (0)

I regularly get Friend requests on Facebook from things I doubt are real people. They are always male and, at a guess, 40-something, have no Friends in common with me, and don't bother to write a message explaining how I know them. If I take the trouble to click through to their profiles, their Friends lists are empty. This week's request, from "Smith Thomson", is muscled, middle-aged, and slightly brooding. He lists his workplace as a US Army base and his birthplace as Houston. His effort is laughably minimal: zero Friends and the only profile content is the cover photograph plus a second photo with a family in front of a Disney castle, probably Photoshopped. I have a nasty, suspicious mind, and do not accept the request.

One of the most interesting projects under the umbrella of the Research Institute for Science of Cyber Security is Detecting and Preventing Mass-Marketing Fraud, led from the University of Warwick by Monica Whitty, and explained here. We tend to think of romance scams in particular, less so advance-fee fraud, as one-to-one rip-offs. Instead, the reality behind them is highly organized criminals operating at scale.

This is a billion-dollar industry with numerous victims. On Monday, the BBC news show Panorama offered a carefully worked example. The journalists followed the trail of these "catfish" by setting up a fake profile and awaiting contact, which quickly arrived. Following clues and payment instructions led the journalists to the scammer himself, in Lagos, Nigeria. One of the victims in particular displays reactions Whitty has seen in her work, too: even when you explain the fraud, some victims still don't recognize the same pattern when they are victimized again. Panorama's saddest moment is an older man who was clearly being retargeted after having already been fleeced of £100,000, his life savings. The new scammer was using exactly the same methodology, and yet he justified sending his new "girlfriend" £500 on the basis that it was comparatively modest, though at least he sounded disinclined to send more. He explained his thinking this way: "They reckon that drink and drugs are big killers. Yeah, they are, but loneliness is a bigger killer than any of them, and trying to not be lonely is what I do every day."

I doubt Panorama had to look very hard to find victims. They pop up a lot at security events, where everyone seems to know someone who's been had: the relative whose computer they had to clean after they'd been taken in by a tech support scam, the friend they'd had to stop from sending money. Last year, one friend spent several months seeking restitution for her mother, who was at least saved from the worst by an alert bank teller at her local branch. The loss of those backstops - people in local bank branches and other businesses who knew you and could spot when you were doing something odd - is a largely unnoticed piece of why these scams work.

In a 2016 survey, Microsoft found that two-thirds of US consumers had been exposed to a tech support scam in the previous year. In the UK in 2016, a report by the US Better Business Bureau says (PDF) , there were more than 34,000 complaints about this type of fraud alone - and it's known that less than 10% of victims complain. Each scam has its preferred demographic. Tech support fraud doesn't typically catch older people, who have life experience and have seen other scams even if not this particular one. The biggest victims of this type of scam are millennials aged 18 to 34 - with no gender difference.

DAPM's meeting mostly focused on dating scams, a particular interest of Whitty's because the emotional damage, on top of the financial damage, is so fierce. From her work, I've learned that the military connection "Smith Thomson" claimed is a common pattern. Apparently some people are more inclined to trust a military background, and claiming that they're located on a military base makes it easy for scammers to dodge questions about exactly what they're doing and where they are and resist pressure to schedule a real-life meeting.

Whitty and her fellow researchers have already discovered that the standard advice we give people doesn't work. "If something looks too good to be true it usually is" is only meaningful at the beginning - and that's not when the "too good to be true" manifests itself. Fraudsters know to establish trust before ratcheting up the emotions and starting to ask - always urgently - for money. By then, requests that would raise alarm flags at the beginning seem like merely the natural next steps in a developed relationship. Being scammed once gets you onto a "suckers list", ripe for retargeting - like Panorama's victim. These, too, are not new; they have been passed around among fraudsters for at least a century.

The point of DAPM's research is to develop interventions. They've had some statistically significant success with instructions teaching people to recognize scams. However, this method requires imparting a lot of information, which means the real conundrum is how you motivate people to participate when most believe they're too smart to get caught. The situation is very like the paranormal claims The Skeptic deals with: no matter how smart you are or how highly educated, you, too, can be fooledz. And, unlike in other crimes, DAPM finds, 52% of these victims blame themselves.

Illustrations: Cupid's Message (via Missouri Historical Society.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 1:02 PM | Permalink | Comments (0) | TrackBacks (0)

"Models are best for understanding, but they are inherently wrong," Helen Dacre said, evoking robotics engineer Bill Smart on sensors. Dacre was presenting a tool that combines weather forecasts, air quality measurements, and other data to help airlines and other stakeholders quickly assess the risk of flying after a volcanic eruption. In April 2010, when Iceland's Eyjafjallajökull blew its top, European airspace shut down for six days at an estimated overall cost of £1.1 billion. Since then, engine manufacturers have studied the effect of atmospheric volcanic ash on aircraft engines, and are finding that a brief excursion through peak levels of concentration is less damaging than prolonged exposure at lower levels. So, do you fly?

This was one of the projects presented at this week's conference of the two-year-old network Challenging Radical Uncertainty in Science, Society and the Environment (CRUISSE). To understand "radical uncertainty", start with Frank Knight, who in 1921 differentiated between "risk", where the outcomes are unknown but the probabilities are known, and uncertainty, where even the probabilities are unknown. Timo Ehrig summed this up as "I know what I don't know" versus "I don't know what I don't know", evoking Donald Rumsfeld's "unknown unknowns". In radical uncertainty decisions, existing knowledge is not relevant because the problems are new: the discovery of metal fatigue in airline jets; the 2008 financial crisis; social media; climate change. The prior art, if any, is of questionable relevance. And you're playing with live ammunition - real people's lives. By the million, maybe.

How should you change the planning system to increase the stock of affordable housing? How do you prepare for unforeseen cybersecurity threats? What should we do to alleviate the impact of climate change? These are some of the questions that interested CRUISSE founders Leonard Smith and David Tuckett. Such decisions are high-impact, high-visibility, with complex interactions whose consequences are hard to foresee.

It's the process of making them that most interests CRUISSE. Smith likes to divide uncertainty problems into weather and climate. With "weather" problems, you make many similar decisions based on changing input; with "climate" problems your decisions are either a one-off or the next one is massively different. Either way, with climate problems you can't learn from your mistakes: radical uncertainty. You can't reuse the decisions; but you *could* reuse the process by which you made the decision. They are trying to understand - and improve - those processes.

This is where models come in. This field has been somewhat overrun by a specific type of thinking they call OCF, for "optimum choice framework". The idea there is that you build a model, stick in some variables, and tweak them to find the sweet spot. For risks, where the probabilities are known, that can provide useful results - think cost-benefit analysis. In radical uncertainty...see above. But decision makers are tempted to build a model anyway. Smith said, "You pretend the simulation reflects reality in some way, and you walk away from decision making as if you have solved the problem." In his hand-drawn graphic, this is falling off the "cliff of subjectivity" into the "sea of self-delusion".

Uncertainty can come from anywhere. Kris de Meyer is studying what happens if the UK's entire national electrical grid crashes. Fun fact: it would take seven days to come back up. *That* is not uncertain. Nor are the consequences: nothing functioning, dark streets, no heat, no water after a few hours for anyone dependent on pumping. Soon, no phones unless you still have copper wire. You'll need a battery or solar-powered radio to hear the national emergency broadcast.

The uncertainty is this: how would 65 million modern people react in an unprecedented situation where all the essentials of life are disrupted? And, the key question for the policy makers funding the project, what should government say? *Don't* fill your bathtub with water so no one else has any? *Don't* go to the hospital, which has its own generators, to charge your phone?

"It's a difficult question because of the intention-behavior gap," de Meyer said. De Meyer is studying this via "playable theater", an effort that starts with a story premise that groups can discuss - in this case, stories of people who lived through the blackout. He is conducting trials for this and other similar projects around the country.

In another project, Catherine Tilley is investigating the claim that machines will take all our jobs . Tilley finds two dominant narratives. In one, jobs will change, not disappear, and automation more of them, enhanced productivity, and new wealth. In the other, we will be retired...or unemployed. The numbers in these predictions are very large, but conflicting, so they can't all be right. What do we plan for education and industrial policy? What investments do we make? Should we prepare for mass unemployment, and if so, how?

Tilley identified two common assumptions: tasks that can be automated will be; automation will be used to replace human labor. But interviews with ten senior managers who had made decisions about automation found otherwise. Tl;dr: sectoral, national, and local contexts matter, and the global estimates are highly uncertain. Everyone agrees education is a partial solution - "but for others, not for themselves".

Here's the thing: machines are models. They live in model land. Our future depends on escaping.

Illustrations: David Tuckett and Lenny Smith.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 10:28 AM | Permalink

These two articles made a good pairing: Oscar Schwartz's critique of AI hype in the Guardian, and Jennings Brown's takedown of IBM's Watson in real-world contexts. Brown's tl;dr: "This product is a piece of shit," a Florida doctor reportedly told IBM in the leaked memos on which Gizmodo's story is based. "We can't use it for most cases."

Watson has had a rough ride lately: in August 2017 Brown catalogued mounting criticisms of the company and its technology; that June, MIT Technology Review did, too. All three agree: IBM's marketing has outstripped Watson's technical capability.

That's what Schwartz is complaining about: even when scientists make modest claims; media and marketing hype it to the hilt. As a result, instead of focusing on design and control issues such as how to encode social fairness into algorithms, we're reading Nick Bostrom's suggestion that an uncontrolled superintelligent AI would kill humanity in the interests of making paper clips or the EU's deliberation about whether robots should have rights. These are not urgent issues, and focusing on them benefits only vendors who hope we don't look too closely at what they're actually doing.

Schwartz's own first example is the Facebook chat bots that were intended to simulate negotiation-like conversations. Just a couple of days ago someone referred to this as bots making up their own language and cited it as an example of how close AI is to the Singularity. In fact, because they lacked the right constraints, they just made strange sentences out of normal English words. The same pattern is visible with respect to self-driving cars.

You can see why: wild speculation drives clicks - excuse me, monetized eyeballs - but understanding what's wrong with how most of us think about accuracy in machine learning is *mathy*. Yet understanding the technology's very real limits is crucial to making good decisions about it.

With medicine, we're all particularly vulnerable to wishful thinking, since sooner or later we all rely on it for our own survival (something machines will never understand). The UK in particular is hoping AI will supply significant improvements because of the vast amount of patient, that is, training, data the NHS has to throw at these systems. To date, however, medicine has struggled to use information technology effectively.

Attendees at We Robot have often discussed what happens when the accuracy of AI diagnostics outstrips that of human doctors. At what point does defying the AI's decision become malpractice? At this year's conference, Michael Froomkin presented a paper studying the unwanted safety consequences of this approach (PDF).

The presumption is that the AI system's ability to call on the world's medical literature on top of generations of patient data will make it more accurate. But there's an underlying problem that's rarely mentioned: the reliability of the medical literature these systems are built on. The true extent of this issue began to emerge in 2005, when John Ioannidis published a series of papers estimating that 90% of medical research is flawed. In 2016, Ioannidis told Retraction Watch that systematic reviews and meta-analyses are also being gamed because of the rewards and incentives involved.

The upshot is that it's more likely to be unclear, when doctors and AI disagree, where to point the skepticism. Is the AI genuinely seeing patterns and spotting things the doctor can't? (In some cases, such as radiology, apparently yes. But clinical trials and peer review are needed.) Does common humanity mean the doctor finds clues in the patient's behavior and presentation that an AI can't? (Almost certainly.) Is the AI neutral in ways that biased doctors may not be? Stories of doctors not listening to patients, particularly women, are legion. Yet the most likely scenario is that the doctor will be the person entering data - which means the machine will rely on the doctor's interpretation of what the patient says. In all these conflicts, what balance do we tell the AI to set?

Much sooner than Watson will cure cancer we will have to grapple with which AIs have access to which research. In 2015, the team responsible for drafting Liberia's ebola recovery plan in 2014 wrote a justifiably angry op-ed in the New York Times. They had discovered that thousands of Liberians could have been spared ebola had a 1982 paper for Annals of Virology been affordable for them to read; it warned that Liberia needed to be included in the ebola virus endemic zone. Discussions of medical AI to date appear to handwave this sort of issue, yet cost structures, business models, and use of medical research are crucial. Is the future open access, licensing and royalties, all-you-can-eat subscriptions?

The best selling point for AI is that its internal corpus of medical research can be updated a lot faster than doctors' brains can be. In 2017, David Epstein wrote at ProPublica, many procedures and practices become entrenched, and doctors are difficult to dissuade from prescribing them even when they've been found useless. In the US, he added, the 21st Century Cures Act, passed in December 2016, threatens to make all this worse by lowering standards of evidence.

All of these are pressing problems no medical AI can solve. The problem, as usual, is us.

Illustrations: Watson wins at Jeopardy (via Wikimedia)

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 3:32 PM | Permalink | Comments (0) | TrackBacks (0)

Most of the "us" who might read this rarely stop to marvel at the wonder that is our daily trust in the society that surrounds us. One of the worst aspects of London Underground's incessant loud reminders to report anything suspicious - aside from the slogan, which is dumber than a bag of dead mice - is that it interrupts the flow of trust. It adds social friction. I hear it, because I don't habitually block out the world with headphones.

Friction is, of course, the thing that so many technologies are intended to eliminate. And they might, if only we could trust them.

Then you read things like this news, that Philip Morris wants to harvest data from its iQOS e-cigarette. If regulators allow, Philip Morris will turn on functions in the device's internal chips that capture data on its user's smoking habits, not unlike ebook readers' fine-grained data collection. One can imagine the data will be useful for testing strategies for getting people to e-smoke longer.

This example did not arrive in time for this week's Nuances of Trust event, hosted by the Alliance for Internet of Things Innovation (AIOTI) and aimed at producing intelligent recommendations for how to introduce trust into the Internet of Things. But, so often, it's the company behind the devices you can't trust. For another example: Volkswagen.

Partly through the problem-solving session, we realized we had regenerated three of Lawrence Lessig's four modalities of constraining behavior: technology/architecture, law, market, social norms. The first changes device design to bar shipping loads of data about us to parts unknown; law pushes manufacturers into that sort of design, even if it cost more; market would mean people refused to buy privacy-invasive devices, and social norms used to be known as "peer pressure". Right now, technology is changing faster than we can create new norms. If a friend has an Amazon Echo at home, does entering their house constitute signing Amazon's privacy policy? Should they show me the privacy policy before I enter? Is it reasonable to ask them to turn it off while I'm there? We could have asked questions like "Are you surreptitiously recording me?" at any time since portable tape recorders were invented, but absent a red, blinking light we felt safe in assuming no. Now, suddenly, trusting my friend requires also trusting a servant belonging to a remote third party. If I don't, it's a social cost - to me, and maybe to my friend, but not to Amagoople.

On Tuesday, Big Brother Watch provided a far more alarming example when director Silkie Carlo launched BBW's report on automated facial recognition (PDF). Now, I know the technically minded will point out grumpily that all facial recognition is "automated" because it's a machine what does it, but what BBW means is a system in which CCTV and other cameras automatically feed everything they gather into a facial recognition system that sprinkles AI fairy dust and pops out Persons of Interest (I blame TV). Various UK police have deployed these AFR systems at concerts and football and rugby games; at the 2016 and 2017 Notting Hill Carnivals; on Remembrance Sunday 2017 to restrict "fixated individuals"; and at peaceful demonstrations. On average, fewer than 9% of matches were accurate; but that's little consolation when police pick you out of the hordes arriving by train for an event and insist on escorting you under watch. The system London's Met Police used had a false positive rate of over 98%! How does a system like that even get out of the lab?

Neither the police nor the Home Office seem to think that bringing in this technology requires any public discussion; when asked they play the Yes, Minister game of pass the policy. Within the culture of the police, it may in fact be a social norm that invasive technologies whose vendors promise magical preventative results should be installed as quickly as possible before anyone can stop them. Within the wider culture...not so much.

This is the larger problem with what AIOTI is trying to do. It's not just that the devices themselves are insecure, their risks capricious, and the motives of their makers suspect. It's that long after you've installed and stopped thinking about a system incorporating these devices someone else can come along to subvert the whole thing. How do you ensure that the promise you make today cannot be broken by yourself or others in future? The problem is near-identical to the one we face with databases: each may be harmless on its own, but mash them together and you have a GDPR fine-to-the-max dataset of reidentification.

Somewhere in the middle of this an AIOTI participant suggested that the IoT rests on four pillars: people, processes, things, data. Trust has pillars, too, that take a long time to build but that can be destroyed in an instant: choice, control, transparency, and, the one we talk about least, but perhaps the most important, familiarity. The more something looks familiar, the more we trust it, even when we shouldn't. Both the devices AIOTI is fretting about and the police systems BBW deplores have this in common: they center on familiar things whose underpinnings have changed without our knowledge - yet their owners want us to trust them. We wish we could.

Illustrations:: Orwell's house at 22 Portobello Road, London.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 11:21 AM | Permalink | Comments (0) | TrackBacks (0)

"Why are robots different?" 2018 co-chair Mark Lemley asked repeatedly at this year's We Robot. We used to ask this in the late 1990s when trying to decide whether a new internet development was worth covering. "Would this be a story if it were about telephones?" Tom Standage and Ben Rooney frequently asked at the Daily Telegraph.

The obvious answer is physical risk and our perception of danger. The idea that autonomously moving objects may be dangerous is deeply biologically hard-wired. A plant can't kill you if you don't go near it. Or, as Bill Smart put it at the first We Robot in 2012, "My iPad can't stab me in my bed." Autonomous movement fools us into thinking things are smarter than they are.

It is probably not much consolation to the driver of the crashed autopiloting Tesla or his bereaved family that his predicament was predicted two years ago at We Robot 2016. In a paper, Madeline Elish called humans in these partnerships "Moral Crumple Zones", because, she argued, in a human-machine partnership, the human would take all the pressure, like the crumple zone in a car.

Today, Tesla is fulfilling her prophecy by blaming the driver for not getting his hands onto the steering wheel fast enough when commanded. (Other prior art on this: Dexter Palmer's brilliant 2016 book Version Control.)

As Ian Kerr pointed out, the user's instructions are self-contradictory. The marketing brochure uses the metaphors "autopilot" and "autosteer" to seduce buyers into envisioning a ride of relaxed luxury while the car does all the work. But the legal documents and user manual supplied with the car tell you that you can't rely on the car to change lanes, and you must keep your hands on the wheel at all times. A computer ingesting this would start smoking.

Granted, no marketer wants to say, "This car will drive itself in a limited fashion, as long as you watch the road and keep your hands on the steering wheel." The average consumer reading that says, "Um...you mean I have to drive it?"

The human as moral crumple zone also appears in analyses of the Arizona Uber crash. Even-handedly, Brad Templeton points plenty of blame at Uber and its decisions: the car's LIDAR should have spotted the pedestrian crossing the road in time to stop safely. He then writes, "Clearly there is a problem with the safety driver. She is not doing her job. She may face legal problems. She will certainly be fired." And yet humans are notoriously bad at the job required of her: monitor a machine. Safety drivers are typically deployed in pairs to split the work - but also to keep each other attentive.

The larger We Robot discussion was part about public perception of risk, based on a paper (PDF) by Aaron Mannes that discussed how easy it is to derail public trust in a company or new technology when statistically less-significant incidents spark emotional public outrage. Self-driving cars may in fact be safer overall than human drivers despite the fatal crash in Arizona; Mannes also mentioned were Three Mile Island, which made the public much more wary of nuclear power, and the Ford Pinto, which spent the 1970s occasionally catching fire.

Mannes suggested that if you have that trust relationship you may be able to survive your crisis. Without it, you're trying to win the public over on "Frankenfoods".

So much was funnier and more light-hearted seven years ago, as a long-time attendee pointed out; the discussions have darkened steadily year by year as theory has become practice and we can no longer think the problems are as far away as the Singularity.

In San Francisco, delivery robots cause sidewalk congestion and make some homeless people feel surveilled; in Chicago and Durham we risk embedding automated unfairness into criminal justice; the egregious extent of internet surveillance has become clear; and the world has seen its first self-driving car road deaths. The last several years have been full of fear about the loss of jobs; now the more imminent dragons are becoming clearer. Do you feel comfortable in public spaces when there's a like a mobile unit pointing some of its nine cameras at you?

Karen Levy, finds that truckers are less upset about losing their jobs than about automation invading their cabs, ostensibly for their safety. Sensors, cameras, and wearables that monitor them for wakefulness, heart health, and other parameters are painful and enraging to this group, who chose their job for its autonomy.

Today's drivers have the skills to step in; tomorrow's won't. Today's doctors are used to doing their own diagnostics; tomorrow's may not be. In the paper by Michael Froomkin, Ian Kerr, and Joëlle Pinea (PDF), automation may mean not only deskilling humans (doctors) but also a frozen knowledge base. Many hope that mining historical patient data will expose patterns that enable more accurate diagnostics and treatments. If the machines take over, where will the new approaches come from?

Worse, behind all that is sophisticated data manipulation for which today's internet is providing the prototype. When, as Woody Hartzog suggested, Rocco, your Alexa-equipped Roomba, rolls up to you, fakes a bum wheel, and says, "Daddy, buy me an upgrade or I'll die", will you have the heartlessness to say no?

Illustrations: Pepper and handler at We Robot 2016.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 4:45 PM | Permalink | Comments (0) | TrackBacks (0)

So here is this week's killer question: "Are you aware of any large-scale systems employing this protection?"

It's a killer question because this was the answer: "No."

Rewind. For as long as I can remember - and I first wrote about biometrics in 1999 - biometrics vendors have claimed that these systems are designed to be privacy-protecting. The reason, as I was told for a Guardian article on fingerprinting in schools in 2006, is that these systems don't store complete biometric images. Instead, when your biometric is captured, whether that's a fingerprint to pay for a school lunch or an iris scan for some other purpose - the system samples points in the resulting image and deploys some fancy mathematics to turn them into a "template", a numerical value that is what the system stores. The key claim: there is no way to reverse-engineer the template to derive the original image because the template doesn't contain enough information.

The claim sounds plausible to anyone used to one-way cryptographic hashes, or who is used to thinking about compressed photographs and music files, where no amount of effort can restore Humpty-Dumpty's missing data. And yet.

Even at the time, some of the activists I interviewed were dubious about the claim. Even if it was true in 1999, or 2003, or 2006, they argued, it might not be true in the future. Plus, in the meantime these systems were teaching kids that it was OK to use these irreplaceable iris scans, fingerprints, and so on for essentially trivial purposes. What would the consequences be someday in the future when biometrics might become a crucial element of secure identification?

Well, here we are in 2017, and biometrics are more widely used, even though not as widely deployed as they might have hoped in 1999. (There are good reasons for this, as James L. Wayman explained in a 2003 interview for New Scientist: deploying these systems is much harder than anyone ever thinks. The line that has always stuck in my mind: "No one ever has what you think they're going to have where you think they're going to have it." His example was the early fingerprint system he designed that was flummoxed on the first day by the completely unforeseen circumstance of a guy who had three thumbs.)

So-called "presentation attacks" - for example, using high-resolution photographs to devise a spoof dummy finger - have been widely discussed already. For this reason, such applications have a "liveness" test. But it turns out there are other attacks to be worried about.

This week, at the European Association for Biometrics held a symposium on privacy, surveillance, and biometrics, I discovered that Andrew Clymer, who said in 2003 that, "Anybody who says it is secure and can't be compromised is silly", was precisely right. As Marta Gomez-Barrero explained, in 2013 she published a successful attack on these templates she called "hill climbing". Essentially, this is an iterative attack. Say you have a database of stored templates for an identification system; a newly-presented image is compared with the database looking for a match. In a hill-climbing attack, you generate synthetic templates and run them through the comparator, and then apply a modification scheme to the synthetic templates until you get a match. The reconstructions Gomez-Barrero showed aren't always perfect - the human eye may see distortions - but to the biometrics system it's the same face. You can fix the human problem by adding some noise to the image. The same is true of iris scans (PDF), hand shapes, and so on.

Granted, someone wishing to conduct this attack has to have access to that database, but given the near-daily headlines about breaches, this is not a comforting thought.

Slightly better is the news that template protection techniques do exist; in fact, they've been known for ten to 15 years and are the subject of ISO standard 24745. Simply encrypting the data doesn't help as much as you might think, because every attempted match requires the template to be decrypted. Just like reused passwords, biometric templates are vulnerable to cross-matching that allows an attacker to extract more information. Second, if the data is available on the internet - this is especially applicable to face-based systems - an attacker can test for template matches.

It was at this point that someone asked the question we began with: are these protection schemes being used in large-scale systems? And...Gomez-Barrerra said: no. Assuming she's right, this is - again - one of those situations where no matter how carefully we behave we are the mercy of decisions outside our control that very few of us even know are out there waiting to cause trouble. It is market failure in its purest form, right up there with Equifax, which none of us chooses to use but still inflicted intimate exposure on hundreds of millions of people; and the 7547 bug, which showed you can do everything right in buying network equipment and still get hammered.

It makes you wonder: when will people learn that you can't avoid problems by denying there's any risk? Biometric systems are typically intended to handle the data of millions of people in sensitive applications such as financial transactions and smartphone authentication. Wouldn't you think security would be on the list of necessary features?

Illustrations: A 1930s FBI examiner at work (via FBI); James Wayman; Marta Gomez-Barrero.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. Stories about the border wars between cyberspace and real life are posted occasionally during the week at the net.wars Pinboard - or follow on Twitter.

Posted by Wendy M. Grossman at 11:58 AM | Permalink | Comments (0) | TrackBacks (0)

Who'd want to be a robot right now, branded a killer before you've even really been born? This week, Huw Price, a philosophy professor, Martin Rees, an emeritus professor of cosmology and astrophysics, and Jaan Tallinn, co-founder of Skype and a serial speaker at the Singularity Summit, announced the founding of the Cambridge Project for Existential Risk. I'm glad they're thinking about this stuff.

Their intention is to build a Centre for the Study of Existential Risk. There are many threats listed in the short introductory paragraph explaining the project - biotechnology, artificial life, nanotechnology, climate change - but the one everyone seems to be focusing on is: yep, you got it, KILLER ROBOTS - that is, artificial general intelligences so much smarter than we are that they may not only put us out of work but reshape the world for their own purposes, not caring what happens to us. Asimov would weep: his whole purpose in creating his Three Laws of Robotics was to provide a device that would allow him to tell some interesting speculative, what-if stories and get away from the then standard fictional assumption that robots were eeeevil.

The list of advisors to Cambridge project has some interesting names: Hermann Hauser, now in charge of a venture capital fund, whose long history in the computer industry includes founding Acorn and an attempt to create the first mobile-connected tablet (it was the size of a 1990s phone book, and you had to write each letter in an individual box to get it to recognize handwriting - just way too far ahead of its time); and Nick Bostrum of the Future of Humanity Institute at Oxford. The other names are less familiar to me, but it looks like a really good mix of talents, everything from genetics to the public understanding of risk.

The killer robots thing goes quite a way back. A friend of mine grew up in the time before television when kids would pay a nickel for the Saturday show at a movie theatre, which would, besides the feature, include a cartoon or two and the next chapter of a serial. We indulge his nostalgia by buying him DVDs of old serials such as The Phantom Creeps, which features an eight-foot, menacing robot that scares the heck out of people by doing little more than wave his arms at them.

Actually, the really eeeevil guy in that movie is the mad scientist, Dr Zorka, who not only creates the robot but also a machine that makes him invisible and another that induces mass suspended animation. The robot is really just drawn that way. But, like CSER, what grabs your attention is the robot.

I have a theory about this that I developed over the last couple of months working on a paper on complex systems, automation, and other computing trends, and this is that it's all to do with biology. We - and other animals - are pretty fundamentally wired to see anything that moves autonomously as more intelligent than anything that doesn't. In survival terms, that makes sense: the most poisonous plant can't attack you if you're standing out of reach of its branches. Something that can move autonomously can kill you - yet is also more cuddly. Consider the Roomba versus a modern dishwasher. Counterintuitively, the Roomba is not the smarter of the two.

And so it was that on Wednesday, when Voice of Russia assembled a bunch of us for a half-hour radio discussion, the focus was on KILLER ROBOTs, not synthetic biology (which I think is a much more immediately dangerous field) or climate change (in which the scariest new development is the very sober, grown-up, businesslike this-is-getting-expensive report from the insurer Munich Re). The conversation was genuinely interesting, roaming from the mysteries of consciousness to the problems of automated trading and the 2010 flash crash. Pretty much everyone agreed that there really isn't sufficient evidence to predict a date at which machines might be intelligent enough to pose an existential risk to humans. You might be worried about self-driving cars, but they're likely to be safer than drunk humans.

There is a real threat from killer machines; it's just that it's not super-human intelligence or consciousness that's the threat here. Last week, Human Rights Watch and the International Human Rights Clinic published Losing Humanity: the Case Against Killer Robots, arguing that governments should act pre-emptively to ban the development of fully autonomous weapons. There is no way, that paper argues, for autonomous weapons (which the military wants so fewer of *our* guys have to risk getting killed) to distinguish reliably between combatants and civilians.

There were some good papers on this at this year's We Robot conference from Ian Kerr and Kate Szilagyi (PDF) and Markus Wegner.

From various discussions, it's clear that you don't need to wait for *fully* autonomous weapons to reach the danger point. In today's partially automated systems, the operator may be under pressure to make a decision in seconds and "automation bias" means the human will most likely accept whatever the machines suggests it will do, the military equivalent of clicking OK. The human in the loop isn't as much of a protection as we might hope against the humans designing these things. Dr Zorka, indeed.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series

Posted by Wendy M. Grossman at 7:00 PM | Permalink | Comments (0) | TrackBacks (0)

"A really smart machine will think like an animal," predicted Temple Grandin at last weekend's Singularity Summit. To an animal, she argued, a human on a horse often looks like a very different category of object than a human walking. That seems true; and yet animals also live in a sensory-driven world entirely unlike that of machines.

A day later, Melanie Mitchell, a professor of computer science at Portland State University, argued that analogies are key, she said, to human intelligence, producing landmark insights like comparing a brain to a computer (von Neumann) or evolutionary competition to economic competition (Darwin). This is true, although that initial analogy is often insufficient and may even be entirely wrong. A really significant change in our understanding of the human brain came with research by psychologists like Elizabeth Loftus showing that where computers retain data exactly as it was (barring mechanical corruption), humans improve, embellish, forget, modify, and partially lose stored memories; our memories are malleable and unreliable in the extreme. (For a worked example, see The Good Wife, season 1, episode 6.)

Yet Mitchell is obviously right when she says that much of our humor is based on analogies. It's a staple of modern comedy, for example, for a character to respond on a subject *as if* it were another subject (chocolate as if it were sex, a pencil dropping on Earth as if it were sex, and so on). Especially incongruous analogies: when Watson asks - in the video clip she showed - for the category "Chicks dig me" it's funny because we know that as a machine a) Watson doesn't really understand what it's saying, and b) Watson is pretty much the polar opposite of the kind of thing that "chicks" are generally imagined to "dig".

"You are going to need my kind of mind on some of these Singularity projects," said Grandin, meaning visual thinkers, rather than the mathematical and verbal thinkers who "have taken over". She went on to contend that visual thinkers are better able to see details and relate them to each other. Her example: the emergency generators at Fukushima located below the level of a plaque 30 feet up on the seawall warning that flood water could rise that high. When she talks - passionately - about installing mechanical overrides in the artificial general intelligences Singularitarians hope will be built one day soonish, she seems to be channelling Peter G. Neumann, who talks often about the computer industry's penchant for repeating the security mistakes of decades past.

An interesting sideline about the date of the Singularity: Oxford's Stuart Armstrong has studied these date predictions and concluded pretty much that, in the famed words of William Goldman, no one knows anything. Based on his study of 257 predictions collected by the Singularity Institute and published on its Web site, he concluded that most theories about these predictions are wrong. The dates chosen typically do not correlate with the age or expertise of the predicter or the date of the prediction. I find this fascinating: there's something like an 80 percent consensus that the Singularity will happen in five to 100 years.

Grandin's discussion of visual thinkers made me wonder whether they would be better or worse at spotting the famed invisible gorilla than most people. Spoiler alert: if you're not familiar with this psychologist test, go now and watch the clip before proceeding. You want to say better - after all, spotting visual detail is what visual thinkers excel at - but what if the demands of counting passes is more all-consuming for them than for other types of thinkers? The psychologist Daniel Kahneman, participating by video link, talked about other kinds of bias but not this one. Would visual thinkers be more or less likely to engage in the common human pastime of believing we know something based on too little data and then ignoring new data?

This is, of course, the opposite of today's Bayesian systems, which make a guess and then refine it as more data arrives: almost the exact opposite of the humans Kahneman describes. So many of the developments we're seeing now rely on crunching masses of data (often characterized as "big" but often not *really* all that big) to find subtle patterns that humans never spot. Linda Avey, founder of the personal genome profiling service 23andMe and John Wilbanks are both trying to provide services that will allow individuals to take control of and understand their personal medical data. Avey in particular seems poised to link in somehow to the data generated by seekers in the several-year-old self-quantified movement.

This approach is so far yielding some impressive results. Peter Norvig, the director of research at Google, recounted both the company's work on recognizing cats and its work on building Google Translate. The latter's patchy quality seems more understandable when you learn that it was built by matching documents issued in multiple languages against each other and building up statistical probabilities. The former seems more like magic, although Slate points out that the computers did not necessarily pick out the same patterns humans would.

Well, why should they? Do I pick out the patterns they're interested in? The story continues...

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series.

Posted by Wendy M. Grossman at 7:25 PM | Permalink | Comments (0) | TrackBacks (0)

"This House fears the rise of artificial intelligence."

This was the motion up for debate at Trinity College Dublin's Philosophical Society (Twitter: @phil327) last night (December 1, 2011). It was a difficult one, because I don't think any of the speakers - neither the four students, Ricky McCormack, Michael Coleman, Cat O'Shea, and Brian O'Beirne, nor the invited guests, Eamonn Healy, Fred Cummins, and Abraham Campbell - honestly fear AI all that much. Either we don't really believe a future populated by superhumanly intelligent killer robots is all that likely, or, like Ken Jennings, we welcome our new computer overlords.

But the point of this type of debate is not to believe what you are saying - I learned later that in the upper levels of the game you are assigned a topic and a position and given only 15 minutes to marshal your thoughts - but to argue your assigned side so passionately, persuasively, and coherently that you win the votes of the assembled listeners even if later that night, while raiding the icebox, they think, "Well, hang on..." This is where politicians and Dail/House of Commons debating style come from, As a participatory sport it was utterly new to me, and it explains a *lot* about the derailment of political common sense by the rise of public relations and lobbying.

Obviously I don't actually oppose research into AI. I'm all for better tools, although I vituperatively loathe tools that try to game me. As much fun as it is to speculate about whether superhuman intelligences will deserve human rights, I tend to believe that AI will always be a tool. It was notable that almost every speaker assumed that AI would be embodied in a more-or-less humanoid robot. Far more likely, it seems to me, that if AI emerges it will be first in some giant, boxy system (that humans can unplug) and even if Moore's Law shrinks that box it will be much longer before AI and robotics converge into a humanoid form factor.

Lacking conviction on the likelihood of all this, and hence of its dangers, I had to find an angle, which eventually boiled down to Walt Kelly and We have met the enemy and he is us. In this, I discovered, I am not alone: a 2007 ThinkArtificial poll found that more than half of respondents feared what people would do with AI: the people who program it, own it, and deploy it.

If we look at the history of automation to date, a lot of it has been used to make (human) workers as interchangeable as possible. I am old enough to remember, for example, being able to walk down to the local phone company in my home town of Ithaca, NY, and talk in person to a customer service representative I had met multiple times before about my piddling residential account. Give everyone the same customer relationship database and workers become interchangeable parts. We gain some convenience - if Ms Jones is unavailable anyone else can help us - but we pay in lost relationships. The company loses customer loyalty, but gains (it hopes) consistent implementation of its rules and the economic leverage of no longer depending on any particular set of workers.

I might also have mentioned automated trading systems, which are making the markets swing much more wildly much more often. Later, Abraham Campbell, a computer scientist working in augmented reality at University College Dublin, said as much as 25 percent of trading is now done by bots. So, cool: Wall Street has become like one of those old IRC channels where you met a cute girl named Eliza...

Campbell had a second example: the Siri, which will tell you where to hide a dead body but not where you might get an abortion. Google's removal of torrent sites from its autosuggestion/Instant feature didn't seem to me egregious censorship, partly because there are other search engines and partly (short-sightedly) because I hate Instant so much already. But as we become increasingly dependent on mediators to help us navigate our overcrowded world, the agenda and/or competence of the people programming them are vital to know. These will be transparent only as long as there are alternatives.

Simultaneously, back in England in work that would have made Jessica Mitford proud, Privacy International's Eric King and Emma Draper were publishing material that rather better proves the point. Big Brother Inc lays out the dozens of technology companies from democratic Western countries that sell surveillance technologies to repressive regimes. King and Draper did what Mitford did for the funeral business in the late 1960s (and other muckrakers have done since): investigate what these companies' marketing departments tell prospective customers.

I doubt businesses will ever, without coercion, behave like humans with consciences; it's why they should not be legally construed as people. During last night's debate, the prospective robots were compared to women and "other races", who were also denied the vote. Yes, and they didn't get it without a lot of struggle. The In the "Robocalypse" (O'Beirne), they'd better be prepared to either a) fight to meltdown for their rights or b) protect their energy sources and wait patiently for the human race to exterminate itself.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series.

Posted by Wendy M. Grossman at 1:35 PM | Permalink | Comments (0) | TrackBacks (0)

Context is king.

Say to a human, "I'll meet you at the place near the thing where we went that time," and they'll show up at the right place. That's from the 1987 movieBroadcast News: Aaron (Albert Brooks) says it; cut to Jane (Holly Hunter), awaiting him at a table.

But what if Jane were a computer and what she wanted to know from Aaron's statement was not where to meet but how Aaron felt about it? This is the challenge facing sentiment analysis.

At Wednesday's Sentiment Analysis Symposium, the key question of context came up over and over again as the biggest challenge to the industry of people who claim that they can turn Tweets, blog postings, news stories, and other mass data sources into intelligence.

So context: Jane can parse "the place", "the thing", and "that time" because she has expert knowledge of her past with Aaron. It's an extreme example, but all human writing makes assumptions about the knowledge and understanding of the reader. Humans even use those assumptions to implement privacy in a public setting: Stephen Fry could retweet Aaron's words and still only Jane would find the cafe. If Jane is a large organization seeking to understand what people are saying about it and Aaron is 6 million people posting on Twitter, Tom can use sentiment analyzer tools to give a numerical answer. And numbers always inspire confidence...

My first encounter with sentiment analysis was this summer during Young Rewired State, when a team wanted to create a mood map of the UK comparing geolocated tweets to indices of multiple deprivation. This third annual symposium shows that here is a rapidly engorging industry, part PR, part image consultancy, and part artificial intelligence research project.

I was drawn to it out of curiosity, but also because it all sounds slightly sinister. What do sentiment analyzers understand when I say an airline lounge at Heathrow Terminal 4 "brings out my inner Sheldon? What is at stake is not precise meaning - humans argue over the exact meaning of even the greatest communicators - but extracting good-enough meaning from high-volume data streams written by millions of not-monkeys.

What could possibly go wrong? This was one of the day's most interesting questions, posed by the consultant Meta Brown to representatives of the Red Cross, the polling organization Harris Interactive, and Paypal. Failure to consider the data sources and the industry you're in, said the Red Cross's Banafsheh Ghassemi. Her example was the period just after Hurricane Irene, when analyzing social media sentiment would find it negative. "It took everyday disaster language as negative," she said. In addition, because the Red Cross's constituency is primarily older, social media are less indicative than emails and call center records. For many organizations, she added, social media tend to skew negative.

Earlier this year, Harris Interactive's Carol Haney, who has had to kill projects when they failed to produce sufficiently accurate results for the client, told a conference, "Sentiment analysis is the snake oil of 2011." Now, she said, "I believe it's still true to some extent. The customer has a commercial need for a dial pointing at a number - but that's not really what's being delivered. Over time you can see trends and significant change in sentiment, and when that happens I feel we're returning value to a customer because it's not something they received before and it's directionally accurate and giving information." But very small changes over short time scales are an unreliable basis for making decisions.

"The difficulty in social media analytics is you need a good idea of the questions you're asking to get good results," says Shlomo Argamon, whose research work seems to raise more questions than answers. Look at companies that claim to measure influence. "What is influence? How do you know you're measuring that or to what it correlates in the real world?" he asks. Even the notion that you can classify texts into positive and negative is a "huge simplifying assumption".

Argamon has been working on technology to discern from written text the gender and age - and perhaps other characteristics - of the author, a joint effort with his former PhD student Ken Bloom. When he says this, I immediately want to test him with obscure texts.

Is this stuff more or less creepy than online behavioral advertising? Han-Sheong Lai explained that Paypal uses sentiment analysis to try to glean the exact level of frustration of the company's biggest clients when they threaten to close their accounts. How serious are they? How much effort should the company put into dissuading them? Meanwhile Verint's job is to analyze those "This call may be recorded" calls. Verint's tools turn speech to text, and create color voiceprint maps showing the emotional high points. Click and hear the anger.

"Technology alone is not the solution," said Philip Resnik, summing up the state of the art. But, "It supports human insight in ways that were not previously possible." His talk made me ask: if humans obfuscate their data - for example, by turning off geolocation - will this industry respond by finding ways to put it all back again so the data will be more useful?

"It will be an arms race," he agrees. "Like spam."

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series.

Posted by Wendy M. Grossman at 9:55 PM | Permalink | Comments (0) | TrackBacks (0)

They say that every architect has, stuck in his desk drawer, a plan for the world's tallest skyscraper; probably every computer company similarly has a plan for the world's fastest supercomputer. At one time, that particular contest was always won by Seymour Cray. Currently, the world's fastest computer is Tianhe-1A, in China. But one day soon, it's going to be Blue Waters, an IBM-built machine filling 9,000 square feet at the National Center for Supercomputing Applications at the University of Illinois at Champaign-Urbana.

It's easy to forget - partly because Champaign-Urbana is not a place you visit by accident - how mainstream-famous NCSA and its host, UIUC, used to be. NCSA is the place from which Mosaic emerged in 1993. UIUC was where Arthur C. Clarke's HAL was turned on, on January 12, 1997. Clarke's choice was not accidental: my host, researcher Robert McGrath tells me that Clarke visited here and saw the seminal work going on in networking and artificial intelligence. And somewhere he saw the first singing computer, an IBM 7094 haltingly rendering "Daisy Bell." (Good news for IBM: at that time they wouldn't have had to pay copyright clearance fees on a song that was, in 1961, 69 years old.)

So much was invented here: Telnet, for example.

"But what have they done for us lately?" a friend in London wondered.

NCSA's involvement with supercomputing began when Larry Smarr, having worked in Europe and admired the access non-military scientists had to high-performance computers, wrote a letter to the National Science Foundation proposing that the NSF should fund a supercomputing center for use by civilian scientists. They agreed, and the first version of NCSA was built in 1986. Typically, a supercomputer is commissioned for five years; after that it's replaced with the fastest next thing. Blue Waters will have more than 300,000 8-core processors and be capable of a sustained rate of 1 petaflop and a peak rate of 10 petaflops. The transformer room underneath can provide 24 megawatts of power - as energy-efficiently as possible. Right now, the space where Blue Waters will go is a large empty white space broken up by black plug towers. It looks like a set from a 1950s science fiction film.

On the consumer end, we're at the point now where a five-year-old computer pretty much answers most normal needs. Unless you're a gamer or a home software developer, the pressure to upgrade is largely off. But this is nowhere near true at the high end of supercomputing.

"People are never satisfied for long," says Tricia Barker, who showed us around the facility. "Scientists and engineers are always thinking of new problems they want to solve, new details they want to see, and new variables they want to include." Planned applications for Blue Waters include studying storms to understand why some produce tornadoes and some don't. In the 1980s, she says, the data points were kilometers apart; Blue Waters will take the mesh down to 10 meters.

"It's why warnings systems are so hit and miss," she explains. Also on the list are more complete simulations to study climate change.

Every generation of supercomputers gets closer to simulating reality and increases the size of the systems we can simulate in a reasonable amount of time. How much further can it go?

They speculate, she said, about how, when, and whether exaflops can be reached: 2018? 2020? At all? Will the power requirements outstrip what can reasonably be supplied? How big would it have to be? And could anyone afford it?

In the end, of course, it's all about the data. The 500 petabytes of storage Blue Waters will have is only a small piece of the gigantic data sets that science is now producing. Across campus, also part of NCSA, senior research scientist Ray Plante is part of the Large Synoptic Survey Telescope project which, when it gets going, will capture a third of the sky every night on 3 gigapixel cameras with a wide field of view. The project will allow astronomers to see changes over a period of days, allowing them to look more closely at phenomena such as bursters and supernovae, and study dark energy.

Astronomers have led the way in understanding the importance of archiving and sharing data, partly because the telescopes are so expensive that scientists have no choice about sharing them. More than half the Hubble telescope papers, Plante says, are based on archival research, which means research conducted on the data after a short period in which research is restricted to those who proposed (and paid for) the project. In the case of LSST, he says, there will be no proprietary period: the data will be available to the whole community from Day One. There's a lesson here for data hogs if they care to listen.

Listening to Plante - and his nearby colleague Joe Futrelle - talk about the issues involved in storing, studying, and archiving these giant masses of data shows some of the issues that lie ahead for all of us. Many of today's astronomical studies rely on statistics, which in turn requires matching data sets that have been built into catalogues without necessarily considering who might in future need to use them: opening the data is only the first step.

So in answer to my friend: lots. I saw only about 0.1 percent of it.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series.

Posted by Wendy M. Grossman at 5:20 PM | Permalink | Comments (0) | TrackBacks (0)

Everyone loves a good headline, and £27 billion always makes a *great* one. In this case, that was the sum that a report written by the security consultancy firm Detica, now part of BAE Systems and issued by the Office of Cyber Security and Information Assurance (PDF) estimates that cybercrime is costing the UK economy annually. The claim was almost immediately questioned by ZDNet's Tom Espiner, who promptly checked it out with security experts. Who complained that the report was full of "fake precision" (LSE professor Peter Sommer), "questionable calculations" (Harvard's Tyler Moore), and "nonsense" (Cambridge's Richard Clayton).

First, some comparisons.

Twenty-seven billion pounds (approximately $40 billion) is slightly larger than a year's worth of the International Federation of the Phonographic Industry's estimate of the cumulative retail revenue lost to piracy by the European creative industries from 2008 to 2015 (PDF) (total €240 billion, about £203 million, eight years, £25.4 billion a year). It is roughly the estimated cost of the BP oil spill, the amount some think Facebook will be worth at an IPO, and noticeably less than Apple's $51 billion cash hoard. But: lots smaller than the "£40 billion underworld" The Times attributed to British gangs in 2008.

Several things baffle about this report. The first is that so little information is given about the study's methodology. Who did the researchers talk to? What assumptions did they make and what statistical probabilities did they assign in creating the numbers and charts? How are they defining categories like "online scams" or "IP theft" (they're clear about one thing: they're not including file-sharing in that figure)? What is the "causal model" they developed?

We know one person they didn't talk to: Computer Weekly notes the omission of Detective superintendent Charlie McMurdie, head of the Metropolitan Police's Central e-Crime Unit, who you'd' think would be one of the first ports of call for understanding the on-the-ground experience.

One issue the report seems to gloss over is how very difficult it is to define and categorize cybercrime. Last year, the Oxford Internet Institute conducted a one-day forum on the subject, out of which came the report Mapping and Measuring Cybercrime (PDF) , published in June 2010. Much of this report is given over to the difficulty of such definitions; Sommer, who participated in the forum, argued that we shouldn't worry about the means of commission - a crime is a crime. More recently - perhaps a month ago - Sommer teamed up with the OII's Ian Brown to publish a report for an OECD project on future global shocks, Reducing Systemic Cybersecurity Risk (PDF). The authors' conclusion: "very few single cyber-related events have the capacity to cause a global shock". This report also includes considerable discussion of cybercrime in assessing whether "cyberwarfare" is a genuine global threat. But the larger point about both these reports is that they disclose their methodology in detail.

And as a result, they make much more modest and measured claims, which is one reason that critics have looked at the source of the OCSIA/Detica report - BAE - and argued that the numbers are inflated and the focus largely limited to things that fit BAE's business interests (that is, IP theft and espionage; the usual demon, abuse of children, is left untouched).

The big risk here is that this report will be used in determining how policing resources are allocated.

"One of the most important things we can do is educate the public," says Sommer. "Not only about how to protect themselves but to ensure they don't leave their computers open to be formed into botnets. I am concerned that the effect of all these hugely military organizations lobbying for funding is that in the process things like Get Safe Online will suffer."

There's a broader point that begins with a personal nitpick. On page four, the report says this: "...the seeds of criminality planted by the first computer hackers 20 years ago." Leaving aside the even smaller nitpick that the *real*, original computer hackers, who built things and spent their enormous cleverness getting things to work, date to 40 and 50 years ago, it is utterly unfair to compare today's cybercrime to the (mostly) teenaged hackers of 1990, who spent their Saturday nights in their bedrooms war-dialling sites and trying out passwords. They were the computer equivalent of joy-riders, caused little harm, and were so disproportionately the targets of freaked-out, uncomprehending law enforcement that the the Electronic Frontier Foundation was founded to spread some sanity on the situation. Today's cybercrime underground is composed of professional criminals who operate in an organized and methodical way. There is no more valid comparison between the two than there is between Duke Nukem and al-Qaeda.

One is not a gateway to the other - but the idea that criminals would learn computer techniques and organized crime would become active online was repeatedly used as justification for anti-society legislation from cryptographic key escrow to data retention and other surveillance. The biggest risk of a report like this is that it will be used as justification for those wrong-headed policies rather than as it might more rightfully be, as evidence of the failure of no less than five British governments to plan ahead on our behalf.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series.

Posted by Wendy M. Grossman at 6:19 PM | Permalink | Comments (0) | TrackBacks (0)

This seems to have been a week for over-excitement. IBM gets an onslaught of wonderful publicity because it built a very large computer that won at the archetypal American TV game, Jeopardy. And Eben Moglen proposes the Freedom box, a more-or-less pocket ("wall wart") computer you can plug in and that will come up, configure itself, and be your Web server/blog host/social network/whatever and will put you and your data beyond the reach of, well, everyone. "You get no spying for free!" he said in his talk outlining the idea for the New York Internet Society.

Now I don't mean to suggest that these are not both exciting ideas and that making them work is/would be an impressive and fine achievement. But seriously? Is "Jeopardy champion" what you thought artificial intelligence would look like? Is a small "wall wart" box what you thought freedom would look like?

To begin with Watson and its artificial buzzer thumb. The reactions display everything that makes us human. The New York Times seems to think AI is solved, although its editors focus, on our ability to anthropomorphize an electronic screen with a smooth, synthesized voice and a swirling logo. (Like HAL, R2D2, and Eliza Doolittle, its status is defined by the reactions of the surrounding humans.)

The Atlantic and Forbes come across as defensive. The LA Times asks: how scared should we be? The San Francisco Chronicle congratulates IBM for suddenly becoming a cool place for the kids to work.

If, that is, they're not busy hacking up Freedom boxes. You could, if you wanted, see the past twenty years of net.wars as a recurring struggle between centralization and distribution. The Long Tail finds value in selling obscure products to meet the eccentric needs of previously ignored niche markets; eBay's value is in aggregating all those buyers and sellers so they can find each other. The Web's usefulness depends on the diversity of its sources and content; search engines aggregate it and us so we can be matched to the stuff we actually want. Web boards distributed us according to niche topics; social networks aggregated us. And so on. As Moglen correctly says, we pay for those aggregators - and for the convenience of closed, mobile gadgets - by allowing them to spy on us.

An early, largely forgotten net.skirmish came around 1991 over the asymmetric broadband design that today is everywhere: a paved highway going to people's homes and a dirt track coming back out. The objection that this design assumed that consumers would not also be creators and producers was largely overcome by the advent of Web hosting farms. But imagine instead that symmetric connections were the norm and everyone hosted their sites and email on their own machines with complete control over who saw what.

This is Moglen's proposal: to recreate the Internet as a decentralized peer-to-peer system. And I thought immediately how much it sounded like...Usenet.

For those who missed the 1990s: invented and implemented in 1979 by three students, Tom Truscott, Jim Ellis, and Steve Bellovin, the whole point of Usenet was that it was a low-cost, decentralized way of distributing news. Once the Internet was established, it became the medium of transmission, but in the beginning computers phoned each other and transferred news files. In the early 1990s, it was the biggest game in town: it was where the Linus Torvalds and Tim Berners-Lee announced their inventions of Linux and the World Wide Web.

It always seemed to me that if "they" - whoever they were going to be - seized control of the Internet we could always start over by rebuilding Usenet as a town square. And this is to some extent what Moglen is proposing: to rebuild the Net as a decentralized network of equal peers. Not really Usenet; instead a decentralized Web like the one we gave up when we all (or almost all) put our Web sites on hosting farms whose owners could be DMCA'd into taking our sites down or subpoena'd into turning over their logs. Freedom boxes are Moglen's response to "free spying with everything".

I don't think there's much doubt that the box he has in mind can be built. The Pogoplug, which offers a personal cloud and a sort of hardware social network, is most of the way there already. And Moglen's argument has merit: that if you control your Web server and the nexus of your social network law enforcement can't just make a secret phone call, they'll need a search warrant to search your home if they want to inspect your data. (On the other hand, seizing your data is as simple as impounding or smashing your wall wart.)

I can see Freedom boxes being a good solution for some situations, but like many things before it they won't scale well to the mass market because they will (like Usenet) attract abuse. In cleaning out old papers this week, I found a 1994 copy of Esther Dyson's Release 1.0 in which she demands a return to the "paradise" of the "accountable Net"; 'twill be ever thus. The problem Watson is up against is similar: it will function well, even engagingly, within the domain it was designed for. Getting it to scale will be a whole 'nother, much more complex problem.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series.

Posted by Wendy M. Grossman at 2:02 PM | Permalink | Comments (0) | TrackBacks (0)

"Anyone who realizes how important the Web is," Tim Berners-Lee said on Tuesday, "has a duty of care." He was wrapping up a two-day discussion meeting at the Royal Society. The subject: Web science.

What is Web science? Even after two days, it's difficult to grasp, in part because defining it is a work in progress. Here are some of the disciplines that contributed: mathematics, philosophy, sociology, network science, and law, plus a bunch of much more directly Webby things that don't fit easily into categories. Which of course is the point: Web science has to cover much more than just the physical underpinnings of computers and network wires. Computer science or network science can use the principles of mathematics and physics to develop better and faster machines and study architectures and connections. But the Web doesn't exist without the people putting content and applications on it, and so Web science must be as much about human behaviour as about physics.

"If we are to anticipate how the Web will develop, we will require insight into our own nature," Nigel Shadbolt, one of the event's convenors, said on Monday. Co-convenor Wendy Hall has said, similarly, "What creates the Web is us who put things on it, and that's not natural or engineered.". Neither natural (biological systems) or engineered (planned build-out like the telecommunications networks), but something new. If we can understand it better, we can not only protect it better, but guide it better toward the most productive outcomes, just as farmers don't haphazardly interbreed species of corn but use their understanding to select for desirable traits.

The simplest parts of the discussions to understand, therefore, were (ironically) the mathematicians. Particularly intriguing was the former chief scientist Robert May, whose approach to removing nodes from the network to make it non-functional applied equally to the Web, epidemiology, and banking risk.

This is all happening despite the recent Wired cover claiming the "Web is dead". Dead? Facebook is a Web site; Skype, the app store, IM clients, Twitter, and the New York Times all reach users first via the Web even if they use their iPhones for subsequent visits (and how exactly did they buy those iPhones, hey?) Saying it's dead is almost exactly the old joke about how no one goes to a particular restaurant any more because it's too crowded.

People who think the Web is dead have stopped seeing it. But the point of Web science is that for 20 years we've been turning what started as an academic playground into a critical infrastructure, and for government, finance, education, and social interaction to all depend on the Web it must have solid underpinnings. And it has to keep scaling - in a presentation on the state of deployment of IPv6 in China, Jianping Wu noted that Internet penetration in China is expected to jump from 30 percent to 70 percent in the next ten to 20 years. That means adding 400-900 million users. The Chinese will have to design, manage, and operate the largest infrastructure in the world - and finance it.

But that's the straightforward kind of scaling. IBMer Philip Tetlow, author of The Web's Awake (a kind of Web version of the Gaia hypothesis), pointed out that all the links in the world are a finite set; all the eyeballs in the world looking at them are a finite set...but all the contexts surrounding them...well, it's probably finite but it's not calculable (despite Pierre Levy's rather fanciful construct that seemed to suggest it might be possible to assign a URI to every human thought). At that level, Tetlow believes some of the neat mathematical tools, like Jennifer Chayes' graph theory, will break down.

"We're the equivalent of precision engineers," he said, when what's needed are the equivalent of town planners and urban developers. "And we can't build these things out of watches."

We may not be able to build them at all, at least not immediately. Helen Margetts outlined the constraints on the development of egovernment in times of austerity. "Web science needs to map, understand, and develop government just as for other social phenomena, and export back to mainstream," she said.

Other speakers highlighted gaps between popular mythology and reality. MIT's David Carter noted that, "The Web is often associated with the national and international but not the local - but the Web is really good at fostering local initiatives - that's something for Web science to ponder." Noshir Contractor, similarly, called out The Economist over the "death of distance": "More and more research shows we use the Web to have connections with proximate people."

Other topics will be far more familiar to net.wars readers: Jonathan Zittrain explored the ways the Web can be broken by copyright law, increasing corporate control (there was a lovely moment when he morphed the iPhone's screen into the old CompuServe main menu), the loss of uniformity so that the content a URL points to changes by geographic location. These and others are emerging points of failure.

We'll leave it to an unidentified audience question to sum up the state of Web science: "Nobody knows what it is. But we are doing it."

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series

Posted by Wendy M. Grossman at 1:57 PM | Permalink | Comments (0) | TrackBacks (0)

"Should I burn the check or eat it?" a broke friend with with bank account difficulties asked once.

Deciding what you can do without in a financial crisis is always tough, whether you're an individual or a government. Do you cut cold weather payments to the elderly? Dump pre-school programs? Sell off nationalized industries, pocket the debt, and use the cash as if it were income instead of irreplaceable capital? Slash arts funding knowing that you will be attacked by every high-profile actor and creator as a philistine? Flood prevention. Investment in new technologies to combat climate change. Police. Every group has its own set of arguments about why it shouldn't bear the brunt of government cuts. Everyone is special.

That may in fact be why the coalition government warned at the outset that slashing budgets would be across the board and that everyone would feel the chill. The UK Film Council, Becta, public sector...

And science research, spending on which is due to be reviewed next month. Even Harris, the former LibDem MP for Oxford West and Abingdon, has argued that science research is the foundation of future economic growth; Professor Brian Cox has compared the possibility of mothballing the expensive particle accelerator projects Diamond and Isis to "building the Olympic stadium and then not using it". (Not building the Olympic stadium - not winning the Olympics - not *bidding* on the Olympics would all have been fine with me, but this is the problem with trying to balance interest groups.)

At first glance, it's easy to see why business secretary Vince Cable would think it's a good idea for scientists to become more commercial: get industry to provide more funding and discontinue work that is "neither commercially useful nor theoretically outstanding", as the Guardian has him saying. While we've all heard the jokes about Drunken Goldfish and Other Irrelevant Scientific Research, the thing is that science - especially basic research - isn't so neatly categorized. When it is - when commercial interests take over too strongly - the underlying fundamental advances are lost, taking with them the next generation of new ideas.

Twenty years ago, when I first started doing technology journalism, I was told there were three great corporate research labs in the US: Xerox PARC, IBM Watson, and Bell Labs. Bell Labs was broken up along with its parent company, AT&T; PARC is not the force it was. Only IBM is still making news with its research. A lot of talent is now congregating at Google. In any event, over the last two decades most corporate research has in general become much more tightly focused on producing results the funding companies can use right away. That was a major reason why MIT's Media Lab was so successful at attracting funding from so many companies: it offered them a way to back less specifically focused research for relatively modest sums.

But basic research is the real blue-sky stuff, where you don't know what you have until some time later. In its heyday, IBM did both: it invented dye lasers, which had relatively little impact within the company but much more outside it, as well as DRAM and disk drives, which more obviously benefited the company itself. James McGroddy, then director of IBM research, told me in 1991 (for Personal Computer World) that even apparently irrelevant scientific research did have benefits for IBM even if they couldn't be easily quantified. For example, the company can more easily take advantage of advances if the people who made them are in its employ. Plus, expertise can cross disciplines: he cited the example of IBM mathematicians who find hard problems to work on within IBM customer needs (such as how to optimize airline schedules). More subtly, the production of Nobel prize-winning work made IBM the kind of place that the best people wanted to be.

All these points are relevant to national research programs, too, and lead directly to points Harris and others have made: that if you remove the facilities that allow scientists to work they will perforce go elsewhere. It is unfortunate but true that highly educated, very talented, creative people - and that is what scientists are - have choices about these things. And once you start to lose this generation of scientists, the next generation will follow of necessity because the way you become a great scientist is to be trained by and work with great scientists during your developmental years. The decisions made in this area today will make the difference between the UK's continuing to be a country that punches well above its weight in terms of size, population, and natural resources and the UK's becoming the third world country the Pope's aide already thinks it is (although hasn't anyone who's had to take one of those buses from plane to jetway thought the same thing?).

There must be some way of balancing the finances such that we do not throw away the future to pay for the present. Julian Huppert has tabled an Early Day Motion in Parliament, and there are demonstrations brewing. Imagine: Sheldon is marching.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series.

Posted by Wendy M. Grossman at 3:41 PM | Permalink | Comments (0) | TrackBacks (0)

The case of murdered Colombian student Anna Maria Chávez Niño, presented at this week's Privacy Open Space, encompasses both extremes of the privacy conundrum posed by a world in which 400 million people post intimate details about themselves and their friends onto a single, corporately owned platform. The gist: Chávez met her murderers on Facebook; her brother tracked them down, also on Facebook.

Speaking via video link to Cédric Laurant, a Brussels-based independent privacy consultant, Juan Camilo Chávez noted that his sister might well have made the same mistake - inviting dangerous strangers into her home - by other means. But without Facebook he might not have been able to identify the killers. Criminals, it turns out, are just as clueless about what they post online as anyone else. Armed with the CCTV images, Chávez trawled Facebook for similar photos. He found the murderers selling off his sister's jacket and guitar. As they say, busted.

This week's PrivacyOS was the fourth in a series of EU-sponsored conferences to collaborate on solutions to that persistent, growing, and increasingly complex problem: how to protect privacy in a digital world. This week's focused on the cloud.

"I don't agree that privacy is disappearing as a social value," said Ian Brown, one of the event's organizers, disputing Mark privacy-is-no-longer-a-social-norm Zuckerberg's claim. The world's social values don't disappear, he added, just because some California teenagers don't care about them.

Do we protect users through regulation? Require subject releases for YouTube or Qik? Require all browsers to ship with cookies turned off? As Lilian Edwards observed, the latter would simply make many users think the Internet is broken. My notion: require social networks to add a field to photo uploads requiring users to enter an expiration date after which it will be deleted.

But, "This is meant to be a free world," Humberto Morán, managing director of Friendly Technologies, protested. Free as in speech, free as in beer, or free as in the bargain we make with our data so we can use Facebook or Google? We have no control over those privacy policy contracts.

"Nothing is for free," observed NEC's Amardeo Sarma. "You pay for it, but you don't know how you pay for it." The key issue.

What frequent flyers know is that they can get free flights once in a while in return for their data. What even the brightest, most diligent, and most paranoid expert cannot tell them is what the consequences of that trade will be 20 years from now, though the Privacy Value Networks project is attempting to quantify this. It's hard: any photographer will tell you that a picture's value is usually highest when it's new, but sometimes suddenly skyrockets decades later when its subject shoots unexpectedly to prominence. Similarly, the value of data, said David Houghton, changes with time and context.

It would be more right to say that it is difficult for users to understand the trade-offs they're making and there are no incentives for government or commerce to make it easy. And, as the recent "You have 0 Friends" episode of South Park neatly captures, the choice for users is often not between being careful and being careless but between being a hermit and participating in modern life.

Better tools ought to be a partial solution. And yet: the market for privacy-enhancing technologies is littered with market failures. Even the W3C's own Platform for Privacy Preferences (P3P), for example, is not deployed in the current generation of browsers - and when it was provided in Internet Explorer users didn't take advantage of it. The projects outlined at PrivacOS - PICOS and PrimeLife - are frustratingly slow to move from concept to prototype. The ideas seem right: providing a way to limit disclosures and authenticate identity to minimize data trails. But, Lilian Edwards asked: is partial consent or partial disclosure really possible? It's not clear that it is, partly because your friends are also now posting information about you. The idea of a decentralized social network, workshopped at one session, is interesting, but might be as likely to expand the problem as modulate it.

And, as it has throughout the 25 years since the first online communities were founded, the problem keeps growing exponentially in size and complexity. The next frontier, said Thomas Roessler: the sensor Web that incorporates location data and input from all sorts of devices throughout our lives. What does it mean to design a privacy-friendly bathroom scale that tweets your current and goal weights? What happens when the data it sends gets mashed up with the site you use to monitor the calories you consume and burn and your online health account? Did you really understand when you gave your initial consent to the site what kind of data it would hold and what the secondary uses might be?

So privacy is hard: to define, to value, to implement. As Seda Gürses, studying how to incorporate privacy into social networks, said, privacy is a process, not an event. "You can't do x and say, Now I have protected privacy."

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. This blog eats non-spam comments for reasons surpassing understanding.

Posted by Wendy M. Grossman at 3:29 PM | Permalink | Comments (0) | TrackBacks (0)

"And what's your mother's maiden name?"

A lot of attention has been paid over the years to the quality of passwords: how many letters, whether there's a sufficient mix of numbers and "special characters", whether they're obviously and easily guessable by anyone who knows you (pet's name, spouse's name, birthday, etc.), whether you've reset them sufficiently recently. But, as someone noted this week on UKCrypto, hardly anyone pays attention to the quality of the answers to the "password hint" questions sites ask so they can identify you when you eventually forget your password. By analogy, it's as though we spent all our time beefing up the weight, impenetrability, and lock quality on our front doors while leaving the back of the house accessible via two or three poorly fitted screen doors.

On most sites it probably doesn't matter much. But the question came up after the BBC broadcast an interview with the journalist Angela Epstein, the loopily eager first registrant for the ID card, in which she apparently mentioned having been asked to provide the answers to five rather ordinary security questions "like what is your favorite food". Epstein's column gives more detail: "name of first pet, favourite song and best subject at school". Even Epstein calls this list "slightly bonkers". This, the UKCrypto poster asked, is going to protect us from terrorists?

Dave Birch had some logic to contribute: "Why are we spending billions on a biometric database and taking fingerprints if they're going to use the questions instead? It doesn't make any sense." It doesn't: she gave a photograph and two fingerprints.

But let's pretend it does. The UKCrypto discussion headed into technicalities: has anyone studied challenge questions?

It turns out someone has: Mike Just, described to me as "the world expert on challenge questions". Just, who's delivered two papers on the subject this year, at the Trust (PDF) and SOUPS (PDF) conferences, has studied both the usability and the security of challenge questions. There are problems from both sides.

First of all, people are more complicated and less standardized than those setting these questions seem to think. Some never had pets; some have never owned cars; some can't remember whether they wrote "NYC", "New York", "New York City", or "Manhattan". And people and their tastes change. This year's favorite food might be sushi; last year's chocolate chip cookies. Are you sure you remember accurately what you answered? With all the right capitalization and everything? Government services are supposedly thinking long-term. You can always start another Amazon.com account; but ten years from now, when you've lost your ID card, will these answers be valid?

This sort of thing is reminiscent of what biometrics expert James Wayman has often said about designing biometric systems to cope with the infinite variety of human life: "People never have what you expect them to have where you expect them to have it." (Note that Epstein nearly failed the ID card registration because of a burn on her finger.)

Plus, people forget. Even stuff you'd think they'd remember and even people who, like the students he tested, are young.

From the security standpoint, there are even more concerns. Many details about even the most obscure person's life are now public knowledge. What if you went to the same school for 14 years? And what if that fact is thoroughly documented online because you joined its Facebook group?

A lot depends on your threat model: your parents, hackers with scripted dictionary attacks, friends and family, marketers, snooping government officials? Just accordingly came up with three types of security attacks for the answers to such questions: blind guess, focused guess, and observation guess. Apply these to the often-used "mother's maiden name": the surname might be two letters long; it is likely one of the only 150,000 unique surnames appearing more than 100 times in the US census; it may be eminently guessable by anyone who knows you - or about you. In the Facebook era, even without a Wikipedia entry or a history of Usenet postings many people's personal details are scattered all over the online landscape. And, as Just also points out, the answers to challenge questions are themselves a source of new data for the questioning companies to mine.

My experience from The Skeptic suggests that over the long term trying to protect your personal details by not disclosing them isn't going to work very well. People do not remember what they tell psychics over the course of 15 minutes or an hour. They have even less idea what they've told their friends or, via the Internet, millions of strangers over a period of decades or how their disparate nuggets of information might match together. It requires effort to lie - even by omission - and even more to sustain a lie over time. It's logically easier to construct a relatively small number of lies. Therefore, it seems to me that it's a simpler job to construct lies for the few occasions when you need the security and protect that small group of lies. The trouble then is documentation.

Even so, says Birch, "In any circumstance, those questions are not really security. You should probably be prosecuted for calling them 'security'."

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, follow on Twitter, or send email to netwars@skeptic.demon.co.uk.

Posted by Wendy M. Grossman at 6:24 PM | Permalink | Comments (0) | TrackBacks (0)

What does a venture capitalist do in a recession?

"Panic." Hermann Hauser says, then laughs. It is, in fact, hard to imagine him panicking if you've heard the stories he tells about his days as co-founder of Acorn Computers. He's quickly on to his real, more measured, view.

"It's just the bottom of the cycle, and people my age have been through this a number of times before. Though many people are panicking, I know that normally we come out the other end. If you just look at the deals I'm seeing at the moment, they're better than any deals I've seen in my entire life." The really positive thing, he says, is that, "The speed and quality of innovation are speeding up and not slowing down. If you believe that quality of innovation is the key to a successful business, as I do, then this is a good era. We have got to go after the high end of innovation - advanced manufacturing and the knowledge-based economy. I think we are quite well placed to do that." Fortunately, Amadeus had just raised a fund when the recession began, so it still has money to invest; life is, he admits, less fun for "the poor buggers who have to raise funds."

Among the companies he is excited about is Plastic Logic, which is due to release its first product next year, a competitor to the Kindle that will have a much larger screen, be much lighter, and will also be a computing platform with 3g, Bluetooth, and Wi-fi all built in, all built on plastic transistors that will be green to produce, more responsive than silicon - and sealed against being dropped in the bath water. "We have the world beat," he says. "It's just the most fantastic thing."

Probably if you ask any British geek above the age of 39, an Acorn BBC Micro figured prominently in their earliest experiences with computing. Hauser was and is not primarily a technical guy - although his idea of exhilarating vacation reading is Thermal Physics, by Charles Kittel and Herbert Kroemer - but picking the right guys to keep supplied with tea and financing is a rare skill, too.

"As I go around the country, people still congratulate me on the BBC Micro and tell me how wonderful it was. Some are now professors in computer science and what they complain about is that as people switched over to PCs - on the BBC Micro everybody knew how to program. The main interface was a programming interface, and it was so easy to program in BASIC everybody did it. Kids have no clue what programming is about - they just surf the Net. Nobody really understands any more what a computer does from the transistor up. It's a dying breed of people who actually know that all this is built on CMOS gates and can build it up from there."

Hauser went on to found an early effort in pen computing - "the technology wasn't good enough" and "the basic premise that I believed in, that pen computing would be important because everybody knew how to wield a pen just wasn't true" - and then the venture capital fund Amadeus, through which he helped fund, among others, leading Bluetooth chip supplier CSR. Britain, he says, is a much more hospitable environment now than it was when he was trying to make his Cambridge bank manager understand Acorn's need for a £1 million overdraft. Although, he admits now, "I certainly wouldn't have invested in myself." And would have missed Acorn's success.

"I think I'm the only European who's done four billion-dollar companies," he says. "Of course I've failed a lot. I assume that more of my initiatives that I've founded finally failed than finally succeeded."

But times have changed since consultants studied Acorn's books and told them to stop trading immediately because they didn't understand how technology companies worked. "All the building blocks you need to have to have a successful technology cluster are now finally in place," he says. "We always that the technology, but we always lacked management, and we've grown our own entrepreneurs now in Britain." He calls Stan Boland, CEO of 3g USB stock manufacturer Icera and Acorn's last managing director a "rock star" and "one of the best CEOs I have come across in Europe or the US." In addition, he says, "There is also a chance of attracting the top US talent, for the first time." However, "The only thing I fear and that we have to be careful about is that the relative decline doesn't turn into an absolute decline."

One element of Britain's changing climate with respect to technology investment that Hauser is particularly proud of is helping create tax credits and taper relief for capital gains through his work on Leon Mandelson's advisory panel on new industry and new jobs. "The reason I have done it is that I don't believe in the post-industrial society. We have to have all parts of industry in our country."

Hauser's latest excitement is stem cells; he's become the fourth person in the world to have his entire genome mapped. "It's the beginning of personal medicine."

The one thing that really bemuses him is being given lifetime achievement awards. "I have lived in the future all my life, and I still do. It's difficult to accept that I've already created a past. I haven't done yet the things I want to do!"

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, follow on Twitter, or send email to netwars@skeptic.demon.co.uk.

Posted by Wendy M. Grossman at 4:15 PM | Permalink | Comments (0) | TrackBacks (0)

So the question of last weekend very quickly became: how do you tell plausible fantasy from wild possibility? It's a good conversation starter.

One friend had a simple assessment: "They are all nuts," he said, after glancing over the weekend's program. The problem is that 150 years ago anyone predicting today's airline economy class would also have sounded nuts.

Last weekend's (un)conference was called Convergence, but the description tried to convey the sense of danger of crossing the streams. The four elements that were supposed to converge: computing, biotech, cognitive technology, and nanotechnology. Or, as the four-colored conference buttons and T-shirts had it, biotech, infotech, cognotech, and nanotech.

Unconferences seem to be the current trend. I'm guessing, based on very little knowledge, that it was started by Tim O'Reilly's FOO camps or possibly the long-running invitation-only Hackers conference. The basic principle is: collect a bunch of smart, interesting, knowledgeable people and they'll construct their own program. After all, isn't the best part of all conferences the hallway chats and networking, rather than the talks? Having been to one now (yes, a very small sample), I think in most cases I'm going to prefer the organized variety: there's a lot to be said for a program committee that reviews the proposals.

The day before, the Center for Responsible Nanotechnology ran a much smaller seminar on Global Catastrophic Risks. It made a nice counterweight: the weekend was all about wild visions of the future; the seminar was all about the likelihood of our being wiped out by biological agents, astronomical catastrophe, or, most likely, our own stupidity. Favorite quote of the day, from Anders Sandberg: "Very smart people make very stupid mistakes, and they do it with surprising regularity." Sandberg learned this, he said, at Oxford, where he is a philosopher in the Institute for the Future of Humanity.

Ralph Merkle, co-inventor of public key cryptography, now working on diamond mechanosynthesis, said to start with physics textbooks, most notably the evergreen classic by Halliday and Resnick. You can see his point: if whatever-it-is violates the laws of physics it's not going to happen. That at least separates the kinds of ideas flying around at Convergence and the Singularity Summit from most paranormal claims: people promoting dowsing, astrology, ghosts, or ESP seem to be about as interested in the laws of physics as creationists are in the fossil record.

A sidelight: after years of The Skeptic, I'm tempted to dismiss as fantasy anything where the proponents tell you that it's just your fear that's preventing you from believing their claims. I've had this a lot - ghosts, alien spacecraft, alien abductions, apparently these things are happening all over the place and I'm just too phobic to admit it. Unfortunately, the behavior of adherents to a belief just isn't evidence that it's wrong.

Similarly, an idea isn't wrong just because its requirements are annoying. Do I want to believe that my continued good health depends on emulating Ray Kurzweil and taking 250 pills a day and, a load of injections weekly? Certainly not. But I can't prove it's not helping him. I can, however, joke that it's like those caloric restriction diets - doing it makes your life *seem* longer.

Merkle's other criterion: "Is it internally consistent?" This one's harder to assess, particularly if you aren't a scientific expert yourself.

But there is the technique of playing the man instead of the ball. Merkle, for example, is a cryonicist and is currently working on diamond mechanosynthesis. Put more simply, he's busy designing the tools that will be needed to build things atom by atom when - if - molecular manufacturing becomes a reality. If that sounds nutty, well, Merkle has earned the right to steam ahead unworried because his ideas about cryptography, which have become part of the technology we use every day to protect ecommerce transactions, were widely dismissed at first.

Analyzing language is also open to the scientifically less well-educated: do the proponents of the theory use a lot of non-standard terms that sound impressive but on inspection don't seem to mean anything? It helps if they can spell, but that's not a reliable indicator - snake oil salesmen can be very professional, and some well-educated excellent scientists can't spell worth a damn.

The Risks seminar threw out a useful criterion for assessing scenarios: would it make a good movie? If your threat to civilization can be easily imagined as a line delivered by Bruce Willis, it's probably unlikely. It's not a scientifically defensible principle, of course, but it has a lot to recommend it. In human history, what's killed the most people while we're worrying about dramatic events like climate change and colliding asteroids? Wars and pandemics.

So, where does that leave us? Waiting for deliverables, of course. Even if a goal sounds ludicrous working towards it may still produce useful results. A project like Aubrey de Grey's ideas about "curing aging" by developing techniques for directly repairing damage (or SENS, for Strategies for Engineered Negligible Senescence) seems a case in point. And life extension is the best hope for all of these crazy ideas. Because, let's face it: if it doesn't happen in our lifetime, it was impossible.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, at her personal blog, or by email to netwars@skeptic.demon.co.uk (but please turn off HTML).

Posted by Wendy M. Grossman at 4:42 PM | Permalink | Comments (0) | TrackBacks (0)

The Xerox machine in the second season of Mad Men has its own Twitter account, as do many of the show's human characters. Other TV characters have MySpace pages and Facebook groups, and of course they're all, legally or illegally, on YouTube.

Here at the American Film Institute's Digifest in Hollywood - really Hollywood, with the stars on the sidewalks and movie theatres everywhere - the talk is all of "cross-platform". This event allows the AFI's Digital Content Lab to show off some of the projects it's fostered over the last year, and the audience is full of filmmakers, writers, executives, and owners of technology companies, all trying to figure out digital television.

One of the more timely projects is a remix of the venerable PBS Newshour with Jim Lehrer. A sort of combination of Snopes, Wikipedia, and any of a number of online comment sites, the goal of The Fact Project is to enable collaboration between the show's journalists and the public. Anyone can post a claim or a bit of rhetoric and bring in supporting or refuting evidence; the show's journalistic staff weigh in at the end with a Truthometer rating and the discussion is closed. Part of the point, said the project's head, Lee Banville, is to expose to the public the many small but nasty claims that are made in obscure but strategic places - flyers left on cars in supermarket parking lots, or radio spots that air maybe twice on a tiny local station.

The DCL's counterpart in Australia showed off some other examples. Areo, for example, takes TV sets and footage and turns them into game settings. More interesting is the First Australians project, which in the six-year process of filming a TV documentary series created more than 200 edited mini-documentaries telling each interviewee's story. Or the TV movie Scorched, which even before release created a prequel and sequel by giving a fictional character her own Web site and YouTube channel. The premise of the film itself was simple but arresting. It was based on one fact, that at one point Sydney had no more than 50 weeks of water left, and one what-if - what if there were bush fires? The project eventually included a number of other sites, including a fake government department.

"We go to islands that are already populated," said the director, "and pull them into our world."

HBO's Digital Lab group, on the other hand, has a simpler goal: to find an audience in the digital world it can experiment on. Last month, it launched a Web-only series called Hooking Up. Made for almost no money (and it looks it), the show is a comedy series about the relationship attempts of college kids. To help draw larger audiences, the show cast existing Web and YouTube celebrities such as LonelyGirl15, KevJumba, and sxePhil. The show has pulled in 46,000 subscribers on YouTube.

Finally, a group from ABC is experimenting with ways to draw people to the network's site via what it calls "viewing parties" so people can chat with each other while watching, "live" (so to speak), hit shows like Grey's Anatomy. The interface the ABC party group showed off was interesting. They wanted, they said, to come up with something "as slick as the iPhone and as easy to use as AIM". They eventually came up with a three-dimensional spatial concept in which messages appear in bubbles that age by shrinking in size. Net old-timers might ask churlishly what's so inadequate about the interface of IRC or other types of chat rooms where messages appear as scrolling text, but from ABC's point of view the show is the centrepiece.

At least it will give people watching shows online something to do during the ads. If you're coming from a US connection, the ABC site lets you watch full episodes of many current shows; the site incorporates limited advertising. Perhaps in recognition that people will simply vanish into another browser window, the ads end with a button to click to continue watching the show and the video remains on pause until you click it.

The point of all these initiatives is simple and the same: to return TV to something people must watch in real-time as it's broadcast. Or, if you like, to figure out how to lure today's 20- and 30-somethings into watching television; Newshour's TV audience is predominantly 50- and 60-somethings.

ABC's viewing party idea is an attempt - as the team openly said - to recreate what the network calls "appointment TV". I've argued here before that as people have more and more choices about when and where to watch their favourite scripted show, sports and breaking news will increasingly rule television because they are the only two things that people overwhelmingly want to see in real time. If you're supported by advertising, that matters, but success will depend on people's willingness to stick with their efforts once the novelty is gone. The question to answer isn't so much whether you can compete with free (cue picture of a bottle of water) but whether you can compete with freedom (cue picture of evil file-sharer watching with his friends whenever he wants).

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, at her personal blog, or by email to netwars@skeptic.demon.co.uk (but please turn off HTML).

Posted by Wendy M. Grossman at 5:06 PM | Permalink | Comments (0) | TrackBacks (0)

Just how smart are humans anyway? Last week's Singularity Summit spent a lot of time talking about the exact point at which computer processing power would match that of the human brain, but that's only the first step. There's the software to make the hardware do stuff, and then there's the whole question of consciousness. At that point, you've strayed from computer science into philosophy and you might as well be arguing about angels on the heads of pins. Of course everyone hopes they'll be alive to see these questions settled, but in the meantime all we have is speculation and the snide observation that it's typical that a roomful of smart people would think that all problems can be solved by more intelligence.

So I've been trying to come up with benchmarks for what constitutes artificial intelligence, and the first thing I think is that the Turing test is probably too limited. In it, a judge has to determine which of two typing correspondents is the machine and which the human, That's fine as far as it goes, but one of the consistent threads that un through all this is a noticeable disdain for human bodies.

While our brain power is largely centralized, it still seems to me likely that both its grey matter and the rest of our bodies are an important part of the substrate. How we move through space, how our bodies react and feed our brains is part and parcel of how our minds work, however much we may wish to transcend biology. The fact that we can watch films of bonobos and chimpanzees and recognise our own behaviour in their interactions should show us that we're a lot closer to most animal species than we think - and a lot further from most machines.

For that sort of reason, the Turing test seems limited. A computer passes that test if, when paired against a human, the judge can't tell which is which. At the moment, it seems clear the winner is going to be spambots - some spam messages are already devised cleverly enough to fool even Net-savvy individuals into opening them sometimes. But they're hardly smart - they're just programmed that way. And a lot depends on the capability of the judge - some people even find Eliza convincing, though it's incredibly easy to send off-course into responses that are clearly those of a machine. Find a judge who wants to believe and you're into the sort of game that self-styled psychics like to play.

Nor can we judge a superhuman intelligence by the intractable problems it solves. One of the more evangelist speakers last weekend talked about being able to instantly create tall buildings via nanotechnology. (I was, I'm afraid, irresistibly reminded of that Bugs Bunny cartoon where Marvin pours water on beans to produce instant Martians to get rid of Bugs.) This is clearly just silly: you're talking about building a gigantic building out of molecules. I don't care how many billions of nanobots you have, the sheer scale means it's going to take time. And, as Kevin Kelly has written, no matter how smart a machine is, figuring out how to cure cancer or roll back aging won't be immediate either because you can't really speed up the necessary experiments. Biology takes time.

Instead, one indicator might be variability of response; that is, that feeding several machines the same input - or giving the same machine the same input at different times - produces different, equally valid interpretations. If, for example, you give a 10th grade class Jane Austen's Pride and Prejudice to read and report on, different students might with equal legitimacy describe it as a historical account of the economic forces affecting 18th century women, a love story, the template for romantic comedy, or even the story of the plain sister in a large family whose talents were consistently overlooked until her sisters got married.

In The Singularity Is Near, Ray Kurzweil laments that each human must read a text separately and that knowledge can't be quickly transferred from one to another the way a speech recognition program can be loaded into a new machine in seconds - but that's the point. Our strength is that our intelligences are all different, and we aren't empty vessels into which information is poured but stews in which new information causes varying chemical reactions.

You might argue that search engines can already do this, in that you don't get the same list of hits if you type the same keywords into Google versus Yahoo! versus Ask.com, and if you come back tomorrow you may get a different response from any one of them. That's true. It isn't the kind of input I had in mind, but fair enough.

The other benchmark that's occurred to me so far is that machines will be getting really smart when they get bored.

ZDNet UK editor Rupert Goodwins has a variant on this from when he worked at Sinclair Research. "If it went out one evening, drank too much, said the next morning, 'never again' and repeated the exercise immediately. Truly human." But see? There again: a definition of human intelligence that requires a body.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, at her personal blog, or by email to netwars@skeptic.demon.co.uk (but please turn off HTML).

Posted by Wendy M. Grossman at 5:43 PM | Permalink | Comments (0) | TrackBacks (0)

"I call it tracking," said a young woman. She had healthy classic-length hair, a startling sheaf of varyingly painful medical problems, and an eager, frequent smile. She spends some minutes every day noting down as many as 40 different bits of information about herself: temperature, hormone levels, moods, the state of the various medical problems, the foods she eats, the amount and quality of sleep she gets. Every so often, she studies the data looking for unsuspected patterns that might help her defeat a problem. By this means, she says she's greatly reduced the frequency of two of them and was working on a third. Her doctors aren't terribly interested, but the data helps her decide which of their recommendations are worth following.

And she runs little experiments on herself. Change a bunch of variables, track for a month, review the results. If something's changed, go back and look at each variable individually to find the one that's making the difference. And so on.

Of course, everyone with the kind of medical problem - diabetes, infertility, allergies, cramps, migraines, fatigue - that medicine can't really solve - has done something like this for generations. Diabetics in particularly have long had to track and control their blood sugar levels. What's different is the intensity - and the computers. She currently tracks everything in an Excel spreadsheet, but what she's longing for is good tools to help her with data analysis.

From what Gary Wolf, the organizer of this group, Quantified Self, says - about 30 people are here for its second meeting, after hours at Palo Alto's Institute for the Future to swap notes and techniques on personal tracking - getting out of the Excel spreadsheet is a key stage in every tracker's life. Each stage of improvement thereafter gets much harder.

Is this a trend? Co-founder Kevin Kelley thinks so, and so does the Washington Post, which covered this group's first meeting. You may not think you will ever reach the stage of obsession that would lead you to go to a meeting about it, but in fact, if the interviews I did with new-style health companies in the past year is any guide, we're going to be seeing a lot of this in the health side of things. Home blood pressure monitors, glucose tests, cholesterol tests, hormone tests - these days you can buy these things in Wal-Mart.

The key question is clearly going to be: who owns your health data? Most of the medical devices in development assume that your doctor or medical supplier will be the one doing the monitoring; the dozens of Web sites highlighted in that Washington Post article hope there's a business in helping people self-track everything from menstrual cycles to time management. But the group in Palo Alto are more interested in self-help: in finding and creating tools everyone can use, and in interoperability. One meeting member shows off a set of consumer-oriented prototypes - bathroom scale, pedometer, blood pressure monitor, that send their data to software on your computer to display and, prospectively, to a subscription Web site. But if you're going to look at those things together - charting the impact of how much you walk on your weight and blood pressure - wouldn't you also want to be able to put in the foods you eat? There could hardly be an area where open data formats will be more important.

All of that makes sense. I was less clear on the usefulness of an idea another meeting member has - he's doing a start-up to create it - a tiny, lightweight recording camera that can clip to the outside of a pocket. Of course, this kind of thing already has a grand, old man in the form of Steve Mann, who has been recording his life with an increasingly small sheaf of devices for a couple of decades now. He was tired, this guy said, of cameras that are too difficult to use and too big and heavy; they get left at home and rarely used. This camera they're working on will have a wide-angle lens ("I don't know why no one's done this") and take two to five pictures a second. "That would be so great," breathes the guy sitting next to me.

Instantly, I flash on the memory of Steve Mann dogging me with flash photography at Computers, Freedom, and Privacy 2005. What happens when the police subpoenas your camera? How long before insurance companies and marketing companies offer discounts as inducements to people to wear cameras and send them the footage unedited so they can study behavior they currently can't reach?

And then he said, "The 10,000 greatest minutes of your life that your grandchildren have to see," and all you can think is, those poor kids.

There is a certain inevitable logic to all this. If retailers, manufacturers, marketers, governments, and security services are all convinced they can learn from data mining us why shouldn't we be able to gain insights by doing it ourselves?

At the moment, this all seems to be for personal use. But consider the benefits of merging it with Web 2.0 and social networks. At last you'll be able to answer the age-old question: why do we have sex less often than the Joneses?

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, at her personal blog, or by email to netwars@skeptic.demon.co.uk (but please turn off HTML).

Posted by Wendy M. Grossman at 4:36 PM | Permalink | Comments (0) | TrackBacks (0)

It's easy to found an organization; it's hard to keep one alive even for as long as ten years. This week, the Foundation for Information Policy Research celebrated its tenth birthday. Ten years is a long time in Internet terms, and even longer when you're trying to get government to pay attention to expertise in a subject as difficult as technology policy.

My notes from the launch contain this quote from FIPR's first director, Caspar Bowden, which shows you just how difficult FIPR's role was going to be: "An educational charity has a responsibility to speak the truth, whether it's pleasant or unpleasant." FIPR was intended to avoid the narrow product focus of corporate laboratory research and retain the traditional freedoms of an academic lab.

My notes also show the following list of topics FIPR intended to research: the regulation of electronic commerce; consumer protection; data protection and privacy; copyright; law enforcement; evidence and archiving; electronic interaction between government, businesses, and individuals; the risks of computer and communications systems; and the extent to which information technologies discriminate against the less advantaged in society. Its first concern was intended to be researching the underpinnings of electronic commerce, including the then recent directive launched for public consultation by the European Commission.

In fact, the biggest issue of FIPR's early years was the crypto wars leading up to and culminating in the passage of the Regulation of Investigatory Powers Act (2000). It's safe to say that RIPA would have been a lot worse without the time and energy Bowden spent listening to Parliamentary debates, decoding consultation papers, and explaining what it all meant to journalists, politicians, civil servants, and anyone else who would listen.

Not that RIPA is a fountain of democratic behavior even as things are. In the last couple of weeks we've seen the perfect example of the kind of creeping functionalism that FIPR and Privacy International warned about at the time: the Poole council using the access rules in RIPA to spy on families to determine whether or not they really lived in the right catchment area for the schools their children attend.

That use of the RIPA rules, Bowden said at at FIPR's half-day anniversary conference last Wednesday, sets a precedent for accessing traffic data for much lower level purposes than the government originally claimed it was collecting the data for. He went on to call the recent suggestion that the government may be considering a giant database, updated in real time, of the nation's communications data "a truly Orwellian nightmare of data mining, all in one place."

Ross Anderson, FIPR's founding and current chair and a well-known security engineer at Cambridge, noted that the same risks adhere to the NHS database. A clinic that owns its own data will tell police asking for the names of all its patients under 16 to go away. "If," said Anderson, "it had all been in the NHS database and they'd gone in to see the manager of BT, would he have been told to go and jump in the river? The mistake engineers make too much is to think only technology matters."

That point was part of a larger one that Anderson made: that hopes that the giant databases under construction will collapse under their own weight are forlorn. Think of developing Hulk-Hogan databases and the algorithms for mining them as an arms race, just like spam and anti-spam. The same principle that holds that today's cryptography, no matter how strong, will eventually be routinely crackable means that today's overload of data will eventually, long after we can remember anything we actually said or did ourselves, be manageable.

The most interesting question is: what of the next ten years? Nigel Hickson, now with the Department of Business, Enterprise, and Regulatory Reform, gave some hints. On the European and international agenda, he listed the returning dominance of the large telephone companies on the excuse that they need to invest in fiber. We will be hearing about quality of service and network neutrality. Watch Brussels on spectrum rights. Watch for large debates on the liability of ISPs. Digital signatures, another battle of the late 1990s, are also back on the agenda, with draft EU proposals to mandate them for the public sector and other services. RFID, the "Internet for things" and the ubiquitous Internet will spark a new round of privacy arguments.

Most fundamentally, said Anderson, we need to think about what it means to live in a world that is ever more connected through evolving socio-technological systems. Government can help when markets fail; though governments themselves seem to fail most notoriously with large projects.

FIPR started by getting engineers, later engineers and economists, to talk through problems. "The next growth point may be engineers and psychologists," he said. "We have to progressively involve more and more people from more and more backgrounds and discussions."

Probably few people feel that their single vote in any given election really makes a difference. Groups like FIPR, PI, No2ID, and ARCH remind us that even a small number of people can have a significant effect. Happy birthday.

Wendy M. Grossman's Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, at her personal blog, or by email to netwars@skeptic.demon.co.uk (but please turn off HTML).

Posted by Wendy M. Grossman at 2:48 PM | Permalink | Comments (0) | TrackBacks (0)

A few months ago, a neighbour phoned me and asked if I'd be willing to position a camera on my windowsill. I live at the end of a small dead-end street (or cul-de-sac), that ends in a wall about shoulder height. The railway runs along the far side of the wall, and parallel to it and further away is a long street with a row of houses facing the railway. The owners of those houses get upset because graffiti keeps appearing alongside the railway where they can see it and covers flat surfaces such as the side wall of my house. The theory is that kids jump over the wall at the end of my street, just below my office window, either to access the railway and spray paint or to escape after having done so. Therefore, the camera: point it at the wall and watch to see what happens.

The often-quoted number of times the average Londoner is caught on camera per day is scary: 200. (And that was a few years ago; it's probably gone up.) My street is actually one of those few that doesn't have cameras on it. I don't really care about the graffiti; I do, however, prefer to be on good terms with neighbours, even if they're all the way across the tracks. I also do see that it makes sense at least to try to establish whether the wall downstairs is being used as a hurdle in the getaway process. What is the right, privacy-conscious response to make?

I was reminded of this a few days ago when I was handed a copy of Privacy in Camera Networks: A Technical Perspective, a paper published at the end of July. (We at net.wars are nothing if not up-to-date.)

Given the amount of money being spent on CCTV systems, it's absurd how little research there is covering their efficacy, their social impact, or the privacy issues they raise. In this paper, the quartet of authors – Marci Lenore Meingast (UC Berkeley), Sameer Pai (Cornell), Stephen Wicker (Cornell), and Shankar Sastry (UC Berkeley) – are primarily concerned with privacy. They ask a question every democratic government deploying these things should have asked in the first place: how can the camera networks be designed to preserve privacy? For the purposes of preventing crime or terrorism, you don't need to know the identity of the person in the picture. All you want to know is whether that person is pulling out a gun or planting a bomb. For solving crimes after the fact, of course, you want to be able to identify people – but most people would vastly prefer that crimes were prevented, not solved.

The paper cites model legislation (PDF) drawn up by the Constitution Project. Reading it is depressing: so many of the principles in it are such logical, even obvious, derivatives of the principles that democratic governments are supposed to espouse. And yet I can't remember any public discussion of the idea that, for example, all CCTV systems should be accompanied by identification of and contact information for the owner. "These premises are protected by CCTV" signs are everywhere; but they are all anonymous.

Even more depressing is the suggestion that the proposals for all public video surveillance systems should specify what legitimate law enforcement purpose they are intended to achieve and provide a privacy impact assessment. I can't ever remember seeing any of those either. In my own local area, installing CCTV is something politicians boast about when they're seeking (re)election. Look! More cameras! The assumption is that more cameras equals more safety, but evidence to support this presumption is never provided and no one, neither opposing politicians nor local journalists, ever mounts a challenge. I guess we're supposed to think that they care about us because they're spending the money.
The main intention of Meingast, Pai, et al, however, is to look at the technical ways such networks can be built to preserve privacy. They suggest, for example, collecting public input via the Internet (using codes to identify the respondents on whom the cameras will have the greatest impact). They propose an auditing system whereby these systems and their usage is reviewed. As the video streams become digital, they suggest using layers of abstraction of the resulting data to limit what can be identified in a given image. "Information not pertinent to the task in hand," they write hopefully, "can be abstracted out leaving only the necessary information in the image." They go on into more detail about this, along with a lengthy discussion of facial recognition.

The most depressing thing of all: none of this will ever happen, and for two reasons. First, no government seems to have the slightest qualm of conscience about installing surveillance systems. Second, the mass populace don't seem to care enough to demand these sorts of protections. If these protections are to be put in place at all, it must be done by technologists. They must design these systems so that it's easier to use them in privacy-protecting ways than to use them in privacy-invasive ways. What are the odds?

As for the camera on my windowsill, I told my neighbour after some thought that they could have it there for a maximum of a couple of weeks to establish whether the end of my street was actually being used as an escape route. She said something about getting back to me when something or other happened. Never heard any more about it. As far as I am aware, my street is still unsurveilled.

Wendy M. Grossman’s Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, at her personal blog, or by email to netwars@skeptic.demon.co.uk (but please turn off HTML).

Posted by Wendy M. Grossman at 5:11 PM | Permalink | Comments (0) | TrackBacks (0)

I seem to have spent the summer dodging in and out of science fiction novels featuring four general topics: energy, security, virtual worlds, and what someone at the last conference called "GRAIN" technologies (genetic engineering, robotics, AI, and nanotechnology). So the summer started with doom and gloom and got progressively more optimistic. Along the way, I have mysteriously lost a lot of hats. The phenomena may not be related.

I lost the first hat in June, a Toyota Motor Racing hat (someone else's joke; don't ask) while I was reading the first of many very gloomy books about the end of the world as we know it. Of course, TEOTWAWKI has been oft-predicted, and there is, as Damian Thompson, the Telegraph's former religious correspondent, commented when I was writing about Y2K – a "wonderful and gleeful attention to detail" in these grand warnings. Y2K was a perfect example: a timetable posted to comp.software.year-2000 had the financial system collapsing around April 1999 and the cities starting to burn in October…

Energy books can be logically divided into three categories. One, apocalyptics: fossil fuels are going to run out (and sooner than you think), the world will continue to heat up, billions will die, and the few of us who survive will return to hunting, gathering, and dying young. Two, deniers: fossil fuels aren't going to run out, don't be silly, and we can tackle global warming by cleaning them up a bit. Here. Have some clean coal. Three, optimists: fossil fuels are running out, but technology will help us solve both that and global warming. Have some clean coal and a side order of photovoltaic panels.

I tend, when not wracked with guilt for having read 15 books and written 30,000 words on the energy/climate crisis and then spent the rest of the summer flying approximately 33,000 miles, toward optimism. People can change – and faster than you think. Ten years ago, you'd have been laughed off the British isles for suggesting that in 2007 everyone would be drinking bottled water. Given the will, ten years from now everyone could have a solar collector on their roof.

The difficulty is that at least two of those takes on the future of energy encourage greater consumption. If we're all going to die anyway and the planet is going inevitably to revert to the Stone Age, why not enjoy it while we still can? All kinds of travel will become hideously expensive and difficult; go now! If, on the other hand, you believe that there isn't a problem, well, why change anything? The one group who might be inclined toward caution and saving energy is the optimists – technology may be able to save us, but we need time to create create and deploy it. The more careful we are now, the longer we'll have to do that.

Unfortunately, that's cautious optimism. While technology companies, who have to foot the huge bills for their energy consumption, are frantically trying to go green for the soundest of business reasons, individual technologists don't seem to me to have the same outlook. At Black Hat and Defcon, for example (lost hats number two and three: a red Canada hat and a black Black Hat hat), among all the many security risks that were presented, no one talked about energy as a problem. I mean, yes, we have all those off-site backups. But you can take out a border control system as easily with an electrical power outage as you can by swiping an infected RFID passport across a reader to corrupt the database. What happens if all the lights go out, we can't get them back on again, and everything was online?

Reading all those energy books changes the lens through which you view technical developments somewhat. Singapore's virtual worlds are a case in point (lost hat: a navy-and-tan Las Vegas job): everyone is talking about what kinds of laws should apply to selling magic swords or buying virtual property, and all the time in the back of your mind is the blog posting that calculated that the average Second Life avatar consumes as much energy as the average Brazilian. And emits as much carbon as driving an SUV for 2,000 miles. Bear in mind that most SL avatars aren't figured up that often, and the suggestion that we could curb energy consumption by having virtual conferences instead of physical ones seems less realistic. (Though we could, at least, avoid airport security.) In this, as in so much else, the science fiction writer Vernor Vinge seems to have gotten there first: his book Marooned in Real Time looks at the plight of a bunch of post-Singularity augmented humans knowing their technology is going to run out.

It was left to the most science fictional of the conferences, last week's Center for Responsible Nanotechnology conference (my overview is here) to talk about energy. In wildly optimistic terms: technology will not only save us but make us all rich as well.

This was the one time all summer I didn't lose any hats (red Swiss everyone thought was Red Cross, and a turquoise Arizona I bought just in case). If you can keep your hat while all around you everyone is losing theirs…

Wendy M. Grossman’s Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, at her personal blog, or by email to netwars@skeptic.demon.co.uk (but please turn off HTML).

Posted by Wendy M. Grossman at 3:43 PM | Permalink | Comments (0) | TrackBacks (0)

I spent three days last week in Basel being taken around to see various pieces of research the research outfits around there are doing into nanoscience, courtesy of the European Union of Scientice Journalists' Associations (my affiliation is with the Association of British Science Writers). All very interesting stuff, and difficult to summarize intelligently in a few hundred words, though I made a stab at some of the medical stuff. The thing that most struck me immediately, though, was how different it all was from the image of nanotechnology I'd half-formed from odds and ends I'd read or heard about in the media.

I probably just don't read enough.

The first time I ever heard of nanotechnology, though I'm not sure they used the name, was in a three-part 1988 documentaryTV series called What is Truth?: Seeing is Not Knowing. It was produced by the distinguished science producer and writer Karl Sabbagh, and looked at how we know what we know about things we can't examine directly, such as the contents of memory, the very large (space) and the very small (molecules). Two enduring images stick with me all these years later: a guy riding a bicycle through the CERN particle accelerator to cover the distance to the bit that needed repairs, and their mock-up of what a nanofactory might be like. By then people were already talking about the idea that we could have machines in our homes into which you put ingredients and instructions and out of which you later take whole devices or whatever. The machine was played by a dishwasher and the emerging device by a boom box, and the whole thing looked pretty hokey, but still: molecular manufacturing.

But that's not what the people in Basel were doing at all; at no point in the three days did anyone talk about building consumer devices or the grey goo that belongs in a horror movie. Instead, what kept reappearing was various types of microscopes - atomic force, scanning probe, even a synchrotron. From those, we saw a lot of highly detailed images of really tiny things, such as collagen fibers waiting to cause havoc in the human bloodstream and three-dimensional images of rat brains.

I think everyone's favourite presentation was that of Marc Creus, from the Institut de Microtechnique in Neuchâtel, who said cheerfully he was there to talk about a hole. Actually, a nanopore, 25 nanometers in diameter. The idea is to build on a technique created by the engineer Wallace H. Coulter, who created a simple device – essentially, a box with two chambers divided by a membrane (in its first prototype, the cellophane off a pack of cigarettes) with a small hole in it (originally, melted with the heated point of a sewing needle) – to count microscopic particles suspended in a fluid. A solution passes through the hole simultaneously with an electric current; when a particle goes through, the current shows a change proportional to the size of the particle. The particle, in other words, briefly partially blocks the hole.

The way Creus told it, Coulter had been experimenting with paint, but one night left the paint open. The next night, finding it had dried out, he looked around for another liquid – and wound up using blood. The Coulter Principle, as it's now known, is used all over the world for analyzing blood samples ("complete blood cell" counts). He had trouble getting a patent on it, by the way; the examiner thought it was too simple, and anyway you can't patent a hole. He eventually got his patent in 1953 and became quite wealthy from his device.

Creus is trying to shrink the Coulter Principle with the idea of exploring the nanoscale: nanopores should make it possible to count protein molecules. You could, for example, test for the presence of a particular protein by adding them to a device that already contains its antibodies. The protein bound to the antibody will be a bigger molecule than either on its own.

Even weirder, Urs Staufer, from the same institute, is using nanoscience to explore…Mars. There's something very strange about the notion of using something tiny to study something really large. But the deal is that one of these scanning proble microscopes, specially adapted, will be on the first Mars Scout mission, due to launch in August. A robot arm will go along scooping up samples of…what do you call it when it's Mars? It can't be earth, can it? Anyway, the robot arm pours the sample on a wheel that rotates in front of the microscope, and the images are sent to Tucson and everyone has four hours to decide if they want to look at it more closely and compile the commands to send for the next go-round. The hope is that they'll find ice underneath the surface and will be able to dig down and investigate it.

I suppose all this makes sense. You can't really manufacture anything, at any scale, until you understand how it all works, just as you can't colonize anywhere until you've explored it. If they get down the nanoscale far enough, will they plant a tiny Swiss flag?

Wendy M. Grossman’s Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, at her personal blog, or by email to netwars@skeptic.demon.co.uk (but please turn off HTML).

Posted by Wendy M. Grossman at 12:40 PM | Permalink | Comments (0) | TrackBacks (0)