Cyber-weapons as a form of magic, and why we can't code our way to a safer internet

2018-04-18

A "cyber-weapon" isn't a thing, but a skill. It's not an object you can blow up with a missile or send a UN team to inspect, but the technical knowledge of how to identify and exploit a set of systems, written clearly enough that a computer can do it. It's a recipe that can make millions of copies of itself even as it bakes the cake it describes, and even if all copies were to be deleted, all it would take to recreate it is a single person with the technical knowledge writing it down again.

It's a poem that, written down, unread by human eyes, causes havoc in the real world. You might as well apply the concepts of "deterrence" and "arms control" to a rumor. By calling them "weapons," politicians and the military, while reflecting the uses for them they desire and fear, misunderstand their nature. People who, in other contexts and issues, claim it impossible to control the production and distribution of something as solid as an AR-15, attempt to ensure the security of their computational infrastructure by controlling the production and distribution of pure knowledge, in an era where the circuits inside a car's door could drown out the output of any printing press.

It's hopeless. And what's more, the vulnerabilities exploited by these weapons — the grains of truth that make the rumors work — aren't technical problems, not any more than a homeless person dying of hunger or cold is a medical issue. Yes, the immediate cause is technical — code written in haste to outpace a hype cycle, architectures built for external control and surveillance, and therefore half-insecure by design — but that's just as the short-term rational response of organizations to the business, legal, and political environment in which their find themselves. Safe and private computing, like healthy food, is less an impossibility than an often disingenuous gourmet option upon a fundamentally different social economic default, one often out of reach for those without the economic resources — unstressed, unharried free time being sometimes the scarcest — to acquire them. It can technically be done, it's just that the systemic incentives — perhaps primarily the psychological ones — aren't there.

Melvin Conway coined Conway's Law, which can be paraphrased as the observation that software systems created by an organization cannot but reflect the structure of the organization itself. Insecure, powerful, an arena of commercial and political and manipulation as much as one of interpersonal empathy and shared knowledge, a place we both need and distrust, we don't have any of the possible internets our societies claimed to want to build, but we do have the only one, perhaps, that reflects who we are. That's not a technical problem, and thinking it is (the unspoken axiom that everything is) might be at the root of what ails it.