Neurotech is approaching facial recognition’s dangerous Crappy Valley: good enough to get headlines, not good enough to be reliably used at scale.
What makes Clearview AI so dangerous, after all, isn’t that it allows quick, convenient, and pervasive flawless identification of any person in any image: that would indeed have enormous privacy implications that would probably be a net negative against positive use cases, but it could be regulated in a way to minimize the negative impact and maximize the positive ones. The problem is that it doesn’t really work. It’s good enough to pass customer demos and impress the press, but once you start using it for tens of thousands of images a day, the error rate is so big that you start clogging any investigative system that’s built around the assumption that it works.
The key thing to understand, though, is that their customers don’t care. Institutions that hire their services aren’t historically over-committed to transparent accountability, to put it mildly. What they are buying is the reputation of the technology, not the performance. Enough people, press, juries, etc, believe or pretend to believe that facial recognition works like it does on TV that it gives you a lot of leeway to justify whatever it is you already wanted to do. Whether you’re worried about or attracted to its misuse, it looks like it should work, as we’ve grown used to it on fiction, and it’s anyway intuitive enough conceptually, even if in practice getting a low-enough error rate under realistic conditions is technically very very hard.
Neurotech, in its current stage, is in an even more dangerous conjunction of how it looks, how it works, and what it’s useful for. Consider a likely first use case in law enforcement, a “brain lie detector.”
- The concept is familiar from fiction and eminently plausible in an intuitive sense (if you can peek inside somebody’s brain, it must be easy to see if they are lying, right?), so it increases your social and institutional support.
- What’s more, the popular press has over-hyped very limited results so much that many people believe we’ve already built one or we’re very close to it.
- We haven’t and we’re far from being able to, so it wouldn’t “guarantee” more accurate law enforcement investigations, and whatever false positive rate it has, it’ll lead to an unacceptable number of innocents risking jail because “the brain machine says they are lying.”
- Institutions that still rely on things like polygraphs or “enhanced interrogation techniques” aren’t particularly nitpicky about false positives, and in fact consider them something of a bonus due to very poisonous incentive structures.
This is primarily dangerous at a societal level, and secondarily very bad for neurotech itself. While it superficially leads to more investment in the area, it’s mostly directed to crappy, superficial research, and displaces deeper, slower, but ultimately more fruitful avenues. The infamous “trough of disillusionment” isn’t a feature of technological development per se, but a consequence of the interaction of self-interested over-hyping in tacit collusion with less than sufficiently skeptical media and over-enthusiastic procurement and investment, leading to deployments ahead of capability, maximizing negative side effects and leading to an unnecessary, often long phase of disappointment. Just as it’s getting harder to get the best scientists to work on facial recognition, widespread deployment of crappy neurotech for unethical purposes (and, again, the fact that it works badly it’s not a bug for the worst potential customers, it’s a feature), whatever it does for investment flows, it’ll push away the best researchers, probably delaying progress in the serious development of actually useful science and technology.
On the flip side, the existence of the Crappy Valley means that we have a regulatory strategy that complements very well the development of regulations directly driven by the containment of negative impacts, which is of course a necessary guideline, but
- Only gets political traction after those negative impacts begin to be visible.
- Doesn’t help you with the constituencies that believe the positive impacts justify the negative ones.
The way to sidestep those factors is the early regulation of emerging technologies through the preemptive development of strict standards of performance. Laws prohibiting the deployment of facial recognition without passing performance tests mimicking real-world performance would have slowed it down significantly, and arguably it still wouldn’t have been deployed. You’d have to prove in a huge sample set of very different conditions of image quality, changes in aspect, etc, etc, etc, a false positive rate close to zero. This would have been a much easier regulation to pass when the technology was still in its early stages, with the support of groups that wouldn’t support banning a technology that they believe works.
We are approaching the end of that stage with neurotech. More and more people believe or pretend to believe that it works, more and more large companies and potential customers believe or pretend to believe it works, and it doesn’t really work, not robustly enough to be used for what it would be used. It has yet to be commodified, but we’re getting close, so it’s the last ideal time to push for regulation that, say, would prohibit the deployment of a “brain lie detector” unless it passes tests over statistically representative subsets of the population and potential conditions. It’s easy to argue for, and currently very difficult to meet, requirements like practically zero false positives; even 1% would be monstrously large in any real-world application. What’s more, you can, should, and must require the same false positive rate for people with depression, ADHD, a hangover, lack of sleep, not talking in their mother tongue, prone to epilepsy, taking any sort of medication, suffering from chronic pain, or just with a brain that doesn’t quite work like those of the couple dozen subject cases (most often young white college students) that are enough nowadays to get “Scientists Build Machine That Can Read Minds” headlines. Never mind that “lying” is probably a single label for many different cognitive process, that every brain does everything differently (or otherwise there’d be no cases of recovery from even small strokes), etc, etc. It’d be an absolute mess to validate, even more expensive, risky, and slower than it is for new medications, but the requirements would be entirely justifiable, and harder to argue against.
This would help society and it would help serious neurotech R&D. When it comes to some areas of institutional convenience, bad research, like bad money, displaces good research. Ex ante regulation based on strict performance requirements is an useful complement to that based on, e.g., impacts on privacy rights. In the Crappy Valley we find both technologies on their way to work and technologies that will never work, and all of them are potentially dangerous. Cleaning up that valley would go a long way towards helping us deal with both.