The link: Bikerouter.de (via Scope of Work).
What it is: A map/route planning tool for biking. Think "Waze specifically for biking."
Why I'm posting it: Go to the site. Click on the tools icon. Click on the Profile tab: It has a simple programming language you can use to specify your cost function - that is, what makes you prefer a route over another.
Why it's an important example: Software — AI as our current if problematic shorthand for its cutting edge — can solve your users' problem, but for that it has to know what their problem, or their desire, is. The default approach of learning about your user through extensive surveillance and then solving whatever problem or desire your model says they have is vastly more inefficient, insecure, and slow than just asking them and honoring their wishes. The more useful you want your software to be, the more knowledgeable your target users are, the more you have to trust them to have the knowledge and the ability to express this knowledge. A bike routing app that doesn't trust or allow users to tell it with as much detail and flexibility as they want will never match the power and usefulness of one that does.
The suggestion: Apps that "learn from users in the background" should really be "configurable apps with a guess-the-configuration mode that can be overridden," and "configurable apps" should be "programmable apps with simple buttons that generate code that can be rewritten." Build the most flexible version first, because it's the one that will frame your own thinking about what it can be. Treat your users like grownups, and they'll take your tools to places you'd have never imagined.
(The whispered question: Explicit programmability is the root of computers' power. To empower somebody in a computing environment is to give them the ability to program, that is, to state their intent in explicit, arbitrarily complex, and open-ended ways. What can you say about a platform that doesn't even consider giving its user the possibility? What does this say about where they want the power to be and to remain?)