I don't know if I mentioned it before, but I think AI has so many interesting applications in philosophy. Not by feeding psychological texts into it, not by using it for philosophical debates or by using it at all in the classical sense. What I mean is that it is interesting as a subject to study. A technical understanding of the techniques used in machine learning can give you so many interesting concepts to ponder. The one I want to talk about today is attention.
The main invention that brought us the transformer architecture, which is what all current LLMs are built on, was the "attention layer". I don't want to go into the technical details too much but, in a nutshell, this is what allows an artificial neural network to decide which parts of the input are relevant and which are not. Or in other words: which parts of the input it wants to pay attention to. Isn’t it poetic that attention, the most contested resource in the modern world, seems to be the key to intelligence?
If you see intelligence as a competitive game, this is less surprising. After all, the act of “tricking” someone mainly involves steering their attention away from what you are doing so they will only realize it too late. So in these games you win by deliberately paying attention to things you’re not supposed to. Trying to see past the distractions, ignoring the loud, the flashy and the exciting is what makes you less vulnerable to being tricked. Unfortunately we like being tricked.
Think about it: if you engage in any of the many experiences that were crafted for the mass market, paying attention to anything you’re not supposed to only leads to disappointment. The best analogy is a magic show: if you pay attention to the hidden ropes, the black cloth, the trapdoors, the mirrors and the smoke, you might just see through all the tricks. The suspension of disbelief this whole experience is built on is gone. But it’s not just the magic shows. There’s so many ways you can disappoint yourself by paying attention to the logical consistency of action movies, the electrical installations at Disneyland or the fact that many dishes you get at various restaurants come from the same big bags of frozen food. It’s almost as if the modern world wants to teach us not to look in the wrong corners.
It’s a vicious feedback loop. Big companies try to find out what their customers care about through market research, so they focus their products on these aspects and drop the others for cost efficiency. As a result, consumers who want to enjoy their products are discouraged from shifting their attention so, at some point, an equilibrium is reached where everyone agrees on what to pay attention to. Competitors will have a difficult time if they focus on other aspects because consumers don’t pay attention to them. Unless they actively steer people’s attention by “being loud”. And I don’t mean that in an auditory sense, but more general. Using the primitive attention guidance mechanisms that evolution constructed into the human experience because it helped us to avoid danger a few thousand years ago. The result is a world where everyone’s shouting, flashing and exaggerating.
It isn’t a new thought. Many people before me have written much more comprehensively about how this overstimulation, the constant battle for attention and the resulting shortening of our attention spans is causing us to become dumber. But what I wanted to highlight is the poetic contrast of how much worse human attention is going to get because now machines can do it. It is so tempting. It feels like finally we have a cure for all that madness. A machine that we can send into that crazy world full of screaming and flashing, that will pay attention in our stead to give us only what matters to us. A way to filter out all that noise and distill it into what we were really looking for.
But how are the machines going to know what matters to us? How are they going to know what we are really looking for? We barely even know ourselves. If we put them in charge of our attention, won’t we all attend to exactly what we’re supposed to? It’s even worse than complying to avoid disappointment, if we put the machines between us and the world we will be physically unable to look where we’re not supposed to. Looking at AI “art” beautifully illustrates this. The images produced by generative models look stunning. They seem like they were produced with lots of effort by highly skilled individuals. Except they don’t. They might look like that at first glance. They might look like that if you look at the faces, the subjects, the flashy parts, those parts that people usually pay attention to. But as soon as you look at parts of the image you’re not supposed to, you’ll notice the inconsistencies. What’s going on in the background? What is that thing on the shelf? And, holy shit, what’s wrong with that person’s hand?!
This is the true danger of AI. It’s not going to start building killer robots anytime soon. But it’s an imminent threat that we might lose our ability to look where we’re not supposed to. As you might know, I like to end my posts on an appeal. But in this instance, you’re someone who I know for a fact is reading my blog. A wall of text under a black and white comic on a random site on the internet that only has an RSS feed. You’re probably already someone who enjoys looking where you’re not supposed to. So, please keep doing that.
