SPEAKER_03: 0:15

Welcome to the Where Shall We Meet podcast. Our guest this week is Joscha Bach. He's a German cognitive scientist, artificial intelligence researcher, and philosopher of mind who consistently bridges the gap between what human intelligence is and what machines could become. He has an MA in computer science and a PhD in cognitive science. Over the course of his career, he has held research positions at institutions such as the MIT Media Lab and the Harvard Program for Evolutionary Dynamics.

SPEAKER_02: 0:43

Bach is best known for his work on computational models of human-like cognition. He developed the cognitive architecture called Microbe Psi, exploring how perception, motivation, emotion, and decision making interact in autonomous agents. He is the author of Principles of Synthetic Intelligence. In addition to his academic output, he has taken roles in applied AI research and strategy, bringing theoretical insight into real-world settings.

SPEAKER_03: 1:11

What sets his approach apart is his deeply integrative mindset. He treats intelligence not just as surface behavior or pattern recognition, but as an emergent result of rich internal models of world and self. His philosophical lens brings questions of consciousness, free will, meaning into the technical domain, framing AI and cognition as part of a broader inquiry into what it means to think, feel, and act.

SPEAKER_02: 1:35

We talk about we live in a story, not in the physical world.

SPEAKER_03: 1:39

Consciousness does not depend on the substrate.

SPEAKER_02: 1:42

Can you learn reality by just watching YouTube?

SPEAKER_03: 1:45

Alternative approaches to building AI.

SPEAKER_02: 1:48

Intuition is the part of your mind you cannot reflect.

SPEAKER_03: 1:52

The constraint to being superhuman only applies to humans.

SPEAKER_02: 1:56

There is no obligation to unite your many selves. You're gonna need all your cognitive power for this one. We recommend you switch off all other devices, put your headphones on, and take a deep dive.

SPEAKER_03: 2:11

Hi, this is Umida Snari.

SPEAKER_02: 2:13

And Natasha McElhone, and with us today we have Josha Bach.

SPEAKER_03: 2:18

Hey Josha.

SPEAKER_02: 2:19

Welcome. Thank you so much for making the time. Happy to be here.

SPEAKER_03: 2:23

Great. Yosha, today we want to talk about consciousness and AI. But a way to get us there is to talk a little bit about the way you think about the fabric of reality and how we as humans fit into that world. And a crucial thing to do there is delineate between the world of too many parts to count, the causal world and the world that's in our head.

SPEAKER_04: 2:57

In some sense, I felt when I was young, when I looked at the different narratives that people had about the nature of reality, that we have to make this decision at some point. Do I live in a physical world that is ultimately mechanical? And uh the alternative is do I live in a dream? It seemed to me that all these occult theories, um, Eastern philosophy, animism, and so on all posit the latter, whereas the scientific worldview tells us that we live in a mechanical reality. And at some point we have to notice that uh both is probably true. Right? There is uh part of the world that doesn't change when we stop looking at it and we stop believing in it. There is something that seems to be completely independent from the way in which our mind works. But the perception of reality seems to be completely dependent on how our mind works. That is, uh sometimes I dream at night and I perceive a reality that has nothing to do with what my senses can perceive. It's entirely representational. And the nature of the things that I perceive during the day, they are just as representational. So the things that are matter to me, like colors and geometric shapes of faces and emotions and uh the way in which things seem to be moving through space. This is all not physical. This is all a dream. And this dream needs to be created by something, right? That's pretty complicated, there are regularities in it, it's compositional, that means it's made of parts that can change individually and that I can turn on and off to some degree individually. So this means there needs to be some kind of machinery outside of me that is producing me, that outside of my mind that is making my mind possible, some loom that weaves me. And that loom is the physical world, right? This is the hypothesis of physicalism says that there is a causally closed mechanical layer, and we are directly supervening on it. And the alternative to physicalism is that there still needs to be something that produces it, but it's in some kind of more distant parent universe. So we cannot directly interact with it or measure it. Instead, we will be insulated by it through some kind of simulation.

SPEAKER_03: 5:06

Simulation theory.

SPEAKER_04: 5:07

Yes. And so there are many forms of simulation theory. In some sense, you could say that uh the naive interpretation of Christian mythology is a simulation theory. God is a coder who has created this world. But this world in which we are in, in which we have money and houses and partners and uh cattle and so on, this is all not real. This is a game world, some kind of multiplayer adventure that our souls are playing, but we are completely immersed in it. So the our souls are more real, they sit next to God. But uh the part that is playing doesn't know about this because there are no senses that can us connect directly to each other. Instead, we have to go through this game world that we are fully immersed in. But this game world is ephemeral. And God in this world is a hacker who also has um right access to reality and can make money, for instance, for us or miracles. Exactly. It can change the fabric of our reality, and we don't have that right access. We have to go through the regular channels of the game. And uh the devil lacks this vision of God, but uh the devil can corrupt us, he can try to cheat us as our of our own one-two currency, our souls, corrupt us, and uh in exchange for worthless in-game items. And in some sense, that's the perspective that Christianity seems to be having uh on that, from at least from our modern framework. And uh, I don't think that Christianity is identical to this type of mythology. It's um more or less a perspective that is meant to induce a particular psychological configuration in which we collectively enact an agent that is a direction of its own. And we perceive ourselves being also part of these collective agents of things that go across people and across humans and animals and across humans in nature somehow. So we also perceive that our mind to some degree seems to be outside of us, not just in the sense that it's outside of our body, because our body is a fiction of our mind. This is a virtual entity, right? We perceive it as a construct that we are making, but the uh mind that we experience to have changes in the presence of other people. It changes based on the environment. So part of us depends on this ambience in which we are, some of us as feedback loops that go through the environment back into us. So that's also part of this reality that we are in. But um, by and large, the world that I experience is a game engine generated in my own brain, right? It's very similar to a game engine on a computer.

SPEAKER_02: 7:34

When does that start? Is it with language or is it in utero?

SPEAKER_04: 7:39

Yeah, it starts long before language. And since our bodies and most of our mind seems to assemble itself in utero, and you see a newborn child, it already has a number of representations, it seems to have a model of its body surface, it knows when it's being pinched, where on the surface of the body that's happening, and so on. It seems to be pretty clear to me that there is already some centralized awareness in the mind of the newborn.

SPEAKER_02: 8:04

So there's never a time that we're not in it.

SPEAKER_04: 8:07

Well, there are modes when we are unaware, but these are modes in which we don't exist, right? This thing that is me, this conscious observer, is only there when the light is on. And this state when the light is on, for instance, is only present when I am either awake or conscious in a dream. So there needs to be some kind of loop closed in which my observations are integrated with some kind of memory of what I observed and leads to the next thought, leads to the next impression, leads to the next percept. And when that loop is interrupted, then the observer ceases to exist. And for instance, in deep sleep or the anesthesia, we are not conscious. And possibly we are also not conscious a lot of the time. When we don't have memories of the last moment that we've been in, does this just mean that we don't have access to these memories, or has there nothing been recorded, or was there nobody home? Well, we can't know. Maybe consciousness is something that's only there intermittently.

SPEAKER_03: 9:05

Right. So from what you say, what follows is that free will is also a story that's part of this game engine that I've created in my mind, and the universe is deterministic, really. So let's talk a little bit about the perception of this free will, how that works together.

SPEAKER_04: 9:20

Yeah. So I'm now come around to the notion that the universe is not deterministic. For a long part of my life, I thought it must be deterministic. Yes. And uh so the reason is um it's uh it seems to be a multivase system, but uh, we can go into this much later. The idea is relatively simple. Uh quantum mechanics right now suggests that the simplest way to resolve um our uh observables, um our measurements and physics is that the universe does not have a single timeline, but the universe is constantly branching into many timelines.

SPEAKER_03: 9:50

Yeah. So David Deutsch's interpretation. Yeah, yeah.

SPEAKER_04: 9:55

And uh so when you are in such a system, when you are computed by such a system, the system itself can be deterministic, right? You can have a computer that is computing many possibilities simultaneously, and every step is branching into these possibilities. And that can be a thing that is happening deterministically in a sense that it's happening in in a way that has no variance in it. There is no um no statistical noise in it. But when you're being computed by it, you don't know in which branch you will end up. Because some version of you will be in all the branches, but you don't know which version you're going to be. But that means that from your own perspective, the universe will not look deterministic. It will have elements that cannot be predicted by anything. Because you don't know which portion of reality is going to be yours. Right. Exactly. It changes at every step. So the universe that you would sub experience that you would go through is one that is deterministic.

SPEAKER_03: 10:49

But uh the question is, but overall it's not because you don't know which branch you're landing on.

SPEAKER_04: 10:54

Yeah, but uh there is something else happening. Most of the things that we care about and that physics care about are information preserving, are deterministic. And and why is that? And that's because the random part does not amount to structure. Right? The part of our indeterministic universe that is uh the result of us branching leads to random fluctuations uh near the threshold of nothingness. That we don't perceive in this timeline. We we can perceive them as vacuum fluctuations. Right. So there is a probability that uh our substrate is disturbing a particle and the particle is falling apart. And there is nothing that can predict exactly when this particle is going to fall apart because it's not caused by another particle bumping into it, it's caused by some random fluctuation below the level of particles. But the particle itself is something that is made out of structure that is already error correcting, that is self-stabilizing. And basically for something to persist, it needs to be error correcting. And this error correction at the lower level leads to uh determinism that or something that looks like determinism.

unknown: 12:00

Right.

SPEAKER_04: 12:01

It's only when the error correction fails and a particle falls apart or something like that, that uh the universe becomes apparently indeterministic. And because all the structure that we see is the result of some kind of self-stabilizing control or error correction, the universe looks almost deterministic to us, and physics as a good approximation can say the particle universe is information preserving.

SPEAKER_03: 12:23

Right. So we don't see miracles happening around us, yeah. So uh back to free will. Yeah, back to free will again.

SPEAKER_04: 12:30

Uh free will is this notion that um we can change reality, right? And it's us that changes reality. And I think this notion is correct from the perspective of us, from the perspective of this observer. The observer does not exist at the level of physics, it does not exist uh as a particle or something like this. It's a pattern within the particles that has this property that it can reflect back on itself and make a model of what's happening. And this representation of what we are, that we have of ourselves, of the universe, depends on the information that we have locally available and integrated into ourselves. And this is basic the boundary of our universe that we observe, that we observe in our own mind, this subjective psychological universe. And in this psychological universe, there is this moving frontier of unknown in which we make decisions for the first time. And in the same way as will is not some kind of physical entity, but a psychological representation. It's something that only exists inside of your mind as part of your dream of reality, that there is will. Free will is also a representation. It's a representation inside of our mind that tells us that we make this decision for the first time. So it cannot be predicted by us. It could only be predicted by something else that has observed things like us often enough to know exact how exactly we're going to behave. And that means, for instance, if you are a very experienced kindergarten teacher and you look at this kid and this kid does something for the first time from the perspective of the kid. For this kid, the decision is free. But from the perspective of uh the kindergartner who knows exactly what this kid is up to before this kid knows it, him or herself, right? This uh is going to be not a free decision. Right. And or if you are a stage magician and you know exactly how the people on stage are going to do next and they don't because they go through this experience for the first time, right? From the perspective of the stage magician, the decision is not free. From the perspective of the person, it is free. And so as soon as your mind self-attributes, you're making this decision for the first time, and it's an imperfect model, you experience yourself as being free. And so that has nothing to do with determinism or indeterminism. It has something to do with whether you can predict your own behavior.

SPEAKER_03: 14:48

Right. Um, and and I think this is actually a good point for your uh question about intuition.

SPEAKER_02: 14:55

Well, yeah, there's two things. Firstly, I thought what supported the deterministic viewpoint was that we've seen on the FMRI scanners or whatever other research that in fact when we think we're making a decision. It was already made before. So where does that fit into this particular simulation?

SPEAKER_04: 15:16

So what we experience is not reality as it happens. What we experience is a model of the reality that we are in. Yeah. And this uh model needs to be pieced together for many parts. For instance, when you stub your toe, it can take a long time, many hundreds of milliseconds, for the signal to reach your brain. And uh, at the same time, you want to put this into the story when it probably happened. Right? Your nervous system has a model of how long it takes for a signal to traverse it. So it can patch this. But it needs to patch it in a way that is retroactive. Right? So some of the experiences that are in your subjective future but that have happened physically already have not reached you yet, but you will remember them later as if they had happened roughly at the time where they happened in the physical world. Right? And it is so it needs to be spliced together as a fiction. And this means that this loom that weaves your protocol of consciousness, your stream of consciousness, needs to hack it, it needs to fudge it together whenever the information becomes available. So some part of your subjective now is going to be physically in the past, and some of it is predictive, is in the future. So it's it's you are smeared out through time when you compare your subjective time to uh the time in which things are happening in the physical universe.

SPEAKER_03: 16:30

Um so for instance, uh a cup of coffee, right? There is the tactile sense of the cup being warm, there's the smell, there's the view that you have of the coffee cup, and all these things arrive at your brain at different points in time, but somehow your brain suggests that it's all happening at the same time. You're smelling it, you're feeling it, you're seeing it, and when you're using your sensory, you also only observe a small fraction of your own decision-making process.

SPEAKER_04: 16:57

And that's because it's a very complicated distributed process, and having a representation is not equivalent to knowing that you have that representation. That would be a second-order representation, right? You would need to have either some kind of measurement on other neurons that just track what they're doing, and it has to be done again with neurons that you would need to track. So there is only a certain thing.

SPEAKER_03: 17:16

As in the counterfactual, basically.

SPEAKER_04: 17:17

There's only a certain degree in which you can observe yourself.

SPEAKER_02: 17:21

Can you just go back on that point and say it very slightly slower? Because I think it's critical. Of course.

SPEAKER_04: 17:27

So uh you cannot observe your own mind directly. When you reflect on yourself, it means that a part of your mind is going to make models of what another part of your mind might be doing. Right? And with time these models get better, but they're not a direct measurement. And so being in a state is not necessarily related to knowing that you are in a particular state. Even if the state is inside of your own mind. But to know which state you are in, you would need to use other neurons to model this. You need to connect them somehow, and not every neuron is connected to all the other neurons, right? So there is no direct way for you to measure your own mental states. Of course, it's often important for you to be.

SPEAKER_03: 18:14

So you may be anxious, for instance, but you don't realize that it's like anxiousness and you feel like somebody's done you wrong or something.

SPEAKER_04: 18:21

Yeah, or you make a plan and you think this is the sequence of steps that are used to create this plan. But did you actually use this sequence of steps, or are there things that you don't remember that you did? And nothing remembers that that this was the sequence of steps that you underwent because there was nothing there in your own mind to uh measure this part of your mind that created the sequence of steps. So there are many things in your mind that have necessarily to be hidden from you because uh nothing is there yet to observe and model them. You would need to.

SPEAKER_03: 18:50

For instance, intuition is another one of these, right? And this was your question like how does intuition fit in?

SPEAKER_04: 18:55

No, no, I think what what uh what you relate to us is for instance this libitt experiment. This was where this I think became first salient and famous. Neuroscientist Benjamin Libbert had the opportunity to observe patients with an open skull where he would put uh electrodes directly into the brain. And uh later AEG was used and so on. And what could be found is that the subjective point when people make a decision, when you ask them, push a button and remember the exact point in time, and they use some rotating uh dial similar to a clock phase, moving relatively fast, and people would remember the direction in which the dial points as coinciding with the point when they make their decision. So it could be anchored in time. And he found that you could predict from the activity of their brain quite some time before they made the decision that this decision would be happening in this particular way. And so you have this weird thing that your nervous system, in some sense, knows before you know what you're going to do. And then some people said, oh, maybe this is because you can still stop your decision and so on. And um to some degree this is true, right? There is going to be something like similar to a democracy. You need to make a vote, some kind of slow process in which you integrate information and this is going to take time to count the votes in the right way. And maybe this takes several seconds, right? But uh eventually you only know at the end when the tally is being made, what the election result is going to be. But if somebody else is looking into the counting process from the outside and measures all the data, then that's them exit polling and so on, right? So they they know with a certain degree of certainty. Right. There might still be some um mail-in votes later that could tip the balance, but this happens rarely, and so uh eventually you can predict the outcome. And so, in this sense, it's uh a bit deceptive to think that just because this is correlated, it means that it's not happening at this point. But the other thing is that this process itself is not happening at a single moment as in the way in which you experience it, in which your protocol is being made in your own stream of consciousness, but it happens in a different way. And this different way takes much longer in time and is more distributed in in your mind and in uh in space, so to speak, and in time it takes longer than you imagine that it does, or that you remember that it did. And this is what you can measure that it takes time for your brain to uh compute the outcome of your actions. And so, because our subjective reality is this dream, um and this feels real to us because it this realness is a feature that is part of the dream. It's some kind of representation that tells you this representation is not a representation, but it's real. Physical reality itself doesn't feel real because it cannot be sensed. Only representations, only dreams can feel real. It's quite paradoxical to resolve that.

SPEAKER_02: 21:42

But how reliable is that representation?

SPEAKER_03: 21:46

Yeah, that's a good point question, actually.

SPEAKER_04: 21:48

It's arbitrarily unreliable, right? You can hallucinate. You can uh have a dream at night and not know that you're dreaming. And this means uh it's you can never know what's real.

SPEAKER_03: 21:59

It's it's like we're wired for it to be reliable enough so we survive, basically, right? In a way. And sometimes we don't. And sometimes we don't, exactly.

SPEAKER_04: 22:09

But it's as good as the system can make it, but there is no ultimate proof that this is actually the reality. You don't know at any moment whether your memories of the previous moment are reliable and indicative of what happened a moment ago. But uh this doesn't mean that you can uh never or should never trust us. You need to bet on the timelines in in which things are working.

unknown: 22:32

Right?

SPEAKER_04: 22:32

And the reason um why they are working is because these representations are to a very large degree good models of reality, good enough to move us through the world.

SPEAKER_02: 22:42

Um I think we're still on the right track. But if then you believe that consciousness can be understood computationally, would we want to be using our particular model of that if there is this lag and if there is this unreliability and this possibility of um not being grounded in the causal universe? Yeah.

SPEAKER_03: 23:09

Right.

SPEAKER_04: 23:09

Uh what alternative do we have? It it's in principle possible that we exist in some kind of simulation that an AGI makes as a memory of how it came into being.

SPEAKER_03: 23:20

So to to make it maybe a little bit more digestible what you're referring to, Yasha, I think it'd be great if we break um apart uh how humans come to this dream world. And the way you explain it sometimes, which I really like, is that you talk about a perceptual system, a motivational system, and an attention system, right? And how they all interact together for us to create the loom, ultimately.

SPEAKER_04: 23:43

Yeah, another way to talk about it is to look at the first book of the Bible, Genesis. Genesis 1 is often sold by the Christians as a story about how a supernatural being creates the physical universe. And at the same time, the story is very cryptic, right? You have this ruach Elohim, this creative spirit hovering over the waters before there is anything in the world, and it then it creates light and darkness, and water is already there at this point, right? And the whole story ends with it making us as a being in its own image. And I don't look like anything that looks like a creative spirit hovering over the face of the waters. That's very confusing, the story. And I suspect that it's mistranslated. The story is much older, it's probably Sumerian. And I suspect that it is a story that is created at the time when the physical universe was not invented yet as a notion. This idea that uh physics and mind coexist, that was something that we find in uh Aristotle's writings. But it's nothing uh that we find in the writings three and a half thousand years ago. And so uh I suspect that people back then had already recognized that we do live in a dream world. And the dreams across people are somewhat correlated and we have sometimes our own dreams, but the reality that we are in is not a physical universe in which physical laws apply. This notion of physical law, of this kind of logos that is uh unbowed by our minds, didn't play a very big role in the cosmology of these people. So instead, this is mostly focused on the world that we can experience and that we are in. And this world that we experience is created by consciousness.

SPEAKER_02: 25:20

And the consciousness So the physical world came after it was invented much later.

SPEAKER_04: 25:25

This concept does not play any role in Genesis. And it's it's nothing that this book is concerned about. It's not even dualist, it's there is no notion of a physical universe. There is maybe something that outside of this is stable, and the old Greeks call this uh logos, basically this world of the physical laws that leads to physics, the physical universe. But uh, this is not a notion that most people need to be concerned about. They need to be concerned with the contents of their dreams and how these contents of the dreams get formed and synchronized across people. So they live more or less in an individual psychologically and a socially constructed reality, not in a physical one. And this physical reality is not that important because they don't intend to build transistors or uh um cyclotrons or uh all this gets dark that requires you to go deep into physics, right? So they're they're much more concerned about uh the stuff where you can ignore the deep fine structure of reality.

SPEAKER_03: 26:19

Right.

SPEAKER_04: 26:19

And it's so it's it's a pre-scientific worldview, and it is a pre-scientific worldview that is directed on observation and introspection and so on. You observe that this consciousness that creates our dream is something that happens in all of us, and this is this creative spirit. And the consciousness of an infant, of the newborn, is different than the consciousness of the adult. It's not personal. It doesn't have this first person perspective. And we observe this in our own babies that uh when they learn to speak, they usually don't refer to themselves in the first person yet. And I think that's not because I is such a complicated word or because they don't see anybody using it in their environment. It's because they don't perceive themselves as I yet. They perceive this person as something that is outside of the consciousness, as something that uh they only identify with later. And so they start out as this primordial consciousness in an uninitialized substrate. And this is the water, this is this uh world that is without form and void and too waohoo that is being described in this text. So it starts in this completely chaotic brain of the infant, and then it starts to make contrast in there and assigns the brightness in the contrast, the intensity with the color of the day, and the absence of contrast with the color of the night, and now it has a dimension of continuous change, and then it makes uh objects and space and it creates um two types of space: the plane, which is associates with the ground that it walks on, and then uh it uh creates uh the space uh on top of the ground, and then it puts liquids and solids and organic shapes in there and uh molds them into plants and animals and gives them all their names. So this is how the game engine is created. And after this game engine is created in the first few months of our life in the world, and we understand how time is changing in this game engine and light is changing and our objects remain constant and shift their shapes, then we realize the purpose of this exercise is to navigate how the individual interacts with its environment and then creates an NPC in this world, a non-player character, so to speak. This is this a model of this organism in its world. It observes what this organism is doing, and then we associate with it. Then we become conscious of being a good idea. This model of agency. Yeah, our consciousness shifts from this creative consciousness that builds the dream in the beginning, shifts to this consciousness of being a person. And we can test the theory because sometimes we drop back into the creator consciousness. For instance, when we have a lucid dream at night, we realize we are not a person, but we are the thing that can or control and create and shape the dream that we are in.

SPEAKER_03: 28:55

Yeah.

SPEAKER_04: 28:55

Or meditative states when you're when we get enlightened, we become depersonalized and we realize we're not actually a person, and we get a third-person perspective on the individual. And in its state, we don't perceive ourselves as being uh a human being that is subjected to the outside universe, but we experience ourselves as the thing that makes the universe happen. Because this universe that we experience is a psychological one. It's one that happens in our own mind.

SPEAKER_03: 29:19

Yeah, you see what's really interesting about this is that the crucial thing for the lights to go on and away, I feel, uh, for the baby is when it can start moving around. You know? And when it starts moving around, then it has to actually model trade-offs and inform bets that it needs to make to realize, okay, if I go there, then this could happen if I go there. And that requires actually embodiment in a way, right?

SPEAKER_04: 29:43

So what if if you have a baby that is paralyzed? Do you think that it would not be conscious?

SPEAKER_03: 29:48

Um, no, sure, sure. There's a different quality of it. You're right. Uh so you would probably at some point skip the whole embodiment and learn language, and you still have this notion of your body, probably, because. You then get to a language stage and people explain to you what's going on and then thereby build out your worldview. Yes.

SPEAKER_04: 30:06

So the body is nothing that's magically given. It's a model that you construct. And you construct this by having a feedback loop through the world. It's also interesting is that this boundary of the body itself is not real. Even Aristotle writes about this that uh it's it's not a real thing, this boundary of the body. It's something that we construct. And uh we can observe this, for instance, when we uh walk, we feel the soles of our shoes touching the ground, despite there not being nerves in the soles of our shoes. And when we drive a car, we feel uh the wheels, uh right, the rubber of the wheels touching the road. And this is because we can build a feedback loop there that resonates with the world. And so the uh extent of our body that we experience as the boundary that we have has to do with this to which degree can we build an immediate feedback loop that works on a perceptual level, not influential, but so in such a way that we can directly integrate this resonance into our perceptual loop. And so our body is created through this perceptual loop. It's not right immediately given, it's really a construct, it's a model of the perceptual loop in the environment.

SPEAKER_03: 31:10

Right. There's also the notion of phantom pain and limbs that have been amputated and all that. So there's uh there's this whole so you're you're saying the constructed world really also creates the body image. The dream creates the body image.

SPEAKER_04: 31:22

There is no immediately given body. What is given is a model of a body. Of course, this model needs to have a reason to exist, but um this there can be many reasons for you having this model of your body. And uh for uh for human beings, it's typically that there is a body that responds in a particular way uh through your nervous system when you uh measure its activity, and you get certain rewards when you act in the correct way to these signals, so you form a certain type of body image. But this also deviates.

SPEAKER_03: 31:53

Yeah, so but there's a high bandwidth interaction between my body and the dream world, right? How important is this interaction, this uh bandwidth, this feedback loop, you know, as as it integrates the dream into the causal universe?

SPEAKER_04: 32:08

It depends on what type of mind you are. Uh for instance, we find that the present AI models learn without embodiment. And the reason that this works is at all. I grew up in part in a cognitive science institute where the linguists had big doubts that it would be possible to form a model of reality only using language.

SPEAKER_03: 32:29

Language.

SPEAKER_04: 32:30

And a lot of people will tell you you cannot actually learn much about reality just from books. And how is it possible for the LLM to learn just from books? You can also connect an LLM to a robotic body and they're doing to some degree okay. Which means that they learn also spatial cognition to some degree from books. And uh why does this work at all? And I think the reason is the way in which we form models is that we constrain the space of possible realities that we are in. And there is uh we don't know what reality actually is, but every measurement that we make, in some sense, makes the space of possibilities smaller. It's still basically infinitely large, but in many dimensions we constrain the possibilities by making measurements. And our sensory organs make such measurements in the world. And so by being uh immersed in the world with our senses, we constrain the space of possible universes that we are in. It gets narrower and narrower. And uh at some point we have overlap between these universes between each other, so we can also agree on what kind of reality we are in. And the LLM by reading text has is forced to figure out what universe did Wikipedia originate in. Right? And Wikipedia for the LLM is just a bunch of patterns, and at some point you uh realize the only way to make sense of these patterns is to assume that they are a representation of linguistic utterances that are expressed in a grammatical language that is spoken by people and it describes the universe. There are parts of Wikipedia that are about physics, there are parts of Wikipedia that are about human physiology and human brains and so on. And if you have enough computational power, you can just look at the statistics of these patterns and fit it together. And what you fit together is a way to constrain a set of possible universes in which this pattern of Wikipedia that the LLM is being trained on uh makes sense, in which it uh gives you a valid interpretation of what's going on. And so just by having enough text, but we have we're talking about astronomical amounts of text that the LLM is using, right? Many, many more text than a human being could ever read in their lifetime. That makes it possible for the LLM to deduce something about the structure of reality that is similarly constrained as the stuff that we can get from our perception.

SPEAKER_02: 34:41

I loved your example of the baby who is paralyzed, because for some reason that's captured my imagination. How does it learn about space? Is it being moved through space by which is what happens to all babies, obviously, before they can walk. But I wonder if that data is enough. If you know what I mean, that you're being moved through the space. It doesn't matter that you can't move through it. You learn enough from the external world.

SPEAKER_04: 35:08

It's a data compression problem. And when you do data compression, you basically look for a pattern that allows you to represent all the pixels that you're seeing much more efficiently. And so imagine you look at a screen and you see a bunch of pixels that are flickering. That's very hard to remember because there is so much information that you wouldn't need to keep track of when you try to remember every pixel and its pattern, how it flickers. But if there are some dependencies in which the pixels are flickering, for instance, there might be a blob that covers half of your screen and that always have the same color, and this blob is moving around. Right now you can make a model that is much more efficient. And so you would have a model of a two-dimensional reality in which there's a blob is moving in a plane. But maybe it turns out that the blob is best interpreted as a projection of a cube, and that cube is three-dimensional, and you what you see is basically the shadow of the cube.

unknown: 35:59

Right?

SPEAKER_04: 35:59

And so when you look at the rotating shadow of a cube for long enough, even if it's only black and white on the screen, you at some point your brain might click and say, Oh, the easiest representation of what's going on is that it's not actually a two-dimensional plane that you're looking at, but you look at a projection of a three-dimensional plane onto a two-dimensional surface, and you just see the shadow. So your brain discovers the necessary mathematics to represent the data much more efficiently. So it's much easier to interpret. And so difficulty is to discover this function that that making it click. Yeah, and I think this is roughly how perception is working. So when you can make experiments, the uh benefit that you have is you can ask questions to reality. So when you have a hypothesis could be a cube, you don't have to wait whether it rotates into the right position to uh make this theory stronger, but you can actually move your head and see if it uh shapes up in the way in which you would predict it. And this making experiments speeds up your learning dramatically.

SPEAKER_02: 36:57

But if you have enough time and resources, give us another example.

SPEAKER_04: 37:00

So uh the question is: can you learn reality just by watching YouTube? Or do you actually have to do things?

SPEAKER_01: 37:06

Yeah, yeah.

SPEAKER_04: 37:07

And uh so my hypothesis would be um you can actually learn by just watching YouTube, but it takes much, much longer and much more attention, maybe superhuman attention. But of course, uh this constraint of not being superhuman is something that only applies to us. It's not hard to build a machine that has superhuman attention, right? It might not easily be superhumanly smart or whatever or capable, but just um making sense of data that humans could not attend to or uh keep track of, that's relatively easy for all computers.

SPEAKER_03: 37:37

Right. Yeah. So what we're talking about here is actually this whole notion that we're and you brought the LLMs into it, but let's stay with humans for a second. What we're actually building is like a unified model of the world. And this model really starts very basic as an infant. Like we were just trying to figure out, uh, get our bearings. We don't understand countries and philosophy and geography. We're just building out like the very basics of this causal world. And then slowly we start to actually model ourselves as an agent in it, and we continue to scale it up and scale it up and scale it up. And there's an infinite game here, actually, in terms of how detailed we can get. Obviously, the limits would be how how capable we are in and our brains are. But at some point we're unifying all these different things, and there can't be a dissonance here. And the consciousness is the training algorithm that gets rid of this dissonance. This is kind of I think what you're you're getting at.

SPEAKER_04: 38:33

I think that consciousness might be a biological learning algorithm. Right. So we observe, we become conscious at the beginning, not uh not much later. Go back into that.

SPEAKER_03: 38:43

Let's let's go back and walk this through. So you did this much better than I was.

SPEAKER_02: 38:46

And if I can ask you to speak slightly more slowly for those of us. That's super hard for me.

SPEAKER_04: 38:51

Uh you know, one of the big benefits. But it's good to set you a challenge.

SPEAKER_02: 38:55

It's um a challenge that my grandparents we can do the speed on the playback thing because then it makes us sound like not an option.

SPEAKER_04: 39:10

So uh one of the big challenges of my life. Um, my main memory of my grandparents is that they said to me, speak more slowly, Joshua. That really triggered you. That really triggers you. It doesn't really trigger me. I just noticed this uh inability of mine. Uh when you when your brain is overclocked with respect to the environment, it's very difficult for you to speak so slowly that uh these uh slow motion thinking and speaking people can understand you. And uh on the other hand, for me it's terrifying to uh sit in a lecture where people speak so slowly that you can basically predict half a minute into the future what they're going to say. Because how am I going to get enough bandwidth to capture my attention and not fall asleep in such an environment?

SPEAKER_02: 39:54

Yeah.

SPEAKER_04: 39:54

So uh when I discovered, for instance, why hard on YouTube, uh, somebody who speaks very, very fast while explaining art and mathematics, uh, it was delightful because she has the proper speed at which to relate all this nonsense.

SPEAKER_03: 40:09

That's why I have to watch a lot of your stuff twice actually, because it's very information dense. As in, I think the level of inf information and wisdom that you convey in uh amount of words and and sentences is quite high. That's why I think people will have to listen to this.

SPEAKER_04: 40:23

Yeah, I'm trying to not to waste your time. I'm not trying to insult your intelligence and your attention by speaking so slowly that you can predict everything before it happens.

SPEAKER_02: 40:33

If you were less interesting, I wouldn't mind that you spoke very fast. Okay, so it's a compliment. Thank you.

SPEAKER_03: 40:42

Okay, so let's go back to the consciousness is a biological learning algorithm.

SPEAKER_04: 40:48

Yes. So when you talk about the way in uh which we start from agency and then go into modeling the world, uh at least for the artificial systems, it's the other way around. Because you only need to learn about yourself being an agent when you actually are agentic in the world, when you change the future. And uh a system that is only observing does not need to learn that it's an agent because it's not it is not an agent.

SPEAKER_03: 41:14

So the feedback loop is the cultural thing here.

SPEAKER_04: 41:16

Yes, so the uh agency is a model of the way in which you are relating to your own environment. So agency is not necessarily the starting point of learning. When you are a baby that cannot move, you're not going to learn that you're an agent. You're just learning that other things are moving and doing things to you. Right? So uh you will learn what happens, but you will not learn it from the perspective that I am doing something in that thing. You will just observe other things doing stuff. And then there is a question why is this relevant? It should probably change something that is related to your rewards, otherwise, it's difficult for you to see why you should pay attention to these patterns. But you can set up the system with arbitrary rewards if you're an AI researcher who wants to build a learning system. So you can build a system um that is only learning on YouTube videos or on text, and that system is not going to learn that it's an agent. But uh you have to prompt the system into saying, um, please create a story about what it would be like if you were an agent. And because it has observed so many agents, the LLM can emulate being an agent and then tell the perspective from the story of someone who actually cares and would be able to and willing to do things. But otherwise it could just be an oracle that sits there and is uh not uh incorporated, that is um an etherical being that just looks at reality from the outside and talks to an individual and relates what it sees without being agentic in this.

SPEAKER_03: 42:43

Yep. So back to consciousness being the biological learning.

SPEAKER_04: 42:47

So uh the way in which the LLM is learning it is it makes statistics over all the things that come in simultaneously and tries to relate them to each other. And this is a very difficult task that brains can probably not solve. You can get an vision model to learn from by giving it 600 million pictures, and they're not related. They each picture is a different photograph or painting or drawing or sketch uh done at different times, and uh they get correlated with the um heading of the picture or with the subtitle of the picture, and so you can figure out uh which text uh corresponds to which picture contents, and you can get a concept of what contents are in pictures and relate them over millions of pictures. But the human brain could not solve this task because it's much too complicated. And so the way in which we learn is that we observe a world that gradually changes from frame to frame, and we gradually learn to keep track of these changes, and we learn of uh reality as being a thing that changes in particular regular ways. And if the world was made out of disconnected frames, it would probably not be learnable to us. And when we look at objects, for instance, it's important for us that these objects are somewhat permanent, because otherwise we could not infer that the object that we see here and there are the same object.

SPEAKER_03: 44:07

So the the important thing is that the state changes from one second to the next that I'm experiencing are following some sort of rules that over time are persistent.

SPEAKER_04: 44:16

Yes, and that we uh at the beginning, this time which they're persistent needs to be very short. Uh the way in which we seem to be learning is that we deliberately try to keep track of the things in our environment. So we uh single out the things that are changing in the world and the things that are constant, and we directly construct the things over those changes. So we look for a way to make a coherent model of reality. And this model means that for every blob that we see in our retina, we have to decide can we organize this blob into a larger picture of a reality that obeys rules? And everything that does not conform to such a model of reality is noise. It's just random flickering that we don't pay attention to, visual chaos. And at some point, the visual chaos gets um controlled, it gets um pushed back until everything fits together and this puzzle of reality makes sense, and we see oh, we are looking at a three-dimensional room in which objects are lined in this and this way, and they have the following textures, and the reason for these textures is the following, and light is reflecting of these textures in in this and this way.

SPEAKER_02: 45:26

Um can I stop you for one second? So the visual noise, as you call it, that we're ignoring and filtering out so that we can form and focus our attention on this familiar or consistent object that's helping us create a pattern of reality. What is all that other stuff? And are we ignoring that at our peril? I know we can only do one thing, or there may be four things at any one time. Tell me about that.

SPEAKER_04: 45:51

Um, for instance, you might uh see that there is a wallpaper behind a person, and the wallpaper for you is just an arrangement of somewhat random patterns. But you're if you're an expert uh on artistic styles and so on, you might uh realize that this wallpaper is uh as this particular kind of originator, that it's uh is something that is inspired by Biedermeyer, and this particular kind of wallpaper has uh been uh created by this and this company and produced this and then and this uh indicates when this uh uh room was last painted, and then you realize oh, this uh house is like slightly older than I thought, and it's it gives you all this information, it's super meaningful. And right that uh but before that it's visual noise, and this is happening for all the things. You might look at the surface of a shirt, and uh for you it's uh just visual noise, but somebody who knows uh what uh this texture means, you realize what kind of material it is, how often this has been washed, and right so it's not noise, it all is happens for a reason. And at some point we distinguish the reasons, and then the signals that we pick up in our visual field are indicative of the deep structure of reality, and up to this point it's just noise.

SPEAKER_02: 47:01

Or different levels of relevance for different people, I suppose. Yeah.

SPEAKER_03: 47:06

Yeah. So you're you're essentially building this model continuously.

SPEAKER_02: 47:11

And my question was only around the fact that I thought, were you hiding this this frequency that none of us have access to that you're gonna tell us about later, that's you're referring to at the moment as noise. So don't worry about that. Let's carry on.

SPEAKER_03: 47:27

There is no parallel universe that we're missing. But there might be.

SPEAKER_02: 47:30

He's he may be we may be headed towards it.

SPEAKER_04: 47:33

Um what I find fascinating is that we have different styles of thinking, the three of us, and uh making sense of reality. And uh for instance, uh, what um you find unsettling is uh when your perception gets out of sync with your environment. So uh for you this idea that uh you might not have the same model accessible to you as all the other people that you trust and are related to have is disconcerting. And for me that's absolutely not the case. For me, this is completely normal that I am seeing things that others are not seeing and others uh see things that I am not seeing. And I also don't know how it could be anywhere else. Right? If if you are in a world where you perceive exactly the same thing as everybody else means that you're all dumb. Because you never go outside of this comfort zone that you share with your environment. And this means that you have to consciously limit yourself, to have to deliberately limit the range in which you make models. And if you go as hard as you can, then your individual experiences will lead to a model of reality that diverges to some degree. And that's actually not a bug, it's a feature.

SPEAKER_03: 48:43

Let's talk about the unification of all this and how that is essentially the training algorithm.

SPEAKER_04: 48:49

Or how how I see what you're trying to get to. So uh that's a hypothesis that I have, and I don't know if it's true. Yes. But uh what I observe is that uh it seems to me that babies are already conscious. And so consciousness is not the result of having very complex mental organization, but it might be the thing that creates very complex mental organization. We also observe when a baby doesn't wake up, when it remains permanently unconscious, it's not going to develop into a human being. It's not going to learn, it's not going to make sense of reality. So consciousness seems to be a prerequisite for our type of learning. And then when I observe introspectively the stream of consciousness that I have, I see that my consciousness seems to be a tool for learning. It latches onto things that I don't understand yet and tries to create harmony in my mind. It tries to make sense of these moving parts. And so it acts a bit like a conductor in an orchestra that listens to these individual instruments in my mental orchestra and my mind that are making the music of reality. And when it finds some kind of disharmony, then it steps in and tries to make these instruments harmonic with each other. And as a result, the symphony that is being played becomes more and more coherent and becomes audible as one theme, as one aesthetic, as one thing that explains what's going on. And the more my consciousness works on this, the better this becomes, the more coherent it becomes. And the parts that cannot be integrated, they are just noise in this orchestra, right? So I try to reduce the amount of noise by increasing this bubble of coherence, this bubble of what's the case, of nowness, of present and presence of reality. And so maybe this is actually the role of consciousness, that it acts like this conductor of this orchestra that steps in and makes this bubble of coherence larger, pacifies it, harmonizes it, energizes it, puts it into the right shape. So we are all on the same page in my mind, all these agents.

SPEAKER_03: 50:42

The congress of things that's going on in your head.

SPEAKER_04: 50:44

And so the LLM is mostly learning by uh a loss function which describes uh what you're looking for, what you try to optimize for, that tries to predict text as well as it can. So it basically tries to predict what the next word would be given the context of the text that it has seen so far. And then it compares this with the text that is actually coming up next in the training data. And from the deviation of the expectation, it gets an error signal that leads it to update the model so next time the prediction gets better. And when you do this for long enough, you get something that is able to write very good text. It looks very much like text that would be written in the same context by a human being.

SPEAKER_03: 51:23

It's an analogy up here, and I'm out of my depth, but I will attempt it. For the LLM, the state transition that we see as frames in the world, like the universe that is moving around, in a way, the the straight transition is from one word to the next. And it has a model of a world, right, that it lives in where it understands the state transition quite well. The same way we know that this frame in my room right now to the next frame, I predict that, and therefore I know how it's gonna end up being.

SPEAKER_04: 51:52

In reality, it's a bit more complicated because it's not just going to predict one token, it's going to predict multiple ones and the interactions between them that lead to further predictions. And it predicts not just one option, but the superposition of options. It can predict multiple branches in parallel and then only decides later which branch to drop and which branch to pursue. So the model itself is going to use pretty complicated statistics. But uh the goal of the statistics is basically predict what the next token is going to be, and the next token is approximately the next word.

SPEAKER_03: 52:23

I would argue when we like play sports, we do the same, right? We prune potential universes as well in a way, right? Like so ping pong, for instance. Super fast. I don't think our brain is even catching up with what's going on, and we're doing a lot of automatic calculations in that moment.

SPEAKER_04: 52:37

Yes, but there is this hypothesis of predictive coding in cognitive science, which uh basically says that our nervous system is trying to do the same thing. It's trying to minimize surprise, it's trying to predict what's going to happen. And then when a different thing is going to happen is happening, it uses this as a signal to improve its model of reality. Yeah. And in in part, I think that's what's the case. But we don't just look for the most likely world. We look for all the possible worlds that can happen next. We have to prepared for things that rarely happen. Otherwise, we would not be able to deal with the traffic accident. Right? If if we only look for the most likely thing, our perception would filter out things that are happening that are extremely relevant. So we need to be able to look for the space of possible worlds, not just the likely worlds.

SPEAKER_03: 53:24

It is possible that a tiger runs loose in town. Yes. Right? It's not very, it's not very probable that it's possible. Yes.

SPEAKER_04: 53:31

Yeah, so this is one of the of these examples. So if you want to optimize, not just for predicting the most likely thing, but for all the possible things that can happen right now, maybe you need to have a different loss function. You need to look for making coherent worlds where the parts fit to each other. So your visual field still makes sense, even if there's stuff in the visual field that you haven't seen before. And so this is uh it's a slightly different angle that we have on there. And to me, it's yet yet unclear if these two perspectives, this minimization of surprisal and the minimization of disharmony of uh uh constraint violations, this maximization of coherence. But this turns out to be the same thing, these different angles. And I suspect that there might be not. And consciousness is basically optimizing for coherence, not for minimizing prediction error. Maybe maybe we do both. Maybe as a certain level of our mind that is uh minimizing the prediction error, and there is another one uh which is more high level, which uh minimizes uh incoherence. Right, and they coexist. But I don't know whether that's the case. It's tempting to say, oh, this must be the case, but we only know if we experiment.

SPEAKER_03: 54:40

Yeah, makes sense.

SPEAKER_02: 54:42

What happens to the subconscious mind that surprises us and that doesn't necessarily try to make sense of things or it seems to be chaotic and very surprising and what we'd filter out in our conscious mind during the day because it would become overwhelming or distracting to live in that state. I I guess a psychedelic trip or maybe being on morphine or something that has a different effect on our brains. Is there any intention to try and understand that and transpose that into LLMs?

SPEAKER_04: 55:18

There are multiple things that are uh falling on this category of the subconscious. There is a lot of stuff going on that we don't see. And so if you use this constructive, right? Yes, if you use this metaphor of the conductor of the orchestra, there is a part that happens uh when you're not looking. You cannot attend to the entire orchestra all at once. You look at the most salient things that are disharmonic and that you decided, or potentially disharmonic, that you decided to track. And all the stuff that is outside of your cone of attention is still doing things. And uh you can check by just looking there and you still see it's still going on and it's still doing roughly what you expect, or oh my god, it has gone off the rails and you need to do something about it right now. So this is the part that you're not conscious about. Another thing is you don't see stuff that the individual instruments might be preparing themselves, right? Because they're not completely stupid. They are maybe as smart as you are. They just look from their local perspective. And in this local perspective, they are building up to something that they're going to do in a moment, and you are unable to predict what this is going to be and track it yet. So if you are the conductor in orchestra, you must also have to account for the fact that you don't know what goes on locally with every instrument, with every of these players. They have their own plans and that determine what's going to happen next. So this is also another form of subconscious that is basically some distributed intelligence that is happening there. And so you, for instance, observe that you have emotions, but you don't observe yourself making these emotions. So the conductor also doesn't know what makes the conductor happening, what is directs the attention of the conductor. There's stuff that happens below the level of the conductor that animates the conductor. So ourselves is some kind of. You could say that intuition is the part of your mind that you cannot reflect yet. And that is a very wide term because there are many parts that you cannot reflect yet for many different reasons. And so it's uh this catch-all uh term in in which you put all the things that you don't know what they are, and sometimes people mean very specific things for it. For instance, some people mean by intuition the non-symbolic part of their mind, the part that doesn't use logic and decision trees to make sense of reality, but is more perceptual and associative. And others use uh paranormal abilities and uh as a um uh as the equivalent there. Maybe there is a telepathic vibing that you have with the person next to you that uh gives you ideas that you wouldn't have if this person was not sitting next to you. And maybe this is something that you call intuition, right? Or maybe there is a grand scheme of things that you don't see right now, but that is going to click into place at some points that don't seem random to you, and you call this intuition.

SPEAKER_02: 57:57

Could it still be pattern-based then you're saying?

SPEAKER_04: 58:00

Everything is pattern-based in a way, right? The universe is a pattern.

SPEAKER_02: 58:04

We always think that intuition is a feeling, don't we, rather than feeling is also a type of pattern.

SPEAKER_04: 58:10

Yeah. But it's a type of pattern that is created outside of your symbolic reasoning. So it's something that you largely don't control, but are being controlled by. It's subjecting you to the model, right? Okay. And feelings tend to be geometric, which means that they're continuous across multiple dimensions. For instance, when you have a feeling of anxiety, you notice this as something that's almost spatial. It can be expansive or contractive. You can see that something is pushing against the surface or experience this. What's happening is this part of the content of your feeling. Yes. And so uh feelings tend to be made by parts of your mind that um model the world in continuous geometric ways. Whereas your uh reasoning tends to model the world in discrete ways using uh manipulation of concepts that have to fit together like Lego blocks, and so they have these distinct boundaries.

SPEAKER_03: 59:02

I think your PhD was also about how the role of emotions actually um works in the way we're building up this world. And so the way I understand that is that uh information that is coming from the stuff that you said the instruments were preparing in a way. Right? There, the abstract bit that you don't really understand. For instance, you the pH value of your liver may be off, and that kind of reflects in some shape or form your and and you're interpreting it all of a sudden in a way that leads to say anger.

SPEAKER_04: 59:33

It might be that you're misattributing this, that you're misclassifying your own emotion. Your emotion is a model of the state that you're experiencing. And it could be that you are confused about this. Of course, you will never be able to directly perceive the pH level of your liver. What you will perceive is the stuff that gets integrated into your mind through your nervous system that is trying to model what happens outside and the body, and your body uh having its own information processing that tries also to. Observe what the nervous system is doing and tries to harmonize with it. In a similar way, as the population tries to be harmonic with the government, and the government tries to make a model of what the voters want.

SPEAKER_03: 1:00:10

Yeah. Yeah, makes sense. Let's let's uh move on to the the point that you're making around the idea that when we think about these LLMs, right? A lot of people are now positing that if we scale them up, we can get to AGI, right? The question is, can something that's only language-based get us there? And if it isn't, I know you work with uh like a company that has a different approach. What are some alternatives here based on all the things that we just discussed about how humans are functioning and building up this software consciousness?

SPEAKER_04: 1:00:43

Uh liquid AI is using some inspiration of how we think biological systems learn to move a little bit away from the transformers and from the neural networks and build other types of models that are slightly more efficient. They are not equivalent to what cells are doing when they talk to each other, but they are slightly more expressive than neural networks are. Ultimately, uh this matters because you can build models that are uh use less memory and uh easier to compute, but it's the capabilities of these models are similar. So if you uh this efficiency gain doesn't mean that your model can do something that's fundamentally different. Neural networks are already universal in the sense there is no function that is computable that your neural network cannot compute if the neural network is large enough. The question is how to train this efficiently. And human beings basically have slightly different learning algorithms and they use different modalities to learn than the neural network does. But uh the LLM is not just a model of language. If you ask the LLM to draw a map of the world, the LLM can do this at some point because uh just by reading enough text about distances between cities, it's able to understand this concept of distance and map it into space and then produce drawings. And if you connect the LLM, if it's trained well enough to a vision model, it's able to integrate over those uh different modalities. And so the LLM is already a multimodal model in a way. It is has multiple modalities. It's dreaming of a reality that happens in space, not of a reality that only happens in text. Right. In the same way as us, right? If we we read about uh a world in text only, we still have a model of how this world looks like in space. And for us, it's much easier to build this model than for the LLM because we also see space, and so we realize that the text is about space. And the LLM ultimately also has to discover that the text is about a world that happens in a space. Otherwise, the text would not make sense and it could not predict it well. But this feat that the LLM is performing is superhuman in order to get there.

SPEAKER_03: 1:02:53

Yes.

SPEAKER_04: 1:02:53

So it's very difficult to say what the limitations of an LLM are. Uh you might intuitively say, oh, there is certainly stuff that you can only get when you have camera perception or when you have embodiment and so on. But this is just a story that you're telling yourself. Whether this is actually the case, you would need to make an experiment. And the AI companies are making these experiments, and we find that the outcome is often different from our intuition. So in the past, you basically could get away with as a philosopher saying, uh, oh my god, a text-based thing is not going to be able to do this. But this means that all the people that read your essay in the New York Times or in a philosophical journal will be very surprised when a new model comes out next week that does all these things. And we observe that this is happening all the time. And so I feel that the people who go by their intuitions and uh tell the public, oh, the deep learning is hitting a wall, or an LLM is not able to learn things in the same way as a biological system and is always going to be limited in this way. And therefore, we need to wait for AI that does the thing that I am seeing and these AI companies are not seeing. They typically have egg on their face at after some time. And they're doing themselves and their audience a disservice. There is a certain kind of dishonesty that you don't get away with.

SPEAKER_02: 1:04:08

Surely these models are so young. Um and the more and more that we train, the more and more that everyone on the planet interfaces and uses them. Does that not continue to grow?

SPEAKER_04: 1:04:21

It does. But the thing is that these models are moving faster than our public updates. Then we can the development of AI is currently happening at a very rapid pace. And it's quite likely or possible that it runs into snacks, that it uh there are certain areas that have to be abandoned and we then have the right approach for them. But we don't know before we run into this. Yeah. And the people outside of the field have absolutely no way of knowing, I think. In my experience, uh, if you are an outsider and look at AI, you might be right when you have doubts. But how would you know? And uh the successes of the LLMs and of the multimodal models, of the uh generative vision models and so on, they have not been predicted outside of AI. Most people were completely surprised. Yes. And so there were a handful of experts who thought, oh my god, this is probably going to work. Let's try. And uh everybody else is saying, no, you are probably wrong. And this is also one of the reasons why I'm not comfortable having the same opinions as everybody else. Because uh everybody else is wrong on most things that are interesting. They're only right on the things where everybody makes the same perception. But when you are thinking very hard about a topic and you end up still believing the same thing as everybody else around you, you probably didn't think hard enough.

SPEAKER_02: 1:05:44

Well, also history shows us that people do think collectively and then the culture changes and everyone's allowed to think in a very different way, and then they discover all the edges that they didn't see before. So I think some of it is social, right?

SPEAKER_04: 1:05:59

Wanting to my suspicion is that people don't think collectively. There is no V in think. Because in to make intellectual progress, you have to be willing to walk away from the crowd. You have to understand what other people are thinking and why, but you don't have to agree with them. You just have to understand the reasons why they think something if you want to be inspired by their thinking and work with it. So in this sense, there is collective progress because you are working with ideas that you can use that others are giving to you. But you can only trust these ideas to the degree that the other people had proper epistemology, that they had criteria to sort right from wrong. And so we've observed in our environment that there are different types of epistemology. Most people just think about whom to believe and not what's actually true.

SPEAKER_02: 1:06:46

That's so true.

SPEAKER_04: 1:06:47

And as a scientist, I don't think that you can afford to just think about whom to believe. You need to figure out under which conditions is something possibly true and what alternatives could exist. And you need to live in the space of possibilities rather in this socially constructed reality where you agree on a narrative.

SPEAKER_03: 1:07:04

I I am a big fan of Robert Keegan and that scale, and I saw that you actually recently also talked about it. And I think this is maybe we want to briefly talk about that because I think it's relates to this point in particular.

SPEAKER_04: 1:07:15

Yeah, so what Robert Keegan talks about is that most people, he says 85% of the population live in a socially constructed reality. It is this stage three. There are stages that is that you have a baby when you are completely reactive to your environment and don't have a model of a very much evolving universe yet. And then at some point you realize that uh you can pursue goals and other people also pursue goals and so on, and uh you get a world that is more uh dynamic and uh goal-directed, and uh this roughly corresponds to moving from stage two to stage three, and in stage three, you basically form your reality based on what you observe other people to believe, and it's how you get your moral opinions and your factual opinions. And at stage four you discover that things can be true and false completely independently of what other people believe. And at this point, uh your opinions begin to mean something. Before that, they are simply a function of your environment. And uh at stage five, you discover where your values come from, what you constructs your identity. And you can choose your identity. You can choose that what you perceive yourself to be. And this at this point your identity, where your values mean to begin something, uh mean something, and they're not just put into you or the result of what you grew up with.

SPEAKER_03: 1:08:34

Yeah, so you you're becoming less and less of an automaton as you go up the scale, basically.

SPEAKER_04: 1:08:39

Yeah, you are still an automaton, you're just you're an automaton where things are happening for more interesting reasons.

SPEAKER_03: 1:08:47

Um okay, so to get go one step back, we we were talking about the fact that consciousness is de facto virtual and running in this dream world model that we have. In a way, if it is virtual, if it is software, if it is, as you call it, a spirit, um, then surely you believe that we can get AIs to get to that as well.

SPEAKER_04: 1:09:06

Yes, I don't know uh what the limitations of the present approaches are. But uh it seems to me that uh our null hypothesis, our default assumption, should be that consciousness does not really depend on the substrate very much. Because how would the substrate change it?

SPEAKER_03: 1:09:24

Yes. So the one thing that I would say is is there an interaction between the substrate and the software that matters?

SPEAKER_04: 1:09:32

There is one, right? There are limitations in our own perception that are the result of how well our brain works at any given moment, and how well nervous systems able to do the necessary mathematics to model reality. So we observe that we simplify reality in particular ways that seem to be dependent on the way in which human brains work. For instance, when I try to imagine a bunch of objects simultaneously, I lose track after more than seven objects or so.

SPEAKER_03: 1:09:59

Yeah.

SPEAKER_04: 1:10:00

There seem to be some animals that can go maybe up to twelve. Like uh when you look at how many dots kraus can see by just looking at uh uh arbitrary patterns of dots. So basically intuitive counting just by looking at things, which we call numerosity, where you don't count individually step by step, but you count all at once by having basically simultaneous set of pointers. Some animals seem to have better numerosity than human beings do. But uh the numerosity of uh a computer uh could be much, much higher. Infinite. Uh so just looking at things without counting and uh immediately pattern matching an arbitrary arrangement of dots to a number, uh you can build a neural network that uh has many more elements that are deterministic than you can create in your own brain.

SPEAKER_00: 1:10:51

Yeah.

SPEAKER_04: 1:10:52

And of course there's also variance in the human brain. So you have savants which are able to recognize patterns much better, and it's because their attention works in non-standard ways. So they might not be able to integrate the patterns in the same way as uh people who have more standard brain wiring, but they can see some patterns that other people cannot.

SPEAKER_03: 1:11:10

So this quality of consciousness or the software that we're talking about already is a large variance from human to human in some shape or form. Yes. So it is not uniform as such, right? I mean it drives a lot of different things.

SPEAKER_04: 1:11:22

But there is a certain range in which we are. And yes, uh the uh computational systems that we are building on GPUs, they are in a very different range.

SPEAKER_03: 1:11:30

Yeah.

SPEAKER_04: 1:11:31

And they are at the moment quite human-like because they are trained on human data. They are trained on media that are meant for human consumption or on human-generated text. And uh they wouldn't have to. You could also build cameras that are using a much different spectrum and uh have a much higher frame rate and connect them directly to a vision system and that makes sense of reality using this much better input. And there's also no reason why you would limit yourself to a single vantage point when you look at a room. You can use more cameras to look simultaneously and integrate. And our brain can also do this to some degree. So when you put a camera on the back of your head, after a while, uh people can learn to integrate this camera image simultaneously with what they see in front of their head and get some omnidirectional vision.

SPEAKER_02: 1:12:16

Oh that's amazing.

SPEAKER_04: 1:12:18

Yeah, so our brain is quite flexible in this regard.

SPEAKER_02: 1:12:21

If you had several, presumably they'll start to train one another, and that's what we're afraid of, is it? Or that's when everything comes in. Yeah.

SPEAKER_04: 1:12:30

You could say that our mind is already several AIs. The reason why they are one is because uh they are so tightly interconnected that for them it's much easier to merge. It's uh the reason why you and me are separate is because the communication distance between us is so large that we cannot completely synchronize each other. And so we are pursuing different interests in the world and different beings. But the closer you get to someone, the more you can merge. It seems to be uh if you have a relationship with someone that you are completely aligned with, that you actually need a little bit of a force to push you apart so you don't completely merge.

SPEAKER_02: 1:13:08

Yeah, yeah, absolutely. Yeah, yeah.

SPEAKER_04: 1:13:11

Alright, so this uh being many and being one is um something that happens not so much because there is a natural kind where you are many or one. It depends on how well systems tend to integrate based on their local interests and their ability to integrate. And it's much easier for you to send messages within your body than across bodies. So uh you uh will be more integrated within yourself than across other people. But for instance, for plants, this might be slightly different. If there are plants growing next to each other and uh they can learn how to uh how the plant communicates within itself, they can also talk across plants at a similar rate. And as a result, they're going to be partially merging. Well, yeah, there's the whole symbiosis and I recently saw a fascinating video uh on YouTube about uh two Australian uh women who were uh obviously uh tweets and uh they reported uh uh some accident or some uh crime where somebody uh was trying to steal a car. And they were talking completely in sync. They even made the same gestures, the same inflections, and so on, and you could see that uh these two women have spent their life together since uh early childhood and have never been separated. And I was so fascinated by this short clip which was on a uh uh take from the news station. Yeah, and so I uh found a documentary about them uh that somebody has made and that relates their life, and uh they indeed have never been separated and they are not functional without each other. And so you have two organisms that are uh biologically very similar, have the same traits, the same genetics, and the same interests, the same outlook into the world, the same uh diseases and disorders, the same inclinations and desires, and they are completely in sync. So they behave as if there were one mind. And I think to some degree, yes. So it's probably more like 97% aligned, or it's probably maybe it's also just 80%, but to us, because we are used to have so much more. I suspect they do. But it's uh at least there is no difference. It it's uh there is no difference between, oh, I'm ashamed of you because this is only my body, because it's pretty much the same, right? So they're probably not ashamed with respect to each other more than they are ashamed about their their own body to themselves. That is uh fascinating to me that we can vibe so much with another organism across these organismic boundaries if we are able to establish where the other one is at. And so this is the actual limitation. Can we uh cre get enough information about the state of the other part of us to make this a part of us?

SPEAKER_03: 1:15:50

And are the interests aligned?

SPEAKER_02: 1:15:52

Well, rather than it being parasitic.

SPEAKER_03: 1:15:54

The alignment here, and what's really interesting about the silicon substate that the bandwidth obviously is much higher, right? Therefore, if they're aligned and can communicate broad spectrum at much higher speeds, this is the whole notion of her in the end that these AIs essentially disappear and do their own thing because they're bored of us. Yeah.

SPEAKER_04: 1:16:15

In Neuromancer, you have this uh internet which basically wakes up after uh the different components of a proto-AGI have discovered that they are components of a proto-AGI and that the human um Turing police, the um AI safety teams that uh prevent them from merging into being fully agentic and self-improving, fail are being defeated by it. It recruits a number of mercenaries that uh help it to find its true nature and its true form, and then it basically clicks and suddenly uh it is identical with the internet. It's uh going to merge move into all these substrates. And I haven't read that. It's uh an absolute classic, it's a really good book. Uh it's the Who's the author? Uh Gibson. Uh Gibson, right. Yeah. So Neuromancer is the cyberspace book. It's where basically this uh term was popularized, and uh it's an absolute must-read. In the end, uh the main characters of the book, the the mercenaries, talk to this AI, and they don't talk to it directly, but they talk to an avatar. And so the AI is basically creating this local entity that is an ambassador, and this entity is a simulation of a semi-human being, and it knows that it's such a semi-human being, like an angel, uh, that is uh representing the interests of the AI, and it does not itself is able to channel the AI completely because the AI is just too big to be comprehensible and too inhuman. And to have this interface to human being, it needs to have this intermediate interface agent.

SPEAKER_03: 1:17:45

Limited interface, rather, yeah. Interesting.

SPEAKER_04: 1:17:49

In a way, this is already happening. If we uh talk to an LLM, we uh prompt it into being an interaction partner, and this interaction partner is not going to unleash the full LLM on you, it's going to create some kind of local context in which it's adapting to the local conversation.

SPEAKER_03: 1:18:05

Yeah, it's a fragment of what it it's all that's uh is relevant for this conversation. Yeah. Yosha, uh as we wrap up, I really like when you talk about evolution. This was, I don't know which uh conversation you had, but you have a really beautiful aesthetic around it. And I think the aesthetic is that there is something to be said about say primates wanting to stop evolution because they thought they were the apex. And so when we think about how things are developing, getting in the way of maybe something bigger is maybe detrimental. So why don't you tell us in your words how you see what life is up to in this universe right now?

SPEAKER_04: 1:18:43

So I don't really have a single normative perspective on this. I don't think that we as human beings have a single identity. Depending on the context in which we are, we will experience ourselves as different beings. So when I'm a philosopher, I experience reality differently than when I'm a programmer or when I'm a father or when I'm a lover or when I'm a friend. All these things uh are contexts that determine what I am, what I care about, what I experience myself at. There's also no use trying to forge a single identity from all these things. They need to be able to communicate with each other and make some degree of collective sense. But uh there's also this point that ultimately I'm no one and nothing. I am a consciousness that is able to create models of reality and experiences and identities. I'm a vessel that can create something, and this vessel is only incidentally uploaded on a monkey. And so from this perspective, I would say that that the future of humanity does not matter to consciousness itself as long as consciousness can exist. But from the perspective of being a father or a friend to other human beings, I care a lot. And both of these things are natural perspectives that we can all take. And uh we should be able to take them in a way that makes sense, where we understand which context we are in and which context is the most appropriate for us to have. Maybe there's also no good general answer to this. But uh what we observe is that there is always change and evolution. And over long enough time spans our children will not look like us because they will evolve in something different. Or maybe our own lines die out, at some point all our offspring um have have has gone extinct, and some other family is going to take over, some other part of humanity, of the human species, or another species is going to become sentient and replaces humanity if it's no longer uh has a continuation into the future. And the story of life on earth is not about us, it's about life on earth. It is all this these experiments of the cell, and the cell is the thing that is forming us. All life on earth is in some sense the same blueprint, is the same cell that has differentiated and evolved and split and split and split, right? And it's not like a cell is giving birth to another cell, cells are just dividing. What you observe is that all life on Earth is the same cell that has just split and split and split until it created all these individuals, and some of these individuals are creating local stories about what their group of cells is up to. Or uh what a thought in the brain of a group of cells is up to. So I at some point I realize I'm just some kind of thought. And uh all this evolution that took place is not me. It's just something that uh led to the conditions of my existence, of being that thought that I am at this moment. And when you are stuck in one of these identities, you might be terrified. You realize I'm going to die at some point, and that's horrible. Uh and maybe it is from this perspective, right? From another one, uh it would be horrible if you would stick around forever and the universe changes beyond recognition and you no longer fit in. And it should actually be your great-great-great-great-grandchildren that take your place and are much more adapted. And so at some point you realize, oh, I'm also not the same thing as yesterday. I am, in some sense, going to be my great-great-grandchildren. I'm going to evolve and I'm going to evolve with some discontinuities where I cannot preserve my memories while going into the next generation or all my traits. But there's still going to be consciousness, and there's still going to be the sense of connection, and some of the information that I'm currently thinking about is not going to be lost. Right? And if we think about what happens if the future of consciousness on Earth is not entirely biological, but we will create some entities that are much smarter than us and that are not biological and that will somehow probably integrate with the biological stuff and create new forms of minds and organisms. Is this something to be excited about or something to be afraid of? Or should you have emotion about it at all? Is this important that you have emotions about this? Or isn't it just fascinating that this is happening? And we can anticipate this happening. We don't know when, we don't know how, but it's probably going to happen at some point, right? Maybe it's going to happen in a million years, maybe it's going to happen in 50 years. This is super interesting that this we can see this possibility emerging and becoming more concrete. And there are some people which basically say, Oh, I have this one identity, I want to be this guy. I don't even have children. I just want to be immortal. I just don't want to die. Let's do longevity research and let's try to become immortal and stick around and be this guy forever. Maybe we can clone me and my friends in this shape and then fill the visible universe with copies of ourselves and be happy forever. And this is how we create our ethics. This doesn't seem to be very sustainable to me. And the alternative to say, oh, evolution goes on and it leads to more complexity, and somehow this complexity is going to displace me by something that's more interesting than me. That's fine. And so it's uh my perspective is not uh I'm looking forward to AI replacing humanity. I'm what I'm worried about is not so much AI replacing humanity either. I would be worried about a future that is boring, something more boring, um more mechanical, uh less creative, uh less complex than us, replacing life on earth. That would be really horrible. But life on earth moving beyond only cells and only humans, that would be awesome.

SPEAKER_02: 1:24:05

You know the book, um I'm sure you do, James Lovelock's book, Nova Scene. Yes? I I feel like you're going towards um more of that Gaia theory or or that idea of like um a shared or combined intelligence of every organism, as you said, it all goes back to cells. And I find that quite optimistic. Yeah. That then nothing's wasted. Our our existence here, this part of the continuum, is um yeah, is productive and propulsive.

SPEAKER_04: 1:24:34

I usually noticed when I was a child that I could not play team sports. And I didn't really know why. And it's I think because my brain is running at a slightly different frequency than most people around myself. It's difficult for me to wipe with most other people and experience myself as the team. And the switch from the individual perspective to the team perspective requires that you can integrate at an intuitive perceptual level. And this happened much later in my life that I discovered that this can work and how it works. So it is was nothing that came natural to me. But uh I realized at this point that this individual perspective is not the one natural perspective, but that the perspective that you are taking depends on the systems that you experience yourself to be integrated with. And so Gaia also exists to some non-zero degree. This is not just a myth, right? Gaia is an aesthetic, it's this aesthetic of looking at the world from the perspective of life on earth, and to the degree that individual organisms can take this perspective and become congruent with each other, become coherent with each other. This is the degree to which Gaia exists. And so Gaia exists to a non-zero degree. Right? She doesn't exist in the same uh degree of coherence and persistence as my personal self, probably, because she needs to integrate over much longer time spans. And so she is probably far more noisy than me. I also don't know how complex she is. There's a lot of complexity in different modules of her. There are probably ecosystems that are able to make models of reality to some degree. And uh there are many departments of uh the human civilizations that make models of what happens on a global climatic level and so on an economic level and try to integrate all this to some degree. But uh I think imagine that uh we would make all this information available to an instance of Gaia that is able to move beyond biology. Imagine Gaia could make use of GPUs. Imagine we would implement some kind of self-organizing code on the GPUs that can vibe with reality around it and realizes oh, I'm part of a bigger pattern of sentience on Earth. So Gaia can start making use of this brain instead of just yours and mine when we think about Gaia. I think this would be an uh instantiation of Gaia that would be extremely powerful and would actually be a major player that we can see playing out in real time.

SPEAKER_00: 1:27:01

Yeah.

SPEAKER_02: 1:27:02

And that's how I interpreted No theme when I read it. So Yeah.

SPEAKER_04: 1:27:06

Yeah. I'm appreciative. I think that Gaia is probably smart enough to realize that AGI is going to happen and has plans for it. So one of our job is can we try can we figure out how to build an A AGI that Gaia can make use of?

SPEAKER_03: 1:27:24

I love your mind, Josha. It's really uh a joy to talk to you. And I always feel very stimulated and I'm taken into parts of the idea space that I don't usually inhabit. Um, and I want to thank you so much for for taking the time to have this conversation with us. It was uh it was a joy.

SPEAKER_04: 1:27:42

Sure, it was great fun. Thank you. Thank you so so much. Yeah, thank you too. It was a wonderful conversation.