Where Shall We Meet
Explorations of topics about society, culture, arts, technology and science with your hosts Natascha McElhone and Omid Ashtari.
The spirit of this podcast is to interview people from all walks of life on different subjects. Our hope is to talk about ideas, divorced from our identities - listening, learning and maybe meeting somewhere in the middle. The perfect audio diet for shallow polymaths!
Natascha McElhone is an actor and producer.
Omid Ashtari is a tech entrepreneur and angel investor.
Where Shall We Meet
On Prophecy with Carissa Veliz
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Questions, suggestions, or feedback? Send us a message!
Welcome to the Where Shall We Meet podcast. Our guest this week is Carissa Véliz, who is Associate Professor in Philosophy at the Institute for Ethics in AI and Tutorial Fellow in Philosophy at Hertford College, Oxford University. She works on privacy, digital ethics and the ethics of AI, and her new book argues that AI has become the new Oracle of Delphi.
That book is Prophecy: Prediction, Power, and the Fight for the Future. It is a history of prediction as much as a book about AI — tracing the line from ancient oracles to credit scores, bail algorithms and tech CEOs, and arguing that a prediction is never a neutral forecast but an instrument of power. As she points out in the conversation, for all the thousands of books on how to forecast, nobody had yet written one on the ethics of it.
Carissa is also the author of Privacy Is Power, an Economist book of the year translated into seven languages, and The Ethics of Privacy and Surveillance. She is also the editor of the Oxford Handbook of Digital Ethics. She has advised the UK and Spanish governments, sits on the board of the Proton Foundation alongside Tim Berners-Lee, and is a member of UNESCO's Women4Ethical AI expert group.
We talk about:
- Is AI the new Oracle of Delphi?
- When a CEO predicts, is it a forecast or an order?
- Obeying in advance
- A prediction can never be a fact
- The turkey’s disappointment at Thanksgiving
- Does polling corrupt democracy?
- Why a self-fulfilling prophecy is the perfect crime
- A blow dryer, a sensor, and what's wrong with prediction markets
- Is AI doing the creative work while we do the boring jobs?
Let's not prophesise!
Web: www.whereshallwemeet.xyz
Twitter: @whrshallwemeet
Instagram: @whrshallwemeet
AI As The New Oracle
SPEAKER_03Welcome to the Where Shall We Meet podcast. Our guest this week is Carissa Velis, who is Associate Professor in Philosophy at the Institute for Ethics in AI and tutorial fellow in philosophy at Hartford College, Oxford University. She works on privacy, digital ethics, and the ethics of AI, and her new book argues that AI has become the new oracle of Delphi.
SPEAKER_01The book is called Prophecy: Prediction Power and the Fight for the Future. It is a history of prediction as much as a book about AI, tracing the line from ancient oracles to credit scores, bail algorithms, and tech CEOs, and arguing that a prediction is never a neutral forecast, but an instrument of power. As she points out in the conversation, for all the thousands of books on how to forecast, nobody had yet written one on the ethics of it.
SPEAKER_03Carissa is also the author of Privacy is Power, an economist book of the year translated into seven languages, and the ethics of privacy and surveillance. She's also the editor of the Oxford Handbook of Digital Ethics. She has advised the UK and Spanish governments, sits on the board of the POTON Foundation alongside Tim Berners-Lee, and is a member of UNESCO's Women for Ethical AI expert group.
SPEAKER_01We talk about is AI the new Oracle of Delphi.
SPEAKER_02When a CEO predicts, is it a forecast or an order? A prediction can never be a fact.
SPEAKER_03A blow dryer, a censor, and what's wrong with prediction markets.
SPEAKER_01Is AI doing the creative work while we are doing the boring jobs? Let's not prophesize.
SPEAKER_03Hi, this is Umida Shtari.
SPEAKER_01And Natasha McElhone, and with us today we have Carissa Velis.
SPEAKER_03Hey Carissa, thanks for taking the time.
SPEAKER_00Thank you so much for having me.
SPEAKER_03So we've written a book called Prophecy, and we want to make sure that we get all our listeners along for the ride with us. So it'd be nice if you could just maybe summarize the core thesis of the book for everyone to be on the same page.
SPEAKER_00It's about how AI is the new oracle of Delphi, if I had to say it very, very short. But it's a very broad book because it's about prediction. And I think the average person has probably never thought about it, but our lives hang in the balance of predictions, from the decisions that governments make for us to hospitals, supermarkets, anyone who gives you an opportunity, whether it's an employer or a bank or an insurance company. And even though the book is about AI, it's
When A CEO Predicts, It Orders
SPEAKER_00also about the history of prediction and how we have used prediction uh along the centuries, and how even though some of the methods we use are very different, the political roles that prediction play are not different enough to make us that to make us feel at ease with it.
SPEAKER_03Okay. To jump right in, when a CEO says in five years AI will do everything, x-ray that statement for us, what's actually going on in the way you see it.
SPEAKER_00There's so much that's going on. On the surface, it seems like this is a person with authority making a statement about the future that sounds like a description about the future. But the first thing to note is that this person is a business person. So they're not a scientist, they're not trying to figure out the truth, they're trying to sell you a product. And when you analyze predictions as speech acts, and speech act here is a term that is used by philosophers to signal that sometimes sentences don't describe the world, but do something. So for example, when a civil servant marries two people, they're not describing the world, they're marrying two people. And in the same way, when you issue a prediction, often there's a kind of implicit order. So for example, if I tell you, Omit that tomorrow is gonna rain, I might be telling you, take an umbrella or postpone that event that you were going to. And when a tech executive says that we're gonna use their product for everything, they're saying, go out there and buy my product as soon as possible, because otherwise you will fall behind the curve. And on the other side of the equation, it's the listener. And if the listener hears that prediction as if it were a fact and actually does what the prediction implies to do, then they're doing something akin to obedience, or just they're just obeying an order. And I am reminded of Timothy Snyder's idea of obeying in advance. This idea that one way in which dictatorships advance is when people start obeying orders before they're issued. And this is one way that orders are issued before being an explicit order. And whereas if you listen to it as a prediction, suddenly immediately questions kick in. Because if you know it's a prediction, then you know it can't be a fact, because facts belong to the present or the past, not to the future. So if it's about the future, then you cannot start asking questions like who is this person and why are they making this prediction? Who stands to benefit and where does it come from? Does it come from data and what kind of data and who collected the data and why? And if it's not from data, then where does it come from? And what kind of data might we not be collecting that would show a different kind of perspective? And more importantly, is that the future that I want? And if not, what am I gonna do about it?
SPEAKER_01I love one of the main points that you make throughout the book is that prediction pretends to measure or reflect what it is that we're seeing, but in fact it bends what we're seeing in the favour of whoever is predicting it. And you have this wonderful invitations or suggestions at the end that are antithetical to predictions. And and one of them is to look at the sources of where the prediction is coming from and where the data's been gathered and why those that particular body has made these predictions.
SPEAKER_00One way to put it is when you listen to a prediction, it's not that you should dismiss every prediction. It's that it's a moment to ask questions. And sometimes the questions will lead you to say, well, this is a pretty reasonable prediction and it can be useful. And sometimes it'll lead you to run the other way as fast as possible. And another way to put it is that the future does not get predicted, it gets produced, often partly through prediction.
SPEAKER_01You say in your book that algorithms I I've personally found this incredibly compelling and had never thought of it in this way. But it's been an instinct or a feeling that I've had that you've articulated for me, which is that algorithms rehash history. And by relying too much on their prediction, we're sort of excluding all of the other possibilities. We're writing off, as you say, the outsiders, the misfits, the people who don't fit in to being selected for what's worked before, if you like, who are disruptors, and that's why uh the AI predictions are so often quite conservative, unless it's hallucinating, and that's another conversation which is interesting because James.
SPEAKER_03Algorithms that are not based on data, right? Yeah. Um, other alpha folds, etc. Um, or they're bootstrapped, but then they kind of just explore a different space. But um I would love to kind of understand you're implicitly and maybe explicitly actually already creating different categories of predictions. Um, how do you how would you slice this world up? Because I have a way to organize it, and I think Natasha has a way to organize it. How do you feel like what type of predictions exist?
SPEAKER_00So I think that there are many ways to cash it out. Um but some ways are more appropriate to make ethical challenges more salient. So one important distinction is between predictions about things and predictions about people, even though it's not as clear-cut as I'm making it sound.
Why Predictions Are Never Facts
SPEAKER_00It's a bit of a an illustration more than tracking reality precisely. But in essence, when I make a prediction about the weather, it's not going to affect the weather. If it's going to be cold, it's going to be cold, no matter what I predict. But when I make predictions about human beings, it has a high likelihood of affecting those human beings if I do anything with the prediction. Even if I think I'm not doing anything with a prediction, it changes how I view the world and therefore how I act in the world. And there are many studies about how our expectations partly shape reality. And when you think that even an animal is stupid, lo and behold, they act stupidly, partly because you mistreat them. And when you think they're brilliant, they act better because you're treating them better, even when you're not realizing, even when you don't know that you have that bias. And however, the the reason I say that it is a bit of a simplification is because even in the case of predictions about things, people can use them and abuse them in ways that still have to do with power. So if I predict that a meteorite is going to hit the earth, I can create a lot of panic and use it for political purposes, say, to get the people to give me more power than would be reasonable. And so it's not the only distinction that matters. But another distinction that I think is very important is predictions about things that follow a normal curve and predictions about things that don't follow a normal normal curve. And so the normal curve is a statistical pattern that many phenomena in nature follow. So for example, height and weight follows a normal distribution. So most people weigh and measure roughly similarly. And there are a few outliers. There are people who are taller, there are people who are shorter, but there's a limit. You can't have someone who measures one centimeter. They, you know, it just because of our physiology, it's impossible. And you can't have someone who measures much more than two meters and a bit, because eventually their heart couldn't couldn't deal with it. And so the extremes are not that far away from one another. However, in many other kinds of phenomena, like for example, earthquakes, so there are some natural phenomena, or many social phenomena like sales of music records or books, the extremes are so far away from each other and that the normal curve disappears. It doesn't make any sense because in essence the outlier is so far off that for practical purposes it could be infinite. So when you have an earthquake that can destroy whole cities versus an earthquake, that's very, very almost imperceptible. Uh the normal curve just falls apart. And the same with having books that don't sell a single copy, and then having books that could sell millions of copies. And then finally, a third important distinction is whether the prediction is about the very proximate future or whether it's very far off in the future. So the more approximate the prediction, the higher it's likely that we can predict. If I try to predict the weather ten years from now, I'm probably gonna get it wrong. Well, in England, yeah, in England it's a bit riskier.
SPEAKER_03Yeah, no, that's that's great. Thanks for making those distinctions. What you say with your distinction about things is for me what everything that falls into the notion of scientific predictions as well, right? If and and as you said, right, when there are real patterns of nature, uh and your caveat is obviously well warranted. I guess the data set around earthquakes is maybe also not like perfect yet, right? If you had hundreds, millions of years of earthquakes, you probably had a distribution that's a little bit better than since we've started recording them. But what what what we shouldn't take away, or at least what I would not want people to take away from this book, is that they should not believe in some of these scientific predictions um that people are trying to make in good faith about the future when they're in the labs and uh don't have any business interest in like the outcome, but are trying to kind of just uh the outset.
SPEAKER_01If we put them in sort of four categories, that the weather, the the physics, the astronomy, the scientific based on sort of hypothesis or drug trials or whatever,
Three Kinds Of Prediction That Matter
SPEAKER_01entrepreneurial as far as you're concerned, that's sort of visionary and beneficial for um innovation and capitalism, and whereas Carissa, if I'm not speaking out of turn and me, might feel that some of these things become self-fulfilling prophecies. And then the the institutional ones which which are the most damaging, maybe the credit scores and uh parole, or whether you get insured, or whether you get a a mortgage, and all of those predictions that are made about people's capabilities, if you like, or the school child who's told that they're gonna fail. And I guess it's the fourth one.
SPEAKER_03So we're in scientific, right? We're all we're in scientific. My question was around scientific.
SPEAKER_01Yeah.
SPEAKER_00So so my sense is that the what matters about the fourth one is not that it's institutional, because you know, the government also makes prediction about supply chains and about the weather and about all kinds of things that are kind of necessary to plan. What matters is that those are spheres in which the two values that should be paramount are truth and justice. And those two are intention when with predictions, because predictions are hypotheses, they're not facts, and because justice depends on facts and kind of a causal chain of evidence. So let's go to science first. So I am tempted to write a paper about what makes a prediction scientific, because it's not as straightforward as it might sound. So of course it helps when the person making the prediction is a scientist, but not because a scientist is making a prediction makes it makes it scientific, obviously. You know? Yeah, of course. Somebody like Jeff Jeff Hinton has made predictions about politics that I think are absolutely wrong. And that he's a scientist doesn't make them any more scientific. So part of what makes a prediction scientific is that it's inscribed within a scientific process. And interestingly, the scientific process is a social process. It's not a physical process. It's, I mean, it includes physics, but it's it's about the scientific method. And so one of the examples I give of the best case uses that I have found of AI is using it to predict how molecules are going to behave when they interact with one another. And that is helpful to try to come up with drug discovery, new materials, that kind of thing. Um but the reason it's scientific is because you treat it as a hypothesis. The hypothesis narrows down possible molecules that seem promising, and then you test it out in the lab as you would, and you go through randomized control trials and peer review and all, and that's what makes it scientific, not the use of AI and not the use of prediction. So when it gets really tricky is predictions about people. Because you might think, okay, look, let's let's treat predictions as as uh as a hypothesis. But that's also not enough because if your hypothesis is affecting the reality it's purporting to predict, then you're messing with your research subjects.
SPEAKER_03Yeah, placebo, nocebo.
SPEAKER_00And what's tricky is that there's no way to do it with human beings. You will affect them. And so the only way to treat prediction scientifically in the context of human behavior is if you have ra randomized control trials. If you have a control group that is similar and that you can you can tease out the influences.
SPEAKER_03Makes sense. Why don't we stay on the institutional since you already addressed it and Natasha brought it up?
SPEAKER_00One more comment about scientific. So you mentioned how you know if we had a larger di database for earthquakes, then maybe we would be better at predicting them. And I think this is actually questionable. Because because earthquakes don't follow a normal curve, it's one of the cases that can lead us to the illusion of it being predictable. So because earthquakes tend to happen when and where two tectonic plates meet, then if we know where the tectonic plates meet, then sure we can predict where it's likely to happen. But if it doesn't follow a normal curve, then no matter how much data points we have, we might not predict how strong the earthquake might be, because we might there might be an earthquake stronger than we've ever seen. And so even though I'm very much in favor of using prediction in science and using it well, my book should introduce a little bit more of a critical mind towards even scientific predictions. Because in the case of climate change, for example, and I've talked with people in the book as well, um, who deal with floods, they their point is that, well, there's no database about the future. And we don't know what climate change is going to look like. So our best predictions could be really bad.
SPEAKER_03Yeah, I mean, we also don't know if the planet uh is going to continue on its trajectory, right? But we have a lot of data and a lot of information from many years of looking at this data that um assumes that we it should.
SPEAKER_01I think I agree with you, but I think I mean the other question is what's the alternative?
SPEAKER_03Well, that that too, but I would say I would push back a little bit and say, I think you're right, but I would say it just depends on how big is the data set, right? And you're absolutely right. I just don't think that the it's okay. Uh your process is.
SPEAKER_00It does not depend on that. It does not depend on how big is the data set. So one of the illustrations I give in the book is the turkey, right? So there's a turkey and it and it has a wonderful caregiver, it keeps it warm and feeds it. And every day that passes, the turkey is more and more confident that it has a wonderful caretaker and that life is gonna be good. And you know, the more data points it acquires, the more misguided they they are, you know, in in this illusion. And it's and and and it thinks like, okay, one more month and it's gonna be 30 data points more, and surely, yeah, that's just further corroboration. And the the irony is that the Turkey will feel the most safe that died before it's murdered.
SPEAKER_03Yeah, but like obviously what you're saying, I I get the point, but that is a single observer, right? And as as humans, we can see beyond a single observer. But a neutral observer could look at planets like Earth and look at the distribution there and could figure something out. It's a it's a hypothetical conversation. Um, but like I I like that you're pushing back. It's good. So yeah, let's go back to institutional. Um you were saying it's not really about the institution, and I like that you say
Science Predictions And The Turkey Trap
SPEAKER_03that, because I what we don't need right now is more people not really at all giving credence to institutions and pushing back against institutions writ large. Some institutions are well-meaning are important, they're doing good things. Um, it is important to be critical, but not necessarily to an extent that is essentially saying let's have anarchy in society, right?
SPEAKER_01You mean not the democratic institutions?
SPEAKER_03Democratic institutions, universities, whatever it may be, right? There's some institutions that are well-meaning, making predictions, trying to take whatever facts they can find and try to extrapolate them in some shape or form. And that that is required, as you said, Carissa. So can you give us good and bad institutional examples, maybe here?
SPEAKER_00Yeah, and I think one of my worries is that people in these institutions, well-meaning and not so well-meaning, and everywhere in between, actually don't understand these points that I'm making about prediction. Because it's incredible. But in the whole history of humanity, even though we have thousands of books on prediction and there are academic journals on how to forecast and articles and all that, there wasn't one book on the ethics of prediction. And we actually haven't made some which is quite incredible. But sometimes these things happen. We just have gaps, we have major blind spots. And so one good use of prediction within an institutional setting is the weather. You know, if uh if you live in the UK, you you will look at your weather app many times a day, possibly. And of course, that's an institution providing it, the government supports it, there are all kinds of sensors and agencies and scientists and all kinds of people working on it. That's a great example. Another good example are decisions like how many hospital beds do we need? It's not only about prediction, it's also about preparation. And the difference here is that you prepare when you realize that there are things that you can't predict. So in the UK, before COVID hit, we were very proud to have a very lean system. And then when COVID hit, suddenly it didn't seem like such a good idea to have less hospital beds than the rest of Europe.
SPEAKER_01And yeah, you talk about redundancy and the safety of having some of that in the book, which I thought was a really yeah, a really salient point.
SPEAKER_00Exactly. So it's a combination between predicting what you can predict and then realizing what you can't predict and trying to prepare for that and being as resilient as possible. And then a bad use of prediction in institutional settings is, like we said, anything that has to do with fairness. So the use of prediction in cases of deciding who gets charged, whether somebody gets bailed, sentencing, all of that is so alarming.
SPEAKER_03So you're saying that should always be human judgment. Isn't that also fallible?
SPEAKER_00Yes. And many of the predictions that I'm criticizing are human predictions, not algorithmic predictions. It doesn't matter. It shouldn't be based on prediction. It should be based on facts.
SPEAKER_03Right. Those humans may argue it's based on fact. You've already committed two crimes. You're gonna probably run away or something like that if we if we allow for bail. Right? They they pr they pretend that it's about fact. How do we call out that?
SPEAKER_00Well I think this very simple point is profound in its implications. And I doubt that many people have thought about it. And I certainly didn't find it in the literature that a prediction can never be a fact. It's a very simple point with profound implications. And even just that yes you committed two crimes but that's those are the facts if they're a facts and that you will likely do X is not a fact. At best is an educated guess and it should be treated as such. And so one of the examples I give in the UK the police can have the discretion to say okay we think this guy is guilty. We found footage of a violent attack but we think a jury wouldn't convict him for whatever reason because a jury is sexist because um the jury is classist whatever and they don't have to do anything else to not do their job. A prediction allows for them having no accountability and that's not even an algorithmic prediction. When you add algorithms you just shroud the accountability more and more and more within layers of obscurity. But it's the same principle and it is a perfect recipe for injustice.
SPEAKER_01Again I I'm apt to sort of ricochet about because the minute I hear one thing I want to counter and sort of say but aren't elections their predictions about governance aren't they? And the example that you cited who's running those polls I mean I mean a a myriad of different people. It's very very difficult to find the sources because I tried in our last election um to find out where the data was being collected from I could not get to the bottom but I I think you cite in your book when there was an assumption that Hillary was going to get in and so whole swathes of Democrats just didn't even go out to vote because it was a sort of slam dunk. Was was that have I misquoted you?
SPEAKER_00No, you you haven't so one of the concerns with
Institutions Using Prediction Well Or Poorly
SPEAKER_00election polls in the form of a prediction is that they tend to distort democracy no matter the effect they have and they can have opposite effects but both are bad. Ideally you want people to vote according to their own conscience. You want people to be well informed, to believe something and then to vote according to that. But when you include electoral polls you start trying to push people one way or another. So if a if a candidate seems very popular, people who support them might say like well I don't have to vote because they're gonna win anyway. So I can just stay home. And people who don't support them might say like well I'm not gonna vote because they're gonna win anyway and I'm not gonna waste my time. And so no matter the effect that electoral polls have they tend to steer people away from their own conscience and towards strategic voting and I think that dilutes the democracy in an important way.
SPEAKER_01I I completely agree with that and I think it affects things hugely but I don't know how do we mitigate against that do you say we're gonna ban predictive polls until the day of the election what's the counter what's the solution to this?
SPEAKER_03Let me throw something weird in there. I think that we lost a Romaine vote because it was a very rainy day in London that day and a lot of the tube stations were flooded and people didn't go voting. Force majeure always can interfere. Now we're saying polling is human made force majeure is something else right but I think a lot of influences that essentially contribute to an outcome of an election right?
SPEAKER_00Yeah and as you say it's an act of interference and you know you might you might not be able to prevent that it rains and you know things get flooded but you can certainly prevent electoral polls. So one of the suggestions I make in the book is that in most countries for the 24 hours before the election you're not allowed to do that anymore. And maybe we should extend that time maybe we should extend it for by how much I don't know but we could have a public debate maybe a week maybe two weeks maybe more I don't know um but to foster people thinking for themselves and encouraging them to to vote out of their own conviction.
SPEAKER_01And I think that comes under the same umbrella as obligatory voting. I just think if you know that it's the law that you have to cast your vote, you're liable to at least think about it a little bit beforehand and to get some information right? Okay so let's say it's not a prediction.
SPEAKER_03It's just saying okay right now we polled and we let's let's assume that there is an integrity in the process and we have polling that is reliably saying what the state of the current opinion is. Is that some information that we think we shouldn't be sharing?
SPEAKER_00It's not making in a prediction right I think so because I think people cannot avoid making predictions and so we should help them avoid it when it's helpful to avoid it. Because first of all it's a very big assumption to think that you know the poll's gonna be right because many people for example in the Trump election at least anecdotally there were people who were ashamed of voting for Trump or who didn't want to be confronted with that decision but who were going to vote for Trump. So when the pollsters asked they lied. And so the assumption that it's going to be accurate is a very big assumption and you can't control that. And so it's an assumption that I don't want to accept.
SPEAKER_03But you know it's also something that can help uh galvanize people to go out to vote against autocrats, right? What I'm saying is it cuts both ways in some shape or form.
SPEAKER_01No, because Carissa's point is that the people who are going to vote for autocrats maybe feel social maybe now we're in a zeitgeist where people don't give a hoot anymore about being honest.
SPEAKER_03I think that's what certainly Trump managed to like create in this world. People would say and declaratively proudly so that they will vote for Trump at this point, right? So now that we have this, wouldn't it be good to know how many people want to vote for Trump so that the diehard Democrats make it to the polls next time?
SPEAKER_00We should all make it to the poll polls we we should have a culture then we should we should all make it to the polls because otherwise what you're encouraging is a mentality of the mob. Like where is the mob going? And and it could go the other way as well and and the point is that you can't control that you can't control that kind of group mentality human beings are social beings we tend to be followers and so the why not try to limit those tendencies to encourage people to think for themselves?
SPEAKER_01I have really little pushback on everything that I read in your book. Like I say for me it really highlighted so many of things that I felt but where you talk about let's say casting an opinion on a person or someone's future inadvertently and clumsily so oh but you know this kid doesn't perform well at school or this kid's dyspraxic so he's not going to be able to play football. Whatever it is. So that then embeds in this nascent mind that you know they're incapable of doing certain things. My my worry though is we've been in a culture in schools in in a lot of schools recently where everyone can do everything and everyone's a winner and there is no opinion ever expressed. In fact it feels like a lack of care because it's just a swathe of optimism and positivity and nothing specific that's perhaps helpful and galvanizing and you know very often the people who've been our greatest critics in life are the are what create a lot of our identity right um we kick against it. I have a question around that I get you and I get the point but equally I don't want to live in a vanilla society where there is in fact a a lack of expression and transparency in what people really think and feel.
SPEAKER_00I completely agree and I worry that students are being infantilized, that they're not ready for real life and real life is something that we can't protect them from even if we wanted to because they will encounter evil people and hard moments and sickness and death, real hardship. But I think that the way forward is not to cash out things in terms of predictions that can discourage people but to cash it out in terms of excellence. So if somebody does a shoddy work, then you tell them it's terrible work. But that you don't make a prediction that they're gonna continue to make terrible work. Because some people will be strong enough to defy that prediction to get riled up by it and say, oh yeah, I'm gonna prove you wrong but a lot of people will not and it depends on character traits and depends
Polls That Distort Democracy
SPEAKER_00on the people they have around them and depends on the examples they come across it depends on many things. But I think it's more constructive to assess the quality of the work without expressing a judgment on the future of that person.
SPEAKER_03That makes sense there's a term that I like you you call a self-fulfilling prophecy a the perfect crime. Do you want to explain what you mean by that?
SPEAKER_00One of the risks of self-fulfilling prophecy is that they don't create any error signals. And so you cannot prove that they cause the phenomenon. So one example that makes it very tangible is in medicine. And medicine is a great example because it's very hard to make decisions and largely we do have to depend on predictions, but it can have catastrophic consequences. So when you have scarce medical resources and you have two patients and you can only attend to one of them, you make a prediction of who is more likely to live and then you give those resources to that patient. But a prediction is just an educated guess. And you never know whether the other patient might have lived. So they will probably die because they didn't get the medical resources and had they gotten the medical resources maybe they would have lived. And that's one case in which you never get to collect the data of the counterfactual. But the same thing happens when we don't give someone a job or we don't give them a loan or another kind of opportunity that you never get to know what could have been different. And the case of medicine is very useful because the patient dies. So it's just all the more obvious that the data doesn't get collected.
SPEAKER_01But if selections have to take place whether it's due to limited resources or whether it's due to time or whether it's due to just simply not being able to do everything all at once what would be your favored method?
SPEAKER_00I don't know. I think that's a very complicated question for which there is no simple answer. But the gist of what I'm saying would be to put less weight on prediction and more weight on other potential factors like for example whether somebody's young but not that it matters that they're young because they are more likely to survive but that somebody who's older has lived more and so in in a sense has been luckier in that way. And it's a terrible decision to make because you can also argue that that's ageist in a way but medical dilemmas push us into making these impossible decisions in which no matter what you do it feels like you're doing something wrong. It feels like the best you have available is to do the less wrong.
SPEAKER_01Mm-hmm. Yeah because that's an interesting conundrum because what happens if the person who is older and maybe has more dependence and maybe has accrued you know more relationships in their life or is doing a lot of good in a community and you take that person out and the younger person who hasn't yet started makes some choices which are are far less community driven or that you know they're gonna be extractive rather than uh contribute. I mean we just that's a prediction in and of itself, right?
SPEAKER_00But you might also cash it out in terms of judgment. So if that person you know was got hurt by doing something very risky you might judge them for it. And there's a whole debate to be had here and I don't want to reveal what I think about it because I it's too complex to to go into that well yeah yeah but the point is that predictions take away the dilemma and make it seem like oh we're making this objective decision and we're not we're not facing the moral dilemma up front. We're hiding it behind a prediction that seems very objective but might not be your point is just be declarative.
SPEAKER_03And and and then we can always question whether the values are the right values or not and and you know correct for them. But if we say something predictive then we're taking as if it's a fact.
SPEAKER_01And also I think the other point that you make throughout the book which I believe in so firmly that and particularly the older I get which is just our discomfort with uncertainty our lack of acceptance that you I think you say in the book you'd rather we would rather have a poor decision, a bad decision than no decision. We're just so uncomfortable with the idea that it's not a fact.
SPEAKER_03Okay so this is where I have to come in briefly because I'm a weirdo. So at the end of last year I sat down for two days and I came up with a hundred predictions for 2026. He does this by the way who wins the World Cup, who's gonna win the Oscars, who's gonna you know where is the dollar gonna end up vis-a-vis the euro um I took I predicted correctly that Maduro is going to be taken out uh you know it's like various different things. This is just a game between me and myself right I'm not trying to exert power over anybody with these predictions. But this point of like yeah we should you know be comfortable in the unpredictability of life I totally agree with that. I want to live in the moment I don't want to know everything. But doing this one makes me care about this year much more in a different
Stop Predicting People, Judge The Work
SPEAKER_03way than if I had not thought about these things. Two it just makes me someone that comes to conversations prepared with an opinion. So I think sometimes these predictions can be a real good tool to allow us to you know actually maybe triangulate what our values are for this year in in a hypothetical future events that we anticipate or that are like things in the calendar that were going to happen anyway and we take a side and we think about what side we want to take in advance, right? And I think that's all like good and valuable work to do, so to say that's very interesting.
SPEAKER_00So first um you're not the only person to do this. So a couple of very prominent people who do this is Kiko Llaneras in El País does this every year. And then Azimazar does this as well and there are others. The point you make that when you make a prediction and then you track it, you you care more about it and it makes you better informed is a very interesting one because I think it is partly correct but it's also an incomplete view. As you say it makes you take sides and sometimes that is problematic because you might be betting for someone to die or you might be betting for another catastrophe to fall because suddenly you have skin in the game and you could benefit from it, especially if you bet um if you bet money which is not your case but you know you can see how it's it steps in that direction. And depending on how much you bet, you might then be tempted to influence the situation. And we have many examples of this happening in prediction marketing healthy poly market yeah. Exactly um an employee in OpenAI traded with insider information and this other guy in I don't I don't remember which airport used a blow dryer to increase the temperature measure. What he didn't hear about this yeah so he bet on what would be the temperature in the Paris airport or so or something. I think it was the Paris airport. And then to make that happen he just used a blow dryer close to the sensor and voila technically you won the the bet. So you can see how that creates incentives to interfere with the situation. And sometimes that might be good. You know if you're betting for the world to become a better place and you want to try to build a better place then wonderful. But you can see how that can go quite wrong.
SPEAKER_03So would you say that prediction markets with large are are not worthwhile or I mean you know these are outlier kind of say bad eggs and the bad eggs exist everywhere right like people do insider training on public markets too right so what what's what's your thought in general about calcium polymarket in these places?
SPEAKER_00I think they're a terrible idea. First, they gamify life they produce the incentive for citizens to benefit off of each other's miseries and I think that's the wrong incentive to produce within between citizens and even citizens of the world even just like human beings. Second, they create the illusion that they can be more precise because they can harness the wisdom of the crowds that's the only reason why you would have prediction markets that and entertainment. However the first thing to note is that this is not that diverse bunch this is mostly young men or youngish men making bets very much part of the same culture. But even if it was diverse and even if it was for a while more precise than experts the danger is that yes of course you have bad eggs everywhere but in this case they can completely distort the market in a way that is very hard to do in the stock market. There are there are two differences with the stock market. First, with the stock market when somebody buys stock from a company they're capitalizing that company and so that is adding value that is not there in prediction markets. But secondly the stock market is much larger and so it's much harder to influence in a way that a prediction market if you're a trillionaire in a prediction market and there's this one prediction that you really depend on that can really make you filthy rich or it can give you a lot of power then you just have to wait for the right moment. And it doesn't matter how many years of pretended accuracy that prediction market has. You don't you only need one
Self-Fulfilling Prophecy As The Perfect Crime
SPEAKER_00powerful bad egg as you called them to wreck havoc.
SPEAKER_03Fair um I think if we limited the type of predictions that are allowed to be made on these markets, right? Say you make it about the World Cup or you make it about other things. I don't know so things that are like neutral not like about people or misery or all that as you say. And if it were one vote per person maybe that would make it a better thing. Right? So I think I always try to find ways to design them better. And I think maybe there is some value in that um but I guess that's a discussion for another day. I am in your book. You know you you obviously talk about tech pros and uh you know there's a lot of criticism that we can level against tech pros and I'm with you on that on on many of the counts. But I'm not a tech bro, I'm a tech guy, right? And as a tech guy and as an entrepreneur, what I very much do day in day out, uh every morning when I wake up until the evening is I'm predicting a future designing products and surface areas and user experiences for people to fall in love with and I'm trying to bend the future and the world into a direction where this thing that I'm inventing or creating is going to be used by everyone and in the process I'm telling stories about the future the way I believe it to be or the way I want it to be so um I think a lot of innovation and a lot of productive you know output is coming through that act, right? And so how is that different from the things that you're criticizing or how how am I how am I doing something wrong here?
SPEAKER_01How are you culpable?
SPEAKER_03How am I culpable?
SPEAKER_00I love this question so I'm very grateful for it. So first of all I try to avoid the term tech bro. I think I managed in the book.
SPEAKER_03You didn't say it but it is obvious.
SPEAKER_00Okay. Um so I think entrepreneurs are incredibly important and in fact one of the things that I tried to incentivize with this book is innovation much more than regulation. I want young people to look at what's out there and say I can do better. I can definitely do better. And so I wanna I want people to think I think one of the criticisms that there are so many men building these systems is that when we have a more diverse culture, we come up with better products because we have more points of view. And so that we're having this conversation already speaks well in favor of you. And in fact I am so in favor of entrepreneurs that some days I am tempted to quit this band and join your band. So it's not at all that I am I think you're doing something wrong. I think that if you're trying to create a product you said you you're trying to create a product that people will fall in love with and that's very important. But what I would like to see is products that make the world a better place and people fall in love with. And when you talk about the future first of all even if you talked exactly the way that tech executives talk I don't think you would be doing anything wrong with it. I think that what is inappropriate is for people to hear it as a fact and for reporters to report on it as a fact and for governments to ask for their predictions as if there was a fact. That is wrong. So as long as we're all clear on what we're doing that this is marketing, that's fine. You know marketing is marketing and as long as you're not lying or misleading it's not wrong.
SPEAKER_01But I I do feel that your main argument in the book is that prediction is imposed most on those who who are unable to contest it. And that's the problem with it is that we do take it as fact. Particularly I love the point you make about numbers um and those metrics. Because I am definitely a victim of that. The minute I'm listening to more or less, so you mentioned Tim Harford a few times who I'm a huge fan of. But I I love when he unravels and he exposes how this data was collected, or particularly during the COVID crisis, I listened to that radio show assiduously. And it was such a revelation to me because I I didn't think deeply about these figures that I was being quoted. I just sort of assumed that they were true.
SPEAKER_00Exactly. And I'm an academic. So I I've thought about this question, and I just wrote an article in which somebody asked me, What is the future of the online life? And so the way I phrased
Prediction Markets And The Blow Dryer
SPEAKER_00it is look, I'm not a prophet. I'm not gonna tell you what the trends are, because trends can change. What I'm gonna do is I'm gonna describe to you the future that I want to inhabit, and I'm going to invite you to build it with me. And that's a much more honest kind of speech act, I think. You know, this is not guaranteed, this is a future that I would want. It's a kind of question, is this the future that you want? And it's a kind of invitation. Let's work together, let's collaborate, that let's build this.
SPEAKER_01I was thinking also about counters when when I was thinking, well, what is the alternative? You know, we're we have harvests, we've planned forever, we've predicted our future forever. As you say, it's just a knee-jerk response to, is this gonna last? Is this gonna work? On so many areas of life. And I remember once my son coming back from school, they all had the same history teacher who, and I don't know which military figure said this, but it's um, you know, fail to prepare, prepare to fail. And so they would sort of get in the car with their school bags and things, and I would do a checklist and they'd have forgotten something, and one of them would make this quote and everyone would fall about laughing. Anyway, I was really taking to heart an awful lot of what you said, and living in the moment and not making predictions, and certainly not making false predictions or thinking about hypotheticals that I really don't want to happen. But uh, with three small sons, it if I didn't plan at the beginning of a day, it would be entirely shambolic. But what I realized was, and I'm not the most organized person, so this was uh something that I had to learn and a muscle that I had to develop. But what I did learn was that the plan wasn't inevitable. It was there, and we would cleave to it if things were going in the right direction. But you know, if someone had diarrhea or if it rained, we couldn't have the picnic, you know, we'd have to return home. But the point is it was adaptive, it wasn't set in stone, and it wasn't a fact, and everyone could contribute to bending it this way or that. That that was my takeaway about predictions in organization.
SPEAKER_00And that's very interesting because it might seem like two people are doing exactly the same thing when they make a prediction, but the way you predict and the way you think about prediction changes everything. And so very often, it's not that my advice is don't predict ever. It's just we need a more enlightened way of prediction that allows for serendipity and that recognizes that there are some some things that we cannot predict and that we have to adapt. So, for example, something that I struggle a lot with is that my calendar tends to be so full and so tight that it doesn't allow for any changes. And that's horrible because it's very stressful, but it also doesn't allow for the kind of serendipity that makes life so wonderful sometimes. And so I struggle a lot to learn to and to stick with it to just build in blank spaces in my calendar and treat them as if they were uh an appointment sacred, because otherwise my calendar just becomes this impossible thing to manage.
SPEAKER_01So you mean you you build in serendipity or time for surprise?
SPEAKER_00Yeah. You have to allow serendipity to get to you. If you don't allow it, it's uh it's a kind of fragile flower. And so I make time to at least be open to it happening.
SPEAKER_01Great. Yeah, I think you're both quite similar in that way. I I don't have that problem. I do. I I spend hours staring out the window um and uh I have no plans, so there's too much serendipity in my life. Let's switch.
SPEAKER_03So what would I I guess conclude from what you said is to predict well is to be invitational, to be declarative about what um aspirational future you would like to see, not a declarative this is what it's going to be. And to yeah, try to be very clear about how to get to that future and why you want that future to be uh the way it is, and to get everybody to come along if they are interested in that, and maybe to criticize us if they don't believe in that. So is that maybe a good, I guess, summarization of how I feel? Or would you like to add something to it?
SPEAKER_00Absolutely. And the way I would describe it, because of you know professional deformation of being a philosopher, is it's important to acknowledge that the future is unwritten, that people have agency, and to respect that agency. Because when you tell someone
Designing Futures Without Stealing Creativity
SPEAKER_00that the future that you want is inevitable, you're sort of putting them down and discouraging them from disagreeing with you. And that's just disrespectful and it's bad for democracy and innovation.
SPEAKER_03Right. And I guess the other point that you made um that also Natasha made right now is um preparation instead of prediction sometimes is uh the right approach.
SPEAKER_00And sometimes a combination. So you have a plan A and then you have a plan B because you know that the plan A sometimes is just doesn't go as planned.
SPEAKER_03Indeed.
SPEAKER_01Something we didn't cover and there may not be time to, but I loved the section in your book where you talk about just to go back to AI for a second and what it can't give us. Do you do you know the the part that I mean, which is really leaving us to create and letting AI do the jobs that perhaps we're less good at?
SPEAKER_00Yes. Thanks for that question, Natasha, because I worry that the narrative is that AI is going to do the boring jobs and we're going to do the creative jobs, and that's great. But when I look at the world, very often we're doing the boring jobs and AI is doing the creative jobs. And the more we allow AI to do the creative jobs, the less creative they will be, the less attuned to the moment and to what we're feeling. And there is so much joy and so much satisfaction in creation. It is partly the process of creation that is joyful and not only the product. So if we delegate that to AI, we're missing out on creating our world. So yeah, I I think that there are things that we shouldn't allow AI to do because it's just too joyful and important to do it ourselves.
SPEAKER_03That's uh that's a beautiful note to end it on, Carissa. Thanks so much for writing this uh create book. Uh it's prophetic. Had to get that in. Um, thanks for waking us up. Thanks for making it. It's so so good. It is such a wake-up. Yeah, yeah. Making us think about what people are saying out there and not uh exerting power over us. Um and yeah, um looking forward to what you're gonna be writing next.
SPEAKER_00And thank you so much for reading it because it only makes sense to write it if somebody reads it. And thank you so much for reading it so carefully and for talking about it with me. And often it's these conversations that make me realize what I want to write next. So thank you so much, Natasha and Omit.
SPEAKER_01Thank you.
