Good morning,

This week’s Stratechery Interview is with Benedict Evans. Evans is an independent tech analyst based in New York. Evans started his career in investment banking with a focus on the telecom sector, before his first stint as an independent analyst focused on mobile. In 2014 Evans joined Andreessen Horowitz as partner and analyst, before departing in 2020. Evans was an inspiration for me when I started Stratechery, and it was a real delight to talk with him this week.

In this Interview we cover Evans’ career, before exploring the differences between European and American approaches to regulation. We cover antitrust questions, including the Apple case, and Evans explains why he’s not sure how much it matters. After that we focus on AI: what are the right frameworks to think about AI’s impact, both on society and on the largest tech companies? Is generative AI the end state, and will it manifest as a generalizable assistant or as an ingredient in new generations of products? Is there a bubble — and what would it mean if it pops? I thought this was a great conversation — Evans’ has well thought-through frameworks for nearly everything — and I trust you will enjoy listening to it as much as I enjoyed having it.

Evans’ website is located at ben-evans.com, where he both posts long-form essays and hosts subscriptions for his newsletter. Evans has written about many of the topics we discuss in this Interview; I will link to them as appropriate, but strongly suggest you subscribe.

As a reminder, all Stratechery content, including interviews, is available as a podcast; click the link at the top of this email to add Stratechery to your podcast player.

On to the Interview:

An Interview with Benedict Evans About Regulation and AI

This interview is lightly edited for clarity.

Background

Benedict Evans, welcome to Stratechery.

BE: Thank you for having me.

Well, the pleasure is definitely all mine. It’s fun to come full circle here, when I started Stratechery, I was actually one of your biggest reply-guys, always linking pieces I had written that were relevant to your tweets, and you graciously passed a few of them on. I actually emailed you at one point asking for advice on how to be an independent analyst, and then I think your advice ended up being, “Well, I’m going to go work for Andreessen Horowitz for a while.” But you’re back! You’re an independent analyst all over again. That’s the history I know, but before we get to that. I want to know what came before that? What’s the Benedict Evans story? I presume you grew up in the UK, but that’s as much as I know.

BE: It sounds very vain, but every now and then I get someone who emails me asking for career advice, and I’m like, “Have you looked at my LinkedIn“? Meaning, I don’t think I can give advice to anybody. An old boss of mine said my career looked like a process of Brownian motion.

Yes, I grew up in the UK, went to university in the UK, did a degree in history, which is essentially analysis, then went and worked in investment banking and equity research, which at the time was a big and interesting industry. It’s gone through probably two, maybe three, mass extinction events since I entered it. So I covered mobile operators back when mobile was a kind of dynamic, exciting, disruptive, aggressive growth industry, “We’re going to give everybody on earth a mobile phone”, most people still thought that was a joke, that was kind of crazy, and they did and then they turned into water companies, they became boring ex-growth utilities. Then I went and worked in strategy for a couple of media telecoms companies and then worked for a research and consultancy boutique in London.

That’s when I first encountered you. I don’t recall the name, but I do remember that.

BE: Yeah. So I was working for a boutique in London called Enders Analysis, and because I was there as a freelancer, I was doing other things as well. That’s what I was doing quite a lot of, but the point was because I was freelance, I was free to publish stuff.

The way that I describe it sometimes is, there’s this great quote from a New Yorker writer in the forties and fifties called A.J. Liebling, who said that he “could write faster than anyone who wrote better, and better than anyone who wrote faster”, and I imagine a Venn diagram of people who are good at analysis, people who understand the industry, people who can say stuff in public. There’s people inside Google and Meta who understand ad tech much more than me, there’s people inside Apple and Google who knew a lot more about what was going on with smartphones in 2010 or 2015 than I did, maybe. There are also people at Apple that couldn’t write in public and weren’t analysts. The people at Goldman Sachs or McKinsey are analysts and they can kind of say stuff in public, but only in very specific parameters. Sell-side analysts have to write for a buy/sell audience, people at McKinsey or BCG or Bain have to say stuff in a particular way so as to not annoy clients.

Then you have people in the tech industry, who there was a brief moment where it was like me and Horace Dediu, and I can’t remember if you were doing this, but we were making charts of iPhone sales and people would say, “Where are you getting this from?” — they published them every quarter! People didn’t really understand that all of these numbers were public and you could go and look them up and make charts.

I was using your charts to be clear. I appreciate the hard work.

BE: The funny thing is I haven’t updated my Apple model since 2018 or something because, “Who cares?”, which is a point that we’ll get onto. So yeah, I was a mobile analyst at a point when mobile was the thing, and so I was writing lots of stuff about mobile and over time mobile stopped being the thing, it stopped being interesting and like you, we moved on to talk about other things.

To finish the circle, then I went and got hired by Andreessen Horowitz at the beginning of 2014. Mark Andreessen’s joke, and people always attribute this to me, but I was just quoting Mark Andreessen’s joke was that, “Andreessen Horowitz is a media company that monetized through venture capital”. At the time they had a big content operation, which was part of the whole thesis of, “How do we be a completely different venture capital firm?”. Part of what I was doing was saying hopefully clever stuff in public on behalf of Andreessen Horowitz about the industry and so I did that until the end of 2019 and then I left to go and do my own thing.

I love this story. That bit you mentioned about the Venn diagram in the middle, it resonates so deeply with me, just that aspect of just being fast and there being this vast gulf, the way I always framed it is, “You have people in the media that write about tech products, you have people who write about earnings, and there’s this massive space in the middle: How does A get to B?” That, from my perspective, was just green space to address and talk about. I always felt you and Horace helped plot the way there, and the Internet just makes that possible. There’s no permissions and that doesn’t mean you’ll succeed, but it means you can actually try new stuff and it’s been cool from afar to observe you navigate that.

BE: Yeah, there’s a whole other conversation, we could do a whole other podcast about analysis on the Internet. The other I think really interesting case study is Imran Amed at Business of Fashion, because he was actually a McKinsey consultant. He was fascinated by fashion, he started a blog, I think a newsletter, about fashion and that became a media company which has a hundred people and a conference. There what he realized was there was no modern professional Internet for B2B publishing about fashion.

So rather, to your point, there were people who would do the LVMH earnings in the FT and the Wall Street Journal, and there were people who would write about the product in Vogue or in another part of the Financial Times or the Wall Street Journal or whatever, but there was no one saying, “Hey, it’s really interesting that Fendi is building up their retail business and that the Vice President of Merchandising has gone from this company to that company,” or whatever the names would be. There was no one doing that in fashion, now there’s a lot of people trying to do that in tech, but again, ten years ago there were fewer people doing that.

Yeah, that’s a perfect analogy, it fits exactly what I was saying.

U.S. vs. E.U. Regulation

Anyhow, I’m thrilled to have you — I’d be thrilled to have you anytime — but in this particular instance, I do want to tap into the European side of you. The conflict between Europe and American tech companies is only getting more and more serious, and as a red-blooded American, I have to admit, it’s hard to not escape some of the sense that the E.U. seems out to get us. Is that an unfair feeling? Is this just going to be a thing that happens?

BE: So first of all, I should make a jokey-but-serious point, which is I’m not European, I’m British.

Yes, fair point.

BE: There is a meaningful difference in this context, and not because of Brexit particularly. There’s probably I would say two kind of basic and important cultural differences here.

The first cultural difference, which I argue about with our mutual acquaintance, Steven Sinofsky, quite often is that American regulation is based on lawsuits rather than regulation, so the atomic unit is, “We will sue you and try to persuade a judge that you broke a law written 50, 75, or 100 years ago, plus the associated case law and precedent that’s accumulated since then”. So right now the DOJ is going to try and persuade a judge that Apple has broken the Sherman Antitrust Act.

The model in the U.K. and in Europe is, we will create a statutory regulator that has the power to write rules, which is something that looks more like, say, the SEC. We can just go out and say, “No, we think that breaks the law and we are going to write a new law, and if that doesn’t currently break any of the rules, we’ll just make another rule that says it does, if it’s against the principle of what you are doing.” This is the whole conversation that’s going on around crypto at the moment, “Are these securities or not?”. The point is you don’t have to go to Congress and get both houses of Congress to write a law about this, the SEC can sit and can try and write laws about this stuff. So that’s the first point — is it a lawsuit or is it a regulator that can write rules?

I think the second issue is that American regulation and laws tend to be focused on finding the right line in the right paragraph and saying, “Do you comply with this or not?”. The classic framing here is that the American auditor’s opinion is, “Does this comply with GAAP? Does this comply with audit law?” and the UK auditor, it’s twenty years since I had to know about this stuff, but the UK audit says, “The accounts give a full and fair depiction of the state of the company.” Those are not the same thing.

(laughing) Definitely not the same thing.

BE: So Americans look at the DMA and say, “But it doesn’t say how to comply with it, what am I supposed to do here?” And Europeans say, “Yeah.” That’s not a criticism, that’s how it works.

You can make legitimate criticisms of both approaches, but they are consciously different approaches. The US model in principle has lots of loopholes because you can find something that isn’t covered by the letter of the law and you’re clear. In Europe you are guessing what the regulator wants and so there’s advantages and disadvantages to both of those.

I think the third answer to your question is that yes, these companies are all American because that’s where all the big companies are and we could have a whole conversation about why, and there’s a dumb Silicon Valley overgrown teenager thing, which is, “Oh Europeans can’t…” That doesn’t get why. These are 15 small countries and it’s a lot harder to build a giant company in a country that’s got 50 million people than one that’s got 400 million people.

Different languages and all sorts of those things. Yep.

BE: You have these very large companies that have a very large amount of power over how people use the Internet. Yeah, they’re all American, but that’s not really the point. The point is that they’re large and they control how people use the Internet, and there’s a very strong, again, cultural difference, and I’ll say this and then stop monologuing, which is that in the US and to some extent the U.K., the approach is, “You can do whatever you want and then we’ll pass a law if there’s a problem”, whereas the European model is much more, “Liberty is embodied in the state and the state grants you freedoms and your legal identity on your freedom is subsumed into the collective”. The collective then says, “You can do this, you can do that”, and that gets reflected in everything from, “Why are American airports terrible and European airports beautiful?”, to, “Why does Europe have employment law and America doesn’t?”, to, “Why does Europe has privacy law and America doesn’t?”, to…

Does this a little bit tap into the big company question though?

BE: Yeah, but the presumption is the fact that you have a very large company that has meaningful power over some aspect of people’s lives that is not de facto, by default regulated, is de facto a failure. So the fact that there is not systematic regulation of technology, even though it’s so big, is a failure of government that needs to be fixed. Whereas the American approach is, “There’s a specific problem, we should have a privacy law”. The European approach is you have this entire field, there should be systematic laws about it, just as there are laws about finance or food or airlines or cars, there should be systematic laws about tech just because it’s a large part of our lives. So that’s the starting assumption.

No apologies for the monologue, that was a great one. Do American tech companies get what you just said, or is there just a fundamental ignorance, or is it just a, “We get this, we don’t believe in it, we’re going to fight it”?

BE: I think there’s certainly a specific cultural gap in not understanding that you are not going to get to go to court and overturn the ruling, that it doesn’t work like that. There’s a specific cultural gap in not understanding, no, they don’t have to first prove that we have a monopoly as defined under that act, just not understanding the mechanics of how it works and where it’s coming from. There’s an essay I wrote a while ago about this and I said there were three ways that an industry will say no when a government tells them what to do.

Yeah, it’s a good one. Tell me the three ways, it’s a great essay.

BE: And I’ll say very briefly, and it’s not about tech, it generalizes to any industry.

So supposing you are a doctor or a lawyer or a construction engineer or an architect or something, and the government comes to you with this new set of rules and your default reaction is to say, “No.” It’s like when your ex-wife comes to you and asks for something, it’s like, “No.” Then you start listening, then maybe you should probably listen and then decide, “What is it that they said?”. But my point is there’s three reasons, and the problem is that the industry will always say no. So as a politician and a policymaker, your immediate reaction is, “Well, they’re just bargaining”, and you discount by 50% everything that they say.

The point I was going to get that I made in this essay, was the three reasons why you might say no, the first reason might be, “This is dumb and annoying and we don’t like it, but whatever”. So this would be like you’ve got to allow chronological feeds in social media. There’s no evidence for this, users don’t actually like it, there’s a very small number of very ideological people who feel very strongly about this. If you actually work for a social media company, you know this is really dumb. But in the end, the world will continue. It’s like the extreme case would be like the iPhone has to have a USB-C socket. This is dumb, it doesn’t matter, in the end, it really doesn’t matter, it’s just annoying.

The second case would be, and do you know what the AB 5 was in California?

Yeah, the ride-sharing law.

BE: Yeah. So California says Uber road drivers should be employees and have health benefits and everything else. Okay, reasonable people can debate whether that’s correct. California passes a law that basically says, “All freelance workers of any kind automatically, if you do any freelance work of any kind, you automatically become an employee.” So you are a lighting technician, you get paid three hours to go to a theater to go and rig some lights. Guess what? You are now a full-time employee of that theater. And I’m not exaggerating, this was actually what happened.

Yep, I dealt with it personally.

BE: Basically, the point is everyone in California went to the California government and said, “You’re out of your mind, you don’t understand what you’re doing.” And they said, “yeah, yeah, yeah, people always say that.” Then there’s six months of chaos and then the whole thing had to get overturned. The point being sometimes when people say, “This is a terrible idea,” it’s because actually you haven’t understood what you’re doing. Maybe Uber drivers should be employees, but that’s not what you’ve done, you’ve done something completely different. So that’s the second case.

The third case is, “You’ve asked for something we can’t do, we actually can’t do it”. “Please make gasoline that doesn’t burn”, that would be nice, but how? “Please make secure encryption with a backdoor.”

The classic example.

BE: Yeah. “No, but you’re going to have to choose one”. The challenge, I think you see a lot with a lot of the argument now about the DMA and the DSA in Europe, is everyone’s saying no about all of this stuff, and the challenge for policy makers is unlike, say, cars where we all grew up with this stuff, when they say, “No, we can’t do that,” or, “That’s dumb”, you don’t really know which it is.

Whose responsibility is that to convey, or is this where the negotiation part comes in?

BE: Well, so some of this is negotiation, some of this is that DG Competition, I looked it up, they’ve got like 800 or 900 people. There’s an interesting rabbit hole that for — this is a legal podcast we could go down — that as far as I can work out the U.K. Competition Authority’s got more people than the whole of the DOJ. The whole of the DOJ Anti-Competition, obviously the whole DOJ is vast, but the competition bit of the DOJ is smaller than the UK Competition Authority, which is not a fair comparison.

By the way, they put out some pretty good stuff to be totally honest. I mean, I talked about this in the context of the Microsoft Activision case. They actually had a theory of the case that made sense. I’m not sure I still agree with it, but it was a marked improvement over what the U.S. regulators were trying to do.

BE: This comes back to my point about lawsuits. The parody or the reductio ad absurdum of the U.S. regulator is you’ve got ten lawyers in the room with an engineer in the corner, whereas a European or British regulator, you’ve got ten practitioners, and there’s a lawyer in the corner.

I said this on Twitter once, and I’ve got these American lawyers telling me I was a moron, which of course is always what you get on Twitter, like, “Lawsuits should be written by the lawyers.” Well, no, that’s not how other countries do it. The way most countries do it is the regulation is written by people who know about the industry, and then you get a lawyer to write the law, to write up what they decide, the lawyer kind of takes the minutes. That’s an interesting difference between the way Europe is going to come at this, they’ve actually got a thousand people whose job it is to look at this.

But what you also have, which the people have been pointing out around the DSA and particularly the arguments around the DMA, is this is the moment in which all the competitors come in and ask for things. So the challenge for the regulator is to differentiate between what’s good for users, what’s good for the market, what’s good for choice, versus what’s just virtual pleading from a competitor. To what extent is your desire to increase competition aligning with just giving what any competitor turns up and asks for.

That’s always been one of the big questions about European regulation, I think from the American perspective. Because it often seems to work out very well for these — I mean Spotify is a big one, so that one maybe feels a little bit more legitimate. But you had the whole Google Shopping case that was Found-em, some tiny little defunct British company and it’s like, “What are we actually doing here? Why are we not letting Google give users what they’re actually shopping for?”.

BE: So there’s a whole interesting challenge. Well, there’s two or three interesting challenges here. One of them is, one of the existential problems, profound structural problems for regulators facing technology, is that we took 75 years to regulate cars. It took 75 years to put seatbelts in cars, and then another 25 years to get everyone to wear them, we’re not going to spend 75 years doing all of this. You might think that’s not a problem, or that is, but we all agree that there are some problems, and we’re not going to take 75 years over it. But you’ve got 150 of these things, and it’s changing all the time.

The core issue with the Google Shopping case is not that it was a tiny European company, it was that it took them ten years. Everyone in any regulatory agency is really conscious of this problem, that on the one hand you can go through your slow, painstaking legal due process of five, ten years, which might make sense if you are looking at the car industry, or a supermarket or something where not much has changed. But it doesn’t work in an industry where you can go through an entire cycle in that period.

But the counter problem is you can do what the FTC tried to do with Within and stop Meta from buying a VR fitness company before there is a VR market, when you are very consciously, deliberately, speculating about what the market might look like in the future.

There’s not a right answer to that, there’s certainly not an easy answer to that. You either wait until it’s all clear, at which point it’s too late, or you try and intervene before it’s all clear, at which point people say, “What the hell are you talking about?”, that’s not a thing yet.

Right. Well, there was a story just this week where Microsoft is now unbundling Teams from the rest of Office all over the world, they did that first in the E.U. Part of the issue is Microsoft won so companies want consistent billing, “So fine, we’ll do it everywhere”. It speaks to how Europe can impact the rest of the world. But on the other hand, does it even matter? The Teams versus Slack war is over, Teams won.

BE: Yeah a) it’s too late and b) this is kind of — this is something I wrote about three or four years ago, at what point should that be integrated into the product?

My argument is Teams is actually quite integrated. That’s one of the reasons it actually is better than Slack, even though it’s not as good of a chat product, that seems like that should matter.

BE: It’s a slightly different product, but let’s face it, everyone uses it because it’s there, and it’s free and it’s included in Office 365, that’s the same for Meet. I think the challenge is, you go back long enough, everything was a separate product. You go back to the 80s, and spell check is a separate thing that you buy for like $200. If you want to do a chart in a spreadsheet, that’s a separate piece of software that you have to buy for $100 or $200. If you want to print, that’s a separate product. You want to print in landscape, there was a product called Sideways that let you print your spreadsheets in landscape. It was like $80 in 1985.

Windows 95 didn’t have an TCP/IP stack.

BE: Exactly. You can say the thing about cars, cars used to not have turn signals. If you were to do a driving test, you had to learn hand signals, that was because cars might not have a turn signal. So there’s always this challenge of what should be integrated and what should not be.

It’s kind of easy, and it is a lot easier to say that about Apple Music than it is to say about the Photos app, which is the latest thing that the E.U. has kind of floated here. That’s kind of another kind of philosophical challenge, in that the DMA is doing a lot of — and this is not just an observation by tech analysts — the DMA and the E.U. is doing a lot of really detailed product micromanagement and second-guessing decisions by engineers in quite specific products, where they don’t really have any kind of product expertise to make those kind of judgments.

There’s a question of, and I think this also comes to the DOJ thing, “Should you be trying to change the whole structure so that Apple, or Google, or Meta or whoever doesn’t have that power?”, which is they allow third-party app stores, allow side-loading scenario. Should you go in and make detailed decisions and say, “No, the button for you to click here should be green and not blue, and it should be two pixels further up on the screen”? Or should you kind of come in afterwards as what’s happened with Spotify, except not take ten years, and say “No, that specific thing you are being anti-competitive. We’re going to tell you you’re not allowed to do that.” Those are very different philosophies to how you would come at this.

I mean, I’m not an antitrust lawyer, and I’ve deliberately never commented on what antitrust law says about Apple, other than to point out that here are the things some of the lawyers are arguing about. But what I read is that American courts really don’t like doing specific detailed product decisions.

Yeah, it’s a very core principle of antitrust law. That’s a guiding principle, which is “We don’t want to be in product decisions”.

BE: Whereas the EU is very explicitly deep in product decisions, or at the very least they’re saying, “You make a product decision, and we’ll decide if we like it, and if we don’t, we’ll fine you $2 billion.”

Does Antitrust Matter

Does it matter in the end though? You wrote a provocative column saying, “Look, all this is a side show, it makes for good headlines. But at the end of the day, it doesn’t really impact anything.” Is that just because it’s always too late?

BE: I think there’s two or three things to think about here. One of them is, part of what I was saying in that was, most people in tech in their actual day-to-day job will not be affected by this. Most people in Apple do not work on the App Store billing process and so don’t have to worry about this. If you are in the chip team at Apple, this is someone else’s problem. If you are in image compression optimization at Meta, precisely how the App Store, the advertising tracking stuff works, is someone else’s problem. Most people in tech don’t work at a company that’s covered by any of this.

It’s only a couple of them, we know who they are.

BE: Yeah, exactly. I used to say it’s like instead of thinking about cars, you should think of manufacturing. Imagine we were arguing about car regulation in the 70s, and you work at Lockheed or General Electric. You’d be like, “Yeah, but I don’t work on that stuff.” I mean as a consumer, sure, but as an engineer, I do not spend my time thinking about airbags because I make gas turbines that go in oil refineries, that’s not my job. So that’s kind of one thing, is most people in tech just — and it’s not actually what they work on.

I think the other issue here is, a) I think a lot of these changes are quite marginal and don’t actually align with what consumers want to do and b) none of them have anything to do with what’s happening next. Those are quite distinct issues, so the issue would be like no one is actually going to use third party app stores really.

We’ve been down this road, we know it’s not going to happen.

BE: We’ve had this conversation, we had this conversation 15 years ago, and ten years ago, and five years ago, there’s a very small number of people who feel very strongly about this, many of them do not appear to have heard or seen any of the reasons why people think the App Store is a good thing. I’ve got no interest at this point in explaining why again, but I honestly don’t think anyone will actually use a third party app store.

I just think an awful lot of this is a kind of, and the same thing with chronological news feeds, chronological news feeds, again some people have very strong opinions about them.

Yeah, that’s a great analogy.

BE: Users don’t like them, they don’t make sense, there are really strong product reasons why we moved away from them. You’ll try it for 30 seconds and say, “Wait, this is crap,” and switch back. So that’s one point is most of this — I always talk about Spotify as the exception that proves the rule, I always thought that Spotify was in the right and Apple was in the wrong.

Absolutely. I thought it was a black-and-white case, Apple is literally building a competing service and is heavily favoring itself.

BE: Yes, there is absolutely no basis in which this is good for users. They’re just confusing users and Apple is breaking the user experience to no benefit. However, I can think of three other examples of that. There’s streaming games and eBooks, and maybe, I can’t think of anything else.

I mean I was at Andreessen Horowitz for six years, I don’t think we ever saw a startup where we thought, “Oh, it would be good if they could do X or Y, but Apple won’t let them.” Yeah, you can say, “Well that’s because no one started the thing in the first place”, but no, it’s actually quite hard to think of cases where this was really important. So that’s one side of it.

The other side is, “Guess what, I’ve got a Perplexity widget on the home screen on my iPhone”, so if we think that generative AI is the future of user interfaces, nothing in any of this changes any of this. There’s not even anything in the DMA that says you can replace Siri.

Yeah, well that’s the story of the DMA. The one thing about the U.S. case that I think is interesting is that I think the piece that is arguably the biggest threat to Apple is the bit about smartwatches, just because it’s really getting into the core of Apple, you’ve written about this a few times. Apple does stuff that it’s kind of half good for users and also half good for them, they can always point to the one or the other. But I do wonder what new AI devices could we get that would be better if they could integrate with our phone to the extent an Apple Watch does, or an AirPods or something like that. So maybe that’s one where they’re backing into actually an example of something that could actually make a long-term difference, if some of these APIs were more broadly available.

BE: Yeah, I mean there’s an antitrust lawyer argument here, which is, “How much is your case actually not about what’s in the courtroom?”.

I think there’s a lot of that here going on here.

BE: Well, I’m not sure which side of this you’re on, there is an argument within tech about how much did the Microsoft/DOJ, how much did that help Apple?

I’m with you, I don’t think it had anything to do with it! I didn’t have time, I had a lot to write about there, so I wasn’t going to dive into all the historical and accuracy parts. I’m glad you did, because you said you’re kind of professionally embarrassed. Why does that happen? Why is there the need to sort of retcon this entire — I worked at Microsoft, getting iTunes on Windows was the most important thing in the world.

BE: The interesting thing about this filing is, and it’s actually, it compares quite a lot to the House Antitrust Commission thing from a couple of years ago, which made claims that were obviously wrong, and that had a footnote and the footnote contradicted the claim, there was just a lot of sloppy work in that document. There’s a lot of sloppy work in this.

I think from a legal point of view, the lawyers who have looked at it said they constructed a clear, straightforward argument. Apple does A, B, C, D, therefore E, therefore F. Like the legal logic hangs together. The problem is when you poke in at the claims they make about how the industry works in each one of them, then there’s holes everywhere. So there’s technology holes everywhere, but maybe not legal holes everywhere.

The point I was going to get to you though is that for antitrust people, it’s like the joke I put in my newsletter, is there was some crack that somebody made about Osama Bin Laden, that every time an American soldier somewhere in the world stubs his toe, Osama Bin Laden would take credit, you’d claim it was him. There’s a car accident at an air base in Okinawa, Osama Bin Laden claims credit. It’s like, come on.

There’s a lot of that here, that of course, if you’ve decided to spend 20 years of your life working in competition law, I would hope that you think it’s important, otherwise why are you doing it? Then you kind of tend to think that everything happened because of you, and everything is a competition problem. The really absurd one, though, is not actually the iPhone thing where some people can make a point. It’s the iPod, is claiming credit for the iPod, where just as you point out, they just got the facts completely wrong.

But the challenge in all of this is like, what is the next thing in tech? Is it going to be a fundamentally new iPhone app? No, that’s tapped, the whitespace is done, where is all the creative energy? The creative energy is in generative machine learning. Everyone really thinks that’s where all the new stuff is going to happen in the next five, ten, 15 years, that’s what you and I are spending most of our time thinking about and writing about.

So I was kind of ambivalent writing anything about the Apple thing, because I was like, “I wrote this five years ago, and ten years ago, why am I still writing about this?”, but the thesis would be something like, which is what we saw with the Rabbit phone, which I’m very skeptical of, but I’ll say to my phone, “Look, I’m going to Paris in a month with my partner. What are the new hotels that we would like and suggest some restaurants.” That at the moment is kind of a bullshit demo. It’s because you got the demo from Rabbit, the Rabbit people. The Rabbit guy says the defaults are all fine, which is not how any normal person books a holiday, you don’t just choose the first result.

Yeah, I don’t trust you with my flights or hotels, that’s right.

BE: But the reason I mention it is, if you had something on your phone or plugged into your online activity in some way that saw everything you did, if I had something that was plugged into Safari, Email, Instagram, TikTok, YouTube for the last year, then it would know, “There’s no way he’ll stay in that hotel, he’ll stay in this hotel.” Or you could, in theory. That’s not science fiction anymore, that’s an engineering problem. Then it actually could kind of say, “There’s no way Benedict would stay in that hotel, he would really like this hotel.”

For that, you need to do a lot of stuff that Apple won’t let you do. I mean Google has some of this, your iPhone has that, Meta has about half of it. Will Apple let you do that right now? Absolutely not. What would that look like? What would that mean? Now that’s a giant question, and we’ll see what Apple announces at WWDC, but that’s kind of a path for disrupting Apple and posing a competitive threat for Apple.

Nothing in any of this really get to that. It would be really tough as a lawyer to sit down and say, “Okay, I’m going to require Apple to allow that”. Because what would that mean?

Well, one of the interesting things about all this regulatory fervor is the extent to which all of them are united in that, “We’re going to protect user privacy.” The problem is that is just completely counter to competition in so many of these areas. If there’s any sort of data requirement, they’re just at fundamental odds. I think that’s the big problem with the DMA, to be honest, is you could write much clearer laws, or much clearer principles, maybe I should say, but if you’re going to include carve outs for privacy and security and things along those lines, that’s where yeah, maybe it’s not a technical loophole, but Apple, there’s some compelling arguments they can make.

BE: Yeah. I was at a competition conference at the beginning of 2020, so just before the pandemic, and one of the academics there said that, it was either an academic or somebody from a European regulator, and he said, “Look, a big social media company goes to the competition authority, and the competition says, ‘You have to do X’, and then they go across the road to the privacy regulator and the privacy regulator” — which America doesn’t have, of course — “The privacy regulator says, you absolutely must not do X.”

Right.

BE: And Adam Mosseri is like, “Well, I’m an engineer, do you want me to make it easier to get the data out or harder to get the data out? I can do either, but I can’t do both, you’re kind of going to have to choose which of those it is that you want.”

And you see that tension right the way through the DMA. “You must allow …”, and this was the thing with messaging. It was the DMA or the DSA, I forget. Anyway, it’s like, “You must allow third-party messaging apps.” The original draft was, “You must allow any,” direct quote, “Any third-party app to interconnect, and they must have access to all,” again, direct quote, “All data that your internal teams have.” So the internal spam team that’s tracking who’s messaging whom and what the patterns of distribution are, all of the data of who talks to whom on WhatsApp, you have to give that to the Iranian Intelligence Agency, that’s what you just said. You may not realize that’s what you’ve just said, but that is what you just said.

Yep. This is where your framework of the “Three No’s” is very applicable.

BE: It’s the equivalent of, you think you just said that Uber drivers have to be employees, but you’ve actually just banned freelance work in California, that’s what you just did.

Well, it’s combined with the third no, which is you can’t do that, which to your point about Adam Mosseri crossing the road and being told opposite things. It’s like, “No, you actually have to choose one.”

BE: There was a great case about this a couple of years ago, which I’ll now forget the precise details, I’m just remembering it or I could Google it, but it was essentially that there was one E.U. law that said that social media companies were not allowed to look at what people were doing, and another law that said social media companies needed to track CSAM, and basically one law banned you from complying with the other law. I’m garbling the details now, but there was a period about a week where these two laws were in direct conflict, and you literally had a law that said, “Adam goes to prison if he does this, and he also goes to prison if he doesn’t do this.”

I always keep referring to regulating cars, we have an awful lot of laws about cars now, but a) they’re kind of in conflict and b) complicated, and they come from different places. So does our tax code prioritize low density development in suburbs? Do we also have a law that penalizes people for having long commutes? At some point, people would point out that there’s a problem here and meanwhile, you can’t go to General Motors and tell people not to live in suburbs, that’s not really a mechanical engineering problem, that’s an urban planning problem and a tax code problem and a bunch of other things, but it’s not mechanical engineering.

Scale Is Different

You made this point, just to zoom out to the AI bit, you wrote an article about AI ethics and this idea that it applies to tech broadly. It makes your point about the employees that are impacted. If you look, it’s so big, it covers everything, you kind of can’t approach it with just a one-size-fits-all approach.

BE: Yeah. I’m sure that there is certainly a field of engineering ethics, but you get to a certain point and you think, “Does engineering ethics cover building cycle lanes if you work for General Motors?”, maybe, but you’re not doing the urban planning. You’re not building the cycle lanes, that’s got nothing to do with General Motors, and it’s certainly got nothing to do with Lockheed.

So there’s a point at which these terms become kind of vague and problematic because you’re trying to capture everything in human society because everything kind of gets touched by software somewhere and it’s easy to say, “Well, you shouldn’t write facial recognition software and give it to the Chinese government.” Fine, except that the Chinese government is quite capable of writing their own, so you haven’t actually achieved anything.

This is actually a thing I wrote about this last week, I read about this when we were talking about the last wave of machine learning panic. It’s like, we’re okay with mugshots, we’re okay with the police having mugshots on the dashboard of their car, what if they have cameras on the car that scan every face near the car? What if you connect every camera in the city to that database? I think people listening to this might not all agree on what we think about that, we know what the Chinese have done, what do we think about that and why? Do you have a really coherent chain of logic as to how we think about those questions?

This is the point you make again and again, is that the scale makes it different.

BE: But that’s not engineering ethics, that’s a politics question.

Yeah. Well, this bit, which I’ve made this point also, and it really resonates with me when you write about this, that there are lots of these issues, you can distill them into a single example, and then you’re like, “No, that’s wrong”, or, “Yes, that’s right”, but tech scales. The whole economic model is the idea is that it’s reproducible at scale. AI is doing that at an even larger effect, and our feelings about that as a society or individuals changes with scale, and I think that it’s easy to just skim over that because, “No, it’s just look at it in this small example” and then, “No, but the big example, it’s different”. It actually is different.

BE: Yeah, two examples, here’s a tangible example. People making deep fakes of people at school, so teenage boys making fake nudes of female classmates. You could look at that and say, “Have you heard of Photoshop?” and you’d be right, but also wrong because Photoshop didn’t let you upload the class yearbook and spit out 1,000 hardcore porn images of all the women, all the girls in your school in an afternoon, and if you can’t do that now, you’re going to be able to do that in a year or two. That’s not the same thing and so the point being the same thing in principle, but at much greater scale actually is a change in principle, actually that becomes a different thing when you can do it at that kind of scale.

I think the other tangible example we have now is the argument about generative AI training data. There’s an awful lot of confusion about this stuff because in principle, these things are not databases. ChatGPT is not a database, and its purpose is not to contain all of the New York Times and be able to reproduce all of it at will, and it actually can’t, it literally can’t do that, that’s not how the training data functions, and so there are people who will look at this and say, “Number one, it’s not a database you don’t understand, it’s not Napster and number two, that’s how people learn.”

People will say this as though they’ve produced some incredibly brilliant and unanswerable response. “Well, people read the New York Times every day for a year, so why can’t my AI model do that?” Well, that’s true, but you’re kind of missing the point as well, which is if I have a model that just reads — if I can say to me, say to a model, “Can you tell me what the news was today? And then explain this story to me”, based on reading 20 media companies’ websites and not paying them and not sending them traffic, it’s entirely legitimate for them to say, “No, you can’t do that and we’re going to block you and if you did it without telling us anyway, we’re going to sue you.” They’re absolutely right and you can’t look at that and say, “Well, that’s just like my ten-year-old reading the newspaper every morning.” Well, no, it’s not just like your ten-year-old reading the newspaper every morning any more than somebody flowing the whole of Facebook into the whole of their school directory into a porn model and generating hardcore porn of 1000 people. “It’s the same as using Photoshop to make one.” Well, no, it’s not the same thing, it’s a new thing.

Do you remember the Sacha Baron Cohen thing about content moderation a couple of years ago?

Yep.

BE: This drove me absolutely crazy and it was hilarious, because the man’s built a whole career on satire and nuance and he’s saying, “Well, if you ran a restaurant, you wouldn’t allow somebody to say Hitler was good, therefore Facebook shouldn’t do that.” It’s not a fucking restaurant, Sacha, it’s a communications platform with several billion users. It’s also not a newspaper and it’s not a telco. You can’t say, “Well, telcos would do X, therefore Y.” It’s not a telco, it’s something else, and you have to look at it in its own right and try and work out how we think about this.

That’s a good extra one to bring on, because it’s an argument that both sides use willy-nilly. It’s very easy to cast over to the singular example and then say, “Oh, it’s the same thing at scale”, and it’s just not.

AI Frameworks

We should go back to our roots here, where we started out. Mobile is big, it’s all about the fight, Android versus Apple. I think Windows phone, there’s Nokia, Samsung, all these sorts of things, BlackBerry.

BE: Younger people on the call, Google what Blackberry was.

(laughing) That is the example of the integrated product that went down because they couldn’t give up their integration. People misapply their Apple analysis to the wrong company, but I digress.

The point though is when new paradigms come along, new platform shifts, I think is the word that you like to use, there’s often, usually always a new winner. Now, what’s interesting is that winner is not necessarily a new company, mobile was a good example of that. Apple and Google are the big winners, they were not new companies, but Microsoft was top of the world before then, or Google or Facebook, however you want to frame it.

Who’s the big winner here? Then the counterpart is, there’s often a big loser, someone falls off the map. Nokia falls off the map, BlackBerry falls off the map with mobile, Yahoo falls off the map with social, you can sort of go back through time, you can argue Apple fell off the map in some respects with the later stages of the PC, where do you see this playing out with the big companies today?

BE: So there’s several deterministic frameworks here, or historical frameworks. One of them is the new thing is generally not the old thing, but better.

There’s the stages of cycle. To begin with, you’ve got ten people fighting. It’s Commodore and Apple and PC and everything else and then you’ve got a period where there’s a couple of winners or maybe one winner. At that point, the network effects really kick in and it becomes very hard to break in. This is the same thing with MySpace. People will always point to MySpace to say, “Well, Facebook’s vulnerable” — that’s like pointing to Commodore to say that Microsoft is vulnerable.

It’s a great analogy.

BE: There’s an early period when you’ve got lots of contenders, and then there’s a period when it’s done and there’s no market entry opportunity, which was Steve Jobs’s mistake with NeXT.

Then the new thing is something else that doesn’t do what you were doing. Microsoft didn’t do mainframes, Google didn’t do PC, operating systems and so on, Facebook didn’t do search. Imagine it being like 2005 people saying, “Well, Facebook’s obviously not going to work because Google’s got all the data.” The new thing is something else, and it comes from a different place.

On the other hand, sometimes you go through a platform shift and people make the jump. So Google made the jump to mobile.

On the other hand, the old thing generally doesn’t get killed. IBM is still around. It’s still a big company, it’s just irrelevant. Microsoft is still around, for ten years, it was irrelevant, Microsoft became the new IBM, it was this big company that did enterprise stuff, it was not relevant to anything that anybody was doing in tech. They managed to jump on and grab onto generative AI and come back to relevance but the last time anyone started a company to build Windows software was like 2002 or something.

Right. 2002 might be generous, but yes.

BE: But the point is the new thing isn’t a better version of the old thing, it also doesn’t kill the old thing, the old thing continues. You can get too deterministic about these frameworks because Google carries on, and then for this Facebook, and Facebook is another thing, and it doesn’t compete directly with Google, it’s a different thing. So that’s one very schematic observation.

Another one might be that everything is a new thing, is probably disruptive to someone. Online flight booking completely obliterates the travel agent business, it kind of changes how airlines work, but it doesn’t really disrupt airlines. The business of airlines doesn’t change because of the web. How they sell tickets and price flights and stuff changes, but the airline businesses of owning or leasing airplanes, owning flight landing slots, buying fuel, that’s it.

And so as we look at this stuff — this is a thing that I have in draft — I think there’s a very primal question about how generative AI evolves, I think, which is not an answer to your question, but a response to it, which is, is this stuff going to be a generalized solution? There we are, 48 minutes in, we’ve got to the only important question in tech. Is this going to be a step change in generalization, or is it going to be embedded within hundreds and thousands of different pieces of software, which is what happened with the last wave of machine learning. Your iPhone has 100 different machine learning models on it, and you don’t see most of them.

Now, embedded in a Bill Gates quote from this time last year, where he said, “The only two revolutionary demos I’ve seen are the GUI and ChatGPT”, and I think what he’s getting at here is that before the GUI, you needed a command line. With a GUI, you don’t need to learn what to type. So you have this order of magnitude change in how much software you can have and how many people can use it. Much more important than the web or mobile, actually, which is an interesting idea, but it’s a fundamental change.

Now, the potential change with ChatGPT, with LLMs generically is, you want to do your taxes you live in Taiwan. You’ve got a piece of software that someone has written that does the Taiwanese tax code and it’s got a GUI and you can click on it. Maybe it’s in Chinese, which is a different problem, I don’t know if you can read Chinese or not.

Fortunately, the tax form is only one page long. There’s something to learn from that, but yes.

BE: The point is, there’s a GUI to click on, for the sake of argument, and someone has to have made the GUI and made the thing that knows how the tax code works.

In theory, I could go to an LLM and say, “I need to file my taxes in the Netherlands, how do I do that?” and it goes and reads the Dutch tax office’s website and then it says, “Right, you’re going to need your mortgage statement and your pay slits, and you are going to need your rent returns, and you’re going to need copies of your bank statements and your brokerage report and upload all of these, and then I’m going to step you through five other questions. I’m going to read your photograph of your bank statement, and I’m going to parse out from that what the tax office wants, and so on and so on and so on.” This is the multimodal agent, and then submit your taxes for you, and so now your taxes are being done in software, but without anyone having to write software that knows about the tax office website.

Yeah, because it’s a generalizable AI.

BE: A massive step change in generalization, hence to Bill Gates’s point, on a par with the GUI.

I’m skeptical about that, which is a whole hour podcast, but that thesis also raises questions for everybody. I don’t think anyone knows what the answers to those would be, or indeed if anybody knows what questions flow out of that yet.

One of the ways I was thinking about this is there’s classes of question here, which are questions for WPP. There’s classes of question that are questions for Accenture or Infosys, and there’s classes of questions that are questions for Bain, BCG, McKinsey. We are a large CPG company, “What does everything I’ve just said mean for us?”. Well, some of it is an Accenture question, “How do you think about your internal IT processes?”, some of it is a WPP question, “How do you tell your story as a brand and how do you think about the fact this might mean there’s going to be a hundred more brands and how do you think about the shift away from physical retail to E-commerce?”, that’s a WPP kind of question. But there’s also, “What does this mean for our supply chain and our manufacturing and the way people might choose how to buy makeup and what it means to be a CPG company?”, is that a WPP question or a Bain question?

Well, isn’t there a also philosophical question? Why do we exist?

BE: Yeah. Well, I mean there’s this kind of classic cliché of talking about Kodak, but I think about Kodak versus Fuji. With Kodak, the dumb Kodak story is that Kodak invents digital cameras in the 70s. If you look at the prototype, it records on an audio cassette, it takes a five pixel by five pixel image or something, it’s ridiculous, they didn’t miss it, there was no way that was a consumer product in 1975 or whenever it was and actually when digital cameras do become a thing, Kodak dives in with both feet. At one point, I think Kodak was the biggest vendor of digital cameras in the USA. The problem is that they have no technology advantage and it’s a low-volume, high-margin consumer electronics product where East Asian consumer electronics companies, of course, going to eat the whole market. Meanwhile, Kodak thought they were going to make loads of money because people would print the pictures.

Right.

BE: So you can actually argue it wasn’t the smartphone camera that killed them, it was a smartphone screen that killed them, and it was Instagram that killed Kodak, not digital cameras. Meanwhile, Fuji says, “We’re a specialty chemicals company”, and so they go into films and emulsions and specialty chemicals. Kodak says, “We’re a camera company. We make images, we make pictures of people’s kids.” Fuji says, “We’re a specialty chemicals company”.

So that question of what is it that we are and what does this mean? Of course it’s a lot easier to say that in hindsight. I mean, the classic thing is newspapers. Newspapers looked at the Internet and thought, “This is great, we’ll save money on printing bills”. Well, they were right, but not for the reason.

(laughing) It turns out that was their entire business.

BE: But the point is what they saw was the expertise and the writing and the journalism and everything else, and they kind of forgot that they were a light manufacturing and trucking company, and it was a light manufacturing and trucking that was the moat that protects the business and that means that suddenly they’ve got a million competitors.

So anyway, this is the problem, it’s always easy to do this in hindsight. There’s a great observation from E. H. Carr about the Russian Revolution that the Bolsheviks were obsessed with history and learning the lessons of all the failed revolutions in the past, and so they look at, one of the things they’re really afraid of is what they call Bonapartism, which is that there will be some charismatic general who will take over, which is what happens in the French Revolution. So they look at Trotsky and they think, “Aha, charismatic general, careful”. Meanwhile, there’s this nice, safe, boring stolid bureaucrat with no charisma called Joseph, something something, Stalin, and he seems safe, and of course he shoots them all in the end. So learning lessons from history and looking at the patterns of how this stuff changes, it’s like, yeah, it’s like the Delphic Oracle. King Croesus goes to Delphi and says, “If I invade the Persian Empire what will happen?”, and the Oracle says, “You’ll destroy a great empire”. He thinks, “Great!”. Guess what?

It was your empire.

BE: “Which empire was he talking about?”

LLMs and Bubbles

Is there a possibility that the whole LLM thing maybe ends up being a bit of a dead end and the outcome of that is actually a dot com like overbuild and overhang where you got all this broadband, you got all this dark fiber that actually enables Web 2.0 and the first version of generative content — but it was user generated in this case — is there some case where we just end up with all these GPUs out there and that is what actually what matters most about this era? Or have we figured it out and we just need to scale it up?

BE: Yeah, I mean, if we’re not in an AI bubble yet, we will be because that’s just the nature of the world. I think, and there’s certainly bubbly thinking, I’ve lost interest in engaging with AGI people because I just lose track of the logical fallacies and the way anytime you ask them a question, they change the subject and run away. I think this question of, “Is this generalized or does it get subsumed into everything?”, that to me is almost more important than the, “Do we have 5 models or 25 or 100?”, which is the other kind of challenge. Databricks has a new model, I’ve read Databricks’ website, I think Andreessen Horowitz even invested in them, I have no idea what Databricks does, it does something cloudy in the enterprise.

They have a lot of data, so it makes sense.

BE: I have no idea, they do bricks with data. They have their own LLM, which appears to be as good as anybody else’s, so there’s clearly not going to be three LLMs on Earth, whether there’ll be a hundred on a few nodes, but there’s not going to be three.

What is that level of generalization? Does this evolve kind of like the last wave of machine learning where it transforms what you can do with software and then it kind of disappears? It’s like the old AI’s, whatever, it doesn’t work yet. Once it works, it’s just software.

Or is this a fundamental step change in how we use these things? This is the thing you have now. It was in a conversation a couple of weeks ago, someone was talking about TV metrics, streaming TV metrics, and they’re talking about a platform that I think it installs on smart TVs and looks at the video streaming, works out what TV show you’re watching and he says, “Well, obviously that’s not AI, that’s just video recognition”. And I thought, “Okay, let me just stop you for a minute there”.

Well, the funny thing about this is with machine learning, it always sort of moved backwards. Nothing is ever AI but post-ChatGPT, it flipped. Now everything is AI, so we can’t decide it’s either one extreme or the other, that’s a perfect example. That’s moving the goal posts in one direction and then the other side goes in the other.

But if you’re building products, I think you wrote this a little bit ago. You had a great article about AI and automation, that it takes a long time to build products, we don’t even know what the products are yet, and then they have to actually built and they have to actually be distributed, they actually make their way out. It sounds like you’re leaning towards the product story, which is kind of more of a long run story or maybe they’re both long run, I guess we’ll have to wait and see.

BE: So the thing I’m sort of puzzling about, and this is kind of a question for you really, is — so this is the piece that I have in draft — is imagine looking at a PC in the late 70s or early 80s and someone shows you VisiCalc and you are not an accountant and you don’t work with numbers. So if you’re an accountant, you see Dan Bricklin stories, VisiCalc is the first computer spreadsheet, he shows this to accountants and they lose bowel control, they’re like, “Oh my gosh”.

(laughing) I’m done.

BE: You’ve just done a week of work in an hour and they’re all these stories of accountants who would buy this thing and use it and they’d do weeks of work in days and you needed the Apple II to run it with enough RAM, and it was like seven or eight grand in today’s money. So if you were an accountant and you’re like, “You have to have this”, if you are a lawyer or doctor, you think, well, “Maybe my bookkeeper would like that”, maybe, but I don’t do numbers all day, I don’t do numbers, I do words, I do buildings, I’m a graphic designer. For the sake of argument, you could see it and think, “Okay, I get why that would be amazingly cool if you did that, but I don’t do that.”

That’s how I feel when I see people talking about coding with ChatGPT. That’s great, but I don’t write code. I redesigned my website two or three years ago and I had to write a lot of custom CSS and it was a nightmare. I did a lot of copy-pasting from Stack Overflow, and then it would’ve been great to have ChatGPT. Today, I have no use for it. I have not worked out a thing that I do where ChatGPT is useful and would help me for something, and that to me is looking at VisiCalc as a lawyer. There’s four or five things where people find this incredibly useful and if you don’t do any of those things and you’re kind of looking at it going, “Okay, I’m waiting for someone to make AutoCAD”.

I mean, I’m curious, my experience is basically you keep using it, there’s nothing that I do where this is useful for me right now, and I’m curious about your experience of it. How do you use these things?

I’ve used it a bit more in the last couple of months, but mostly for Wikipedia-type look-ups or looking up arcane facts where I know I could get it on Google, but I know I’d have to click on four or five pages and dig down to get it. But even then, there still is the overhang, which is if there’s something I want to use in an Article, I still feel I need to verify it, I’m not ready to totally trust it and trust my reputation to it.

It does strike me as this is a product opportunity, which is, and I lean towards your view of the product view and a huge value for humans is being a trust interface. It’s humans-as-API where this idea of, “I want to buy insurance, or, “I want to buy a travel ticket”, but I’m not going to trust the AI to do it right. But if there’s someone in front of it that is assuring me that it’s being done right, then maybe that’s more interesting to me and I’m outsourcing not just sort of the management of it, but the trust factor. But that’s a product story, it’s not a generalizable story.

BE: Years ago, someone pointed out to me something, I was very annoyed at myself for not realizing it, that Google Search is manually curated, Google doesn’t give you the answer, it gives you ten links and ask you to pick the right one.

That’s right.

BE: Google doesn’t say, “This is the answer”, and there’s a product problem here in that an LLM says, “This is the answer”. Or at least as they’re currently constituted, they do.

This was the Gemini problem, in that it was giving you one answer, so the sense of burden of proof, Google took that on, as opposed to it’s always been on the user previously.

BE: So it’s not communicated, how do you communicate the level of uncertainty? Some of this is also in prose, I think, that because it produces perfect prose, that masks the fact that the models behind it might be wrong.

Right.

BE: So the prose is always beautiful. It’s like if you make, and you see this, it’s better to see this in an image generator, you can make go to Midjourney and it’ll make your picture and the perspective, the lighting, color, the focus, everything is perfect. One of the people in the picture has three legs, and it’s like that’s the equivalent of the factual error in a ChatGPT response but the perception of it is it’s easier to understand what’s going on.

But like 98%, what does 95% right mean? Well, it means that every now and then somebody has three legs, it means that every now and then it gets this wrong. So there’s a whole product question around how you present that to people and how you use it, but I think there’s of another thing, there’s a deeper point here, which is what we’re both describing is only one use case, which is, “I’m going to ask, I’m going to treat this as some kind of knowledge engine”.

Well, the other one that’s interesting is I’m seeing what Palantir is doing with their LLM stuff, but for them it’s just a user interface on top of structured data, and the data that comes out is for sure right because it is structured and it’s inverting this sort of concept.

BE: So WPP had an investor date last month I think, and they showed their internal dashboard where you work at WPP and you can pick four or five different models and four or five different image generators, and the model is like you can select, “So I want a model trained on Nike’s tone of voice and I want the model trained on talking to Gen X”. Theoretically I could plug all of that into, theoretically you could do it with prompt engineering, but in practice, no. What you want is you want a button. I want a button and say, I want to pick the client and I want to pick the target demo and I don’t want to do prompt engineering. I mean this is the kind of hilarious thing about prompt engineering is prompt engineering is a command line.

It’s so true, it’s full circle.

BE: The moment you are telling me this is how to ask it for what you want, you’re doing prompt. You’re doing a command line. The other thing, analogy occurred to me listening to you talk a minute ago, is remember when we first started using Google, you kind of had to form a habit of realizing, “Oh, I could use Google for that”.

Then people would say, “But you can’t necessarily trust what’s on the Internet”. But you start thinking, “Well, okay, am I going to trust this source for this kind of question? Yes or no?”, and of course you could argue a big problem in American politics is people not having that layer of critical scrutiny over what they’re being shown.

So a lot of this is like is this the classic response of the incumbent seeing the thing and saying it sucks. It’s the car company looking at the Tesla and saying, “Yeah, but the doors fall off”, or Nokia looking at the iPhone and saying, “Yeah, but you can’t use it with your gloves and it breaks if you drop it and the radio is bad”. It’s like, yeah, the radio will get better and it doesn’t matter that it breaks if you drop it because of X and Y and Z, or is this like looking at a Tesla now and saying it’s been ten years and the doors still fall off, so that excuse doesn’t work anymore.

Well, you and I are both writers, so I guess we are inherently biased in this regard, but I don’t know, I still feel safe in my job. This was amazing, we’re five minutes over, we could go on much longer, I would love to dive in more of these questions. The website is ben-evans.com, you do both columns and the newsletter. The newsletter also has columns in it, so you have to make sure you consume both.

BE: Yeah, so I write long form stuff on my website and then I do a weekly newsletter which has a short column about something that’s interesting, hopefully every week, and I do other things as well. But yes, Google me, Google “Benedict Evans”, my parents had good SEO.

I’m jealous of Benedict. Benjamin, very boring. But it was great to talk to you, I’d love to have you on again. Thanks, and it’s great to catch up.

BE: Sure, great to chat. Thanks.

---

This Daily Update Interview is also available as a podcast. To receive it in your podcast player, visit Stratechery.

The Daily Update is intended for a single recipient, but occasional forwarding is totally fine! If you would like to order multiple subscriptions for your team with a group discount (minimum 5), please contact me directly.

Thanks for being a supporter, and have a great day!