Meta and Open

Posted on

Apple released the Vision Pro on February 2; 12 days later Meta CEO Mark Zuckerberg delivered his verdict:

Alright guys, so I finally tried Apple’s Vision Pro. And you know, I have to say that before this, I expected that Quest would be the better value for most people since it’s really good and it’s like seven times less expensive. But after using it I don’t just think that Quest is the better value — I think the Quest is the better product, period.

You can watch the video for Zuckerberg’s full — and certainly biased! — take, but the pertinent section for this Article came towards the end:

The reality is that every generation of computing has an open and a closed model, and yeah, in mobile, Apple’s closed model won. But it’s not always that way. If you go back to the PC era, Microsoft’s open model was the winner, and in this next generation, Meta is going to be the open model, and I really want to make sure that the open model wins out again. The future is not yet written.

John Gruber asked on Daring Fireball:

At the end, he makes the case that each new generation of computing devices has an open alternative and a closed one from Apple. (It’s interesting to think that these rivalries might be best thought of not as closed-vs.-open, but as Apple-vs.-the-rest-of-the-industry.) I’m not quite sure where he’s going with that, though, because I don’t really see how my Quest 3 is any more “open” than my Vision Pro. Are they going to license the OS to other headset makers?

Cue Zuckerberg yesterday:

Some updates on the metaverse today. We are releasing Meta Horizon OS, our operating system that powers Quest virtual and mixed reality headsets, and we are partnering with some of the best hardware companies out there to design new headsets that are optimized for all the different ways that people use this tech.

Now, in every era of computing, there are always open and closed models. Apple’s closed model basically went out. Phones are tightly controlled and you’re kind of locked into what they’ll let you do. But it doesn’t have to be that way. In the PC era, the open model won out. You can do a lot more things, install mods. You got more diversity of hardware, software and more. So our goal is to make it so the open model defines the next generation of computing again with the metaverse, glasses and headsets. That’s why we’re releasing our operating systems so that more companies can build different things on it.

It’s natural to view this announcement as a reaction to the Vision Pro, or perhaps to Google’s upcoming AR announcment at Google I/O, which is rumored to include a new Samsung headset. However, I think that this sells Zuckerberg and Meta’s strategic acumen short: this is an obvious next step, of a piece with the company’s recent AI announcements, and a clear missing piece in the overall metaverse puzzle.

Meta’s Market

Any question of strategy starts with understanding your market, so what is Meta’s? This is a trickier question than you might think, particularly on the Internet. It’s a definition that has particularly vexed regulators, as I laid out in Regulators and Reality; after describing why the FTC’s extremely narrow definition of “personal social networking” — which excluded everything from Twitter to Reddit to LinkedIn to TikTok to YouTube as Facebook competitors — didn’t make sense, I explained:

The far bigger problem, though, is that everything I just wrote is meaningless, because everything listed above is a non-rivalrous digital service with zero marginal costs and zero transactional costs; users can and do use all of them at the same time. Indeed, the fact that all of these services can and do exist for the same users at the same time makes the case that Facebook’s market is in fact phenomenally competitive.

What, though, is Facebook competing for? Competition implies rivalry, that is, some asset that can only be consumed by one service to the exclusion of others, and the only rivalrous good in digital services is consumer time and attention. Users only have one set of eyes, and only 24 hours in a day, and every second spent with one service is a second not spent with another (although this isn’t technically true, since you could, say, listen to one while watching another while scrolling a third while responding to notifications from a fourth, fifth, and sixth). Note the percentages in this chart of platform usage:

Users spend more time on TikTok than other social media platforms

The total is not 100, it is 372, because none of these services exclude usage of any of the others. And while Facebook is obviously doing well in terms of total users, TikTok in particular looms quite large when it comes to time, the only metric that matters:

Most American adults use multiple online services

This, of course, is why all of these services, including Instagram, Snapchat, and YouTube are trying to mimic TikTok as quickly as possible, which, last time I checked, is a competitive response, not a monopolistic one. You can even grant the argument that Facebook tried to corner the social media market — whatever that is — a decade ago, but you have to also admit that here in 2021 it is clear that they failed. Competition is the surest sign that there was not actually any anticompetitive conduct, and I don’t think it is the FTC’s job to hold Facebook management accountable for failing to achieve their alleged goals.

This idea that time and attention is the only scarce resource on the Internet, and thus the only market that truly matters, is what undergirds Netflix’s shift in reporting away from members and towards engagement; the company has been saying for years that its competitors were not just other streaming services, but everything from YouTube to Twitch streaming to video games to social media. That has always been true, if you squint, but on the Internet, where everything is only a click (or app) away, it’s tangible.

Meta’s Differentiation

Defining the relevant market as time and attention has surprising implications, that even companies raised on the Internet, including Meta, sometimes miss. Indeed, there was a time when Meta might have agreed with the FTC’s definition, because the company made competitive decisions — both big successes and big mistakes — predicated on the assumption that your personal social network was the market that mattered.

Start with an Article I wrote in 2015 entitled Facebook and the Feed:

Zuckerberg is quite clear about what drives him; he wrote in Facebook’s S-1:

Facebook was not originally created to be a company. It was built to accomplish a social mission – to make the world more open and connected.

I am starting to wonder if these two ideas — company versus mission — might not be more in tension now than they have ever been in the past…I suspect that Zuckerberg for one subscribes to the first idea: that people find what others say inherently valuable, and that it is the access to that information that makes Facebook indispensable. Conveniently, this fits with his mission for the company. For my part, though, I’m not so sure. It’s just as possible that Facebook is compelling for the content it surfaces, regardless of who surfaces it. And, if the latter is the case, then Facebook’s engagement moat is less its network effects than it is that for almost a billion users Facebook is their most essential digital habit: their door to the Internet.

A year later and Facebook responded to what was then its most pressing threat, Snapchat, by putting Stories into Instagram. I wrote in The Audacity of Copying Well:

For all of Snapchat’s explosive growth, Instagram is still more than double the size, with far more penetration across multiple demographics and international users. Rather than launch a “Stories” app without the network that is the most fundamental feature of any app built on sharing, Facebook is leveraging one of their most valuable assets: Instagram’s 500 million users.

The results, at least anecdotally, speak for themselves: I’ve seen more Instagram stories in the last 24 hours than I have Snapchat ones. Of course a big part of this is the novelty aspect, which will fade, and I follow a lot more people on Instagram than I do on Snapchat. That last point, though, is, well, the point: I and my friends are not exactly Snapchat’s target demographic today, but for the service to reach its potential we will be eventually. Unless, of course, Instagram Stories ends up being good enough.

It was good enough — Instagram arrested Snapchat’s growth, while boosting its own engagement and user base — so score one for Zuckerberg, right? Instagram had a better network, so they won…or did they simply have more preexisting usage, which while based on a network, was actually incidental to it?

Fast forward a few years and now Facebook’s big competitor was TikTok; I wrote in 2020’s The TikTok War:

All of this explains what makes TikTok such a breakthrough product. First, humans like video. Second, TikTok’s video creation tools were far more accessible and inspiring for non-professional videographers. The crucial missing piece, though, is that TikTok isn’t really a social network…by expanding the library of available video from those made by your network to any video made by anyone on the service, Douyin/TikTok leverages the sheer scale of user-generated content to generate far more compelling content than professionals could ever generate, and relies on its algorithms to ensure that users are only seeing the cream of the crop.

In a follow-up Update I explained why this was a blindspot for Facebook:

First, Facebook views itself first-and-foremost as a social network, so it is disinclined to see that as a liability. Second, that view was reinforced by the way in which Facebook took on Snapchat. The point of The Audacity of Copying Well is that Facebook leveraged Instagram’s social network to halt Snapchat’s growth, which only reinforced that the network was Facebook’s greatest asset, making the TikTok blindspot even larger.

I am, in the end, actually making the same point as the previous section: Meta’s relevant market is user time and attention; it follows that Meta’s differentiation is the fact it marshals so much user time and attention, and that said marshaling was achieved via social networking is interesting but not necessarily strategically relevant. Indeed, Instagram in the end simply copied TikTok, surfacing content from anywhere on your network, and did so to great success.

Llama 3

This is the appropriate framework to understand Meta’s AI strategy with its Llama family of models: Llama 3 was released last week, and like Llama 2, it is open source, or, perhaps more accurately, open weights (with the caveat that hyperscalers need a license to offer Llama as a managed model). I explained why open weights makes sense in a May 2023 Update predicting the Llama 2 release:

Meta isn’t selling its capabilities; rather, it sells a canvas for users to put whatever content they desire, and to consume the content created by other users. It follows, then, that Meta ought to be fairly agnostic about how and where that content is created; by extension, if Meta were to open source its content creation models, the most obvious place where the content of those models would be published is on Meta platforms. To put it another way, Meta’s entire business is predicated on content being a commodity; making creation into a commodity as well simply provides more grist for the mill.

What is compelling about this reality, and the reason I latched onto Zuckerberg’s comments in that call, is that Meta is uniquely positioned to overcome all of the limitations of open source, from training to verification to RLHF to data quality, precisely because the company’s business model doesn’t depend on having the best models, but simply on the world having a lot of them.

The best analogy for Meta’s approach with Llama is what the company did in the data center. Google had revolutionized data center design in the 2000s, pioneering the use of commodity hardware with software-defined functionality; Facebook didn’t have the scale to duplicate Google’s differentiation in 2011, so it went in the opposite direction and created the Open Compute Project. Zuckerberg explained what happened next in an interview with Dwarkesh Patel:

We don’t tend to open source our product. We don’t take the code for Instagram and make it open source. We take a lot of the low-level infrastructure and we make that open source. Probably the biggest one in our history was our Open Compute Project where we took the designs for all of our servers, network switches, and data centers, and made it open source and it ended up being super helpful. Although a lot of people can design servers the industry now standardized on our design, which meant that the supply chains basically all got built out around our design. So volumes went up, it got cheaper for everyone, and it saved us billions of dollars which was awesome.

Zuckerberg then made the analogy I’m referring to:

So there’s multiple ways where open source could be helpful for us. One is if people figure out how to run the models more cheaply. We’re going to be spending tens, or a hundred billion dollars or more over time on all this stuff. So if we can do that 10% more efficiently, we’re saving billions or tens of billions of dollars. That’s probably worth a lot by itself. Especially if there are other competitive models out there, it’s not like our thing is giving away some kind of crazy advantage.

It’s not just about having a better model, though: it’s about ensuring that Meta doesn’t have a dependency on any one model as well. Zuckerberg continued:

Here’s one analogy on this. One thing that I think generally sucks about the mobile ecosystem is that you have these two gatekeeper companies, Apple and Google, that can tell you what you’re allowed to build. There’s the economic version of that which is like when we build something and they just take a bunch of your money. But then there’s the qualitative version, which is actually what upsets me more. There’s a bunch of times when we’ve launched or wanted to launch features and Apple’s just like “nope, you’re not launching that.” That sucks, right? So the question is, are we set up for a world like that with AI? You’re going to get a handful of companies that run these closed models that are going to be in control of the APIs and therefore able to tell you what you can build?

For us I can say it is worth it to go build a model ourselves to make sure that we’re not in that position. I don’t want any of those other companies telling us what we can build. From an open source perspective, I think a lot of developers don’t want those companies telling them what they can build either. So the question is, what is the ecosystem that gets built out around that? What are interesting new things? How much does that improve our products? I think there are lots of cases where if this ends up being like our databases or caching systems or architecture, we’ll get valuable contributions from the community that will make our stuff better. Our app specific work that we do will then still be so differentiated that it won’t really matter. We’ll be able to do what we do. We’ll benefit and all the systems, ours and the communities’, will be better because it’s open source.

There is another analogy here, which is Google and Android; Bill Gurley wrote the definitive Android post in 2011 on his blog Above the Crowd:

Android, as well as Chrome and Chrome OS for that matter, are not “products” in the classic business sense. They have no plan to become their own “economic castles.” Rather they are very expensive and very aggressive “moats,” funded by the height and magnitude of Google’s castle. Google’s aim is defensive not offensive. They are not trying to make a profit on Android or Chrome. They want to take any layer that lives between themselves and the consumer and make it free (or even less than free). Because these layers are basically software products with no variable costs, this is a very viable defensive strategy. In essence, they are not just building a moat; Google is also scorching the earth for 250 miles around the outside of the castle to ensure no one can approach it. And best I can tell, they are doing a damn good job of it.

The positive economic impact of Android (and Chrome) is massive: the company pays Apple around $20 billion a year for default placement on about 30% of worldwide smartphones (and Safari on Apple’s other platforms), which accounts for about 40% of the company’s overall spend on traffic acquisition costs across every other platform and browser. That total would almost certainly be much higher — if Google were even allowed to make a deal, which might not be the case if Microsoft controlled the rest of the market — absent Android and Chrome.

Metaverse Motivations

Android is also a natural segue to this news about Horizon OS. Meta is, like Google before it, a horizontal services company funded by advertising, which means it is incentivized to serve everyone, and to have no one between itself and its customers. And so Meta is, like Google before it, spending a huge amount of money to build a contender for what Zuckerberg believes is a future platform. It’s also fair to note that Meta is spending a lot more than the $40 billion Google has put into Android, but I think it’s reasonable: the risk — and opportunity — for Meta in the metaverse is even higher than the risk Google perceived in smartphones.

Back in 2013, when Facebook was facing the reality that mobile was ending its dreams of being a platform in its own right, I wrote Mobile Makes Facebook Just an App; That’s Great News:

First off, mobile apps own the entire (small) screen. You see nothing but the app that you are using at any one particular time. Secondly, mobile apps are just that: apps, not platforms. There is no need for Facebook to “reserve space” in their mobile apps for partners or other apps. That’s why my quote above is actually the bull case for Facebook.

Specifically, it’s better for an advertising business to not be a platform. There are certain roles and responsibilities a platform must bear with regards to the user experience, and many of these work against effective advertising. That’s why, for example, you don’t see any advertising in Android, despite the fact it’s built by the top advertising company in the world. A Facebook app owns the entire screen, and can use all of that screen for what benefits Facebook, and Facebook alone.

This optimism was certainly borne out by Facebook’s astronomical growth over the last decade, which has been almost entirely about exploiting this mobile advertising opportunity. It also, at first glance, calls into question the wisdom of building Horizon OS, given the platform advertising challenges I just detailed.

The reality, though, is that a headset is fundamentally different than a smartphone: the latter is something you hold in a hand, and which an app can monopolize; the former monopolizes your vision, reducing an app to a window. Consider this PR image of the Vision Pro:

Meta doesn't want to be just an app

This isn’t the only canvas for apps in the Vision Pro: apps can also take over the entire view and provide an immersive experience, and I can imagine that Meta will, when and if the Vision Pro gains meaningful marketshare, build just that; remember, Meta is a horizontal services company, and that means serving everyone. Ultimately, though, Zuckerberg sees the chief allure of the metaverse as being about presence, which means the sensation of feeling like you are in the same place as other people enjoying the same experiences and apps; that, by extension, means owning the layer within which apps live — it means owning your entire vision.

Just as importantly — probably most importantly, to Zuckerberg — owning the OS means not being subject to Apple’s dictate on what can or cannot be built, or tracked, or monetized. And, as Zuckerberg noted in that interview, Meta isn’t particularly keen to subject itself to Google, either. It might be tempting for Meta’s investors to dismiss these concerns, but ATT should have focused minds about just how much this lack of control can cost.

Finally, not all devices will be platforms: Meta’s RayBan sunglasses, for example, could not be “just an app”; what they could be is that much better of a product if Apple made public the same sort of private APIs it makes available to its own accessories. Meta isn’t going to fix its smartphone challenges in that regard, but it is more motivation to do their own thing.

Horizon OS

Motivations, of course, aren’t enough: unlike AI models, where Meta wants a competitive model, but will achieve its strategic goals as long as a closed model doesn’t win, the company does actually need to win in the metaverse by controlling the most devices (assuming, of course, that the metaverse actually becomes a thing).

The first thing to note is that pursuing an Apple-like fully-integrated model would actually be bad for Meta’s larger goals, which, as a horizontal services company, is reaching the maximum number of people possible; there is a reason that the iPhone, by far the most dominant integrated product ever, still only has about 30% marketshare worldwide. Indeed, I would pushback on Zuckerberg’s continued insistence that Apple “won” mobile: they certainly did as far as revenue and profits go, but the nature of their winning is not the sort of winning that Meta should aspire to; from a horizontal services company perspective, Android “won” because it has the most marketshare.

Second, the best route to achieving that marketshare is exactly what Meta announced: licensing their operating system to device manufacturers who are not only motivated to sell devices, but also provide the necessary R&D and disparate channels to develop headsets for a far wider array of customers and use cases.

Third, Meta does have the opportunity to actually accomplish what pundits were sure would befall the iPhone: monopolize developer time and attention. A big reason why pundits were wrong about the iPhone, back when they were sure that it was doomed to disruption, was that they misunderstood history. I wrote in 2013’s The Truth About Windows Versus the Mac:

You’ve heard the phrase, “No one ever got fired for buying IBM.” That axiom in fact predates Microsoft or Apple, having originated during IBM’s System/360 heyday. But it had a powerful effect on the PC market. In the late 1970s and very early 1980s, a new breed of personal computers were appearing on the scene, including the Commodore, MITS Altair, Apple II, and more. Some employees were bringing them into the workplace, which major corporations found unacceptable, so IT departments asked IBM for something similar. After all, “No one ever got fired…”

IBM spun up a separate team in Florida to put together something they could sell IT departments. Pressed for time, the Florida team put together a minicomputer using mostly off-the shelf components; IBM’s RISC processors and the OS they had under development were technically superior, but Intel had a CISC processor for sale immediately, and a new company called Microsoft said their OS — DOS — could be ready in six months. For the sake of expediency, IBM decided to go with Intel and Microsoft.

The rest, as they say, is history. The demand from corporations for IBM PCs was overwhelming, and DOS — and applications written for it — became entrenched. By the time the Mac appeared in 1984, the die had long since been cast. Ultimately, it would take Microsoft a decade to approach the Mac’s ease-of-use, but Windows’ DOS underpinnings and associated application library meant the Microsoft position was secure regardless.

Evans is correct: the market today for mobile phones is completely different than the old market for PCs. And, so is Apple’s starting position; iOS was the first modern smartphone platform, and has always had the app advantage. Neither was the case in PCs. The Mac didn’t lose to Windows; it failed to challenge an already-entrenched DOS. The lessons that can be drawn are minimal.

The headset market is the opposite of the smartphone market: Meta has been at this for a while, and has a much larger developer base than Apple does, particularly in terms of games. It’s not overwhelming like Microsoft’s DOS advantage already was, to be sure, and I’m certainly not counting out Apple, but this also isn’t the smartphone era where Apple had a multi-year head start.

To that end, it’s notable that Meta isn’t just licensing Horizon OS, it is also opening up the allowable app model. From the Oculus Developer blog:

We’re also significantly changing the way we manage the Meta Horizon Store. We’re shifting our model from two independent surfaces, Store and App Lab, to a single, unified, open storefront. This shift will happen in stages, first by making many App Lab titles available in a dedicated section of the Store, which will expand the opportunity for those titles to reach their audiences. In the future, new titles submitted will go directly to the Store, and App Lab will no longer be a separate distribution channel. All titles will still need to meet basic technical, content, and privacy requirements to publish to the Store. Titles are reviewed at submission and may be re-reviewed as they scale to more people. Like App Lab today, all titles that meet these requirements will be published.

App Lab apps are a middle ground between side-loading (which Horizon OS supports) and normal app store distribution: developers get the benefit of an App Store (easy install, upgrades, etc.) without having to go through full App Review; clear a basic bar and your app will be published. This allows for more experimentation.

What it does not allow for is new business models: App Lab apps, if they monetize, still must use Horizon OS’s in-app payment system. To that end, I think that Meta should consider going even further, and offering up a truly open store: granted, this will reduce the long-run monetization potential of Horizon OS, but it seems to me like that would be an excellent problem to have, given it would mean there was a long-run to monetize in the first place.

The Meaning of Open

This remaining limitation does get at the rather fuzzy meaning of “open”: in the case of Horizon OS, Meta means a licensing model for its OS and more freedom for developers relative to Apple; in the case of Llama Meta means open weights and making models into a commodity; in the case of data centers Meta means open specifications, and in the case of projects like React or PyTorch Meta means true open source code.

Meta, in other words, is not taking some sort of philosophical stand: rather, they are clear-eyed about what their market is (time and attention), and their core differentiation (horizontal services that capture more time and attention than anyone); everything that matters in pursuit of that market and maintenance of that differentiation is worth investing in, and if “openness” means that investment goes further or performs better or handicaps a competitor, then Meta will be open.

MKBHDs For Everything

Posted on

Anniversaries are strange: no one but those involved have any reason to even know they exist, which makes them easy to skip; for example, last year I mostly ignored the 10 year anniversary of starting Stratechery because I was more eager to write about ChatGPT Getting a Computer. That leaves today, April 16, as my last chance to mark a tin anniversary: ten years ago Stratechery added a subscription option, making this my full-time job.

It was, for the record, a miserable day: I had a convoluted plan to offer different site experiences for subscribers and non-subscribers: the former would receive daily linked-list updates (a la Daring Fireball) while the latter would still have access to longer-form Articles; I scrapped the concept two days later in favor of simply sending subscribers a daily email, and so the paid newsletter model was born.

That accident of history ended up being why this anniversary is actually more meaningful to me: while I am most well-known for Stratechery, I am equally proud of the paid newsletter model — services like Substack were based on Stratechery — and gratified that so many writers are making a living connecting directly with readers.

Of course newsletters aren’t the only medium where this happens (and, in fact, I’ve never viewed Stratechery as a newsletter; it’s a blog that you can read via email); there are also podcasts (which Stratechery has expanded into) and YouTube (like and subscribe to the Stratechery and Sharp Tech channels!). The king of the latter, at least when it comes to tech, is Marques Brownlee, aka MKBHD.

The Humane AI Pin

Humane introduced the wearable AI Pin at a TED Talk 11 months ago, painting the vision of a future without screens, powered by AI. I was, for the record, pretty skeptical: most new tech categories layer on top of what came before, as opposed to outright replacing them; Humane’s proposal of a wearable device that was completely independent of the phone in your pocket seemed unlikely to succeed. Still, I was cheering for them: I applaud any company trying to build something new, and AI provides the opening for new experimentation.

Unfortunately, last week the reviews of the AI Pin came out and they were universally terrible; The Verge’s headline summed up the sentiment: Humane AI Pin review: not even close. David Pierce wrote:

I came into this review with two big questions about the AI Pin. The first is the big-picture one: is this thing… anything? In just shy of two weeks of testing, I’ve come to realize that there are, in fact, a lot of things for which my phone actually sucks. Often, all I want to do is check the time or write something down or text my wife, and I end up sucked in by TikTok or my email or whatever unwanted notification is sitting there on my screen. Plus, have you ever thought about how often your hands are occupied with groceries / clothes / leashes / children / steering wheels, and how annoying / unsafe it is to try to balance your phone at the same time? I’ve learned I do lots of things on my phone that I might like to do somewhere else. So, yeah, this is something. Maybe something big. AI models aren’t good enough to handle everything yet, but I’ve seen enough glimmers of what’s coming that I’m optimistic about the future.

That raises the second question: should you buy this thing? That one’s easy. Nope. Nuh-uh. No way. The AI Pin is an interesting idea that is so thoroughly unfinished and so totally broken in so many unacceptable ways that I can’t think of anyone to whom I’d recommend spending the $699 for the device and the $24 monthly subscription.

Brownlee agreed; he said at the start of his YouTube review:

So this is the Humane AI Pin. It is a brand new product in a really interesting new form factor of an ultra-futuristic wearable computer. In a time of all these crazy gadgets and Vision Pro and wearable glasses, it’s so sick that we get so many genuinely new first generation products like this to give a shot.

Unfortunately it’s also the new worst product I think I’ve ever reviewed in its current state. There’s just so many things bad about it. It’s so bad, in fact, that I actually think it’s kind of distracting to understand what the point of the device is as we go through it, so I’m going to have to separate it out for this video. First, I’m going to tell you what it is and what it’s supposed to do, and then I’ll tell you what my experience has actually been using it.

Brownlee proceeded to do just that: he presented Humane’s vision, detailed how poorly it lived up to it (including the probably fatal flaw inherent in its decision to pretend smartphones don’t exist), and granted that future software updates might improve the experience. I thought it was a fair,1 if brutal, review, but what had some people up in arms was the title and thumbnail:

I don’t tweet much these days — if you want one of my tips for lasting over a decade, that is one of them, for my own peace of mind more than anything — but Vassallo’s sentiment bothered me enough to fire off a response:

I honestly think I had this anniversary on my mind: I remember when I first started, with only a few hundred followers on Twitter, armed with nothing but my honest takes about tech; my only hope was that those takes would be original enough and valuable enough to build an audience. That they were didn’t diminish the desire to be honest; indeed, one of the biggest challenges creators face as they grow is retaining their edge even when they know people are actually paying attention to what they have to say.

MKBHD’s Responsibility

The more interesting Vassallo tweet, though, and the one that inspired this Article, was this exchange in response to his original tweet:

Ignore the vaguely threatening “for now” ; Vassallo is touching on something profound about Brownlee, that I myself understand intimately: what the Internet has made uniquely possible is total loyalty to your customers, and that is threatening.

Let me start with myself: every dollar of income I have comes from my subscribers,2 all of whom pay the same price.3 If someone doesn’t like what I write, I make it easy to unsubscribe; if they accuse me of writing for favor or illegitimate profit I am happy to unsubscribe them myself, and refund their money. After all, to use Brownlee’s reponse to Vassallo:

My job is to make my subscribers happy, and the best way I’ve found to do that is to stay as true as I can to what I set out to do in the beginning: write things that are original and valuable. When I succeed I’m happy, and the numbers take care of themselves; when I publish something I’m not happy with, I have trouble sleeping. When tech companies or investors or anyone else is mad, I am free to not pay them any attention.

Brownlee, though, is, to Vassallo’s point, something else entirely: 18 million subscribers is an incredible number, even if only — “only” — 3.5 million people have viewed his Humane video. If Humane’s AI Pin wasn’t already dead in the water, it’s fair to say that @levelsio is right:

Who, though, is to blame, and who benefited? Surely the responsibility for the Humane AI Pin lies with Humane; the people who benefited from Brownlee’s honesty were his viewers, the only people to whom Brownlee owes anything. To think of this review — or even just the title — as “distasteful” or “unethical” is to view Humane — a recognizable entity, to be sure — as of more worth than the 3.5 million individuals who watched Brownlee’s review.

This is one of the challenges of scale: Brownlee has so many viewers that it is almost easier to pretend like they are some unimportant blob. Brownlee, though, is successful because he remembers his job is not to go easy on individual companies, but to inform individual viewers who will make individual decisions about spending $700 on a product that doesn’t work. Thanks to the Internet he has absolutely no responsibility or incentive to do anything but that.

Media and the Internet

While Stratechery mostly covers tech, I also spend a lot of time on media; in the early days of Stratechery, this mostly meant newspapers, and my argument was that was in fact a particularly useful angle on the impact of technology broadly: the fact that newspapers were text meant that they felt the impact of the Internet almost immediately, effects that would, as devices became more powerful and ubiquitous, and broadband faster and more reliable, impact everything. And, of course, Stratechery was a written product, so I had a stake in the game.

To that end, I wrote a series of Articles in the run-up to April 16, 2014, laying the groundwork for why I thought my subscription offering would work. The last of these was entitled Newspapers Are Dead; Long Live Journalism, where I wrote:

Remember how the New York Times was started as a means of making money? After the first year the paper had a circulation of 26,000 in a city of over half a million, but had incurred up-front capital costs of $50,000, and first-year expenses of $78,000 (In 1851 dollars; the modern equivalent would be $1.4 million and $2.2 million respectively). Compare that to this blog (which, by happy coincidence, turns one year old tomorrow, making this comparison timely): FiveThirtyEight and the End of Average, the first article in this series, has been read by over 30,000 people; meanwhile, I’ve spent less than $2,000. More readers, way less money.

You may consider the comparison unfair — an entire newsroom putting out a daily edition as compared to a solo blogger posting one Article — but the unfairness is the point. No one shared my article because it was from Stratechery, but then again, no one shares an article today just because it’s from the New York Times; all that matters is the individual article and its worth to the reader and potential sharer. As a writer, this is amazing. When it comes to reader attention, I am competing on an equal footing with The New York Freaking Times! Unfortunately for The New York Times, when it comes to making money they’re competing with Google and Facebook. Most distressingly, though, when it comes to costs, they’re competing with the last 150 years. Everything from printing presses to sales and marketing is deadweight if advertising is not a sustainable model.

The New York Times has, to its immense credit, figured out how to go direct to readers; most other newspapers have struggled: cost structures that were predicated on owning printing presses and delivery trucks simply don’t make sense in a world where distribution is free.

Free distribution, meanwhile, made Stratechery possible: I just needed to write compelling content on my own website. Brownlee, meanwhile, got his free distribution from YouTube (and, to the extent you want to complain about his accurate but attention-grabbing headline, the realities of the YouTube algorithm, the one non-viewer incentive Brownlee needs to respond to, looms large). The key point is that he could do everything else on his own, starting with the webcam on his HP Pavilion laptop that he used to record his first video:

Today Brownlee has a top-of-the-line studio and a staff, but the fundamental principle remains: in media one person can do it all, and build a thriving business that makes everyone from Humane to Apple care deeply about what he or she has to say — far more than they care about any traditional publication.

AI and Intelligence

I mentioned at the beginning that last year I wrote about ChatGPT Gets a Computer in lieu of an anniversary post; that Article, which was built around OpenAI’s announcement of (now-discontinued) ChatGPT plugins, was about the nature of the brain and how large learning models did or did not compare.

Computers are deterministic: if circuit X is open, then the proposition represented by X is true; 1 plus 1 is always 2; clicking “back” on your browser will exit this page. There are, of course, a huge number of abstractions and massive amounts of logic between an individual transistor and any action we might take with a computer — and an effectively infinite number of places for bugs — but the appropriate mental model for a computer is that they do exactly what they are told (indeed, a bug is not the computer making a mistake, but rather a manifestation of the programmer telling the computer to do the wrong thing).

I’ve already mentioned Bing Chat and ChatGPT; on March 14 Anthropic released another AI assistant named Claude: while the announcement doesn’t say so explicitly, I assume the name is in honor of the aforementioned Claude Shannon. This is certainly a noble sentiment — Shannon’s contributions to information theory broadly extend far beyond what Dixon laid out above — but it also feels misplaced: while technically speaking everything an AI assistant is doing is ultimately composed of 1s and 0s, the manner in which they operate is emergent from their training, not proscribed, which leads to the experience feeling fundamentally different from logical computers — something nearly human — which takes us back to hallucinations; Sydney was interesting, but what about homework?

I added in an August Update:

The point of that Article was that ChatGPT’s plugin architecture gave hallucinating creative LLMs access to determinative computers to ascertain truth, not dissimilar to the way a creative being like you or I might use a calculator to solve a math problem. In other words, the LLM is the interface to the source of truth, not the source of truth itself.

That is exactly what this “coming soon” feature is all about: you don’t make an LLM useful for your business by adding your business’s data to the LLM; that is simply a bit more text in a sea of it. Rather, you leverage the LLM as an interface to “computers” that deterministically give you the right answer. In this case, those computers will be “connecting the applications you already have”, which sounds to me an awful lot like enterprise-specific plug-ins.

The specifics of product implementations are not the point, but rather the distinction between a natural language interface based on probabilities and computers based on deterministic calculations; I think using the former to access the latter will remain one of the most important applications of large language models.

There is, though, one more piece: who actually tells the AI what to do, such that it needs access to a computer of its own? In ChatGPT Gets a Computer, which analogized large language models to Jeff Hawkins’ theory of the brain, I expressed hope that the beliefs he expressed about the nature of intelligence in A Thousand Brains: A New Theory of Intelligence extended to large language models. Hawkins writes:

Intelligence is the ability of a system to learn a model of the world. However, the resulting model by itself is valueless, emotionless, and has no goals. Goals and values are provided by whatever system is using the model. It’s similar to how the explorers of the sixteenth through the twentieth centuries worked to create an accurate map of Earth. A ruthless military general might use the map to plan the best way to surround and murder an opposing army. A trader could use the exact same map to peacefully exchange goods. The map itself does not dictate these uses, nor does it impart any value to how it is used. It is just a map, neither murderous nor peaceful. Of course, maps vary in detail and in what they cover. Therefore, some maps might be better for war and others better for trade. But the desire to wage war or trade comes from the person using the map.

Similarly, the neocortex learns a model of the world, which by itself has no goals or values. The emotions that direct our behaviors are determined by the old brain. If one human’s old brain is aggressive, then it will use the model in the neocortex to better execute aggressive behavior. If another person’s old brain is benevolent, then it will use the model in the neocortex to better achieve its benevolent goals. As with maps, one person’s model of the world might be better suited for a particular set of aims, but the neocortex does not create the goals.

To the extent this is an analogy to AI, large language models are intelligent, but they do not have goals or values or drive. They are tools to be used by, well, anyone who is willing and able to take the initiative to use them.

AI and the Sovereign Individual

I don’t think either Brownlee or I particularly need AI, or, to put it another way, are overly threatened by it. Yes, ChatGPT would have written several thousand words far more quickly than the hours it took me to write this Article, but I am (perhaps foolishly) confident that they would not be original and valuable enough to take away my audience; I think it’s the same for Brownlee.

The connection between us and AI, though, is precisely the fact that we haven’t needed it: the nature of media is such that we could already create text and video on our own, and take advantage of the Internet to — at least in the case of Brownlee — deliver finishing blows to $230 million startups.

How many industries, though, are not media, in that they still need a team to implement the vision of one person? How many apps or services are there that haven’t been built, not because one person can’t imagine them or create them in their mind, but because they haven’t had the resources or team or coordination capabilities to actually ship them?

This gets at the vector through which AI impacts the world above and beyond cost savings in customer support, or whatever other obvious low-hanging fruit there may be: as the ability of large language models to understand and execute complex commands — with deterministic computing as needed — increases, so too does the potential power of the sovereign individual telling AI what to do. The Internet removed the necessity — and inherent defensibility — of complex cost structures for media; AI has the potential to do the same for a far greater host of industries.

Brownlee’s 2nd most popular video is only two months old: it’s his overview of the Apple Vision Pro; it’s not, if fiction is our guide, an accident that Meta is developing the Quest and Apple has released the Vision Pro just as AI seems poised to threaten an ever-increasing number of jobs. This was the world painted by Pixar’s Wall-E:

Humans in Pixar's Wall-E

It’s funny, because before I looked up this photo, I could have sworn the humans in the film, lulled to insouciance by an AI that addressed all of their physical needs, were wearing headsets; I guess Pixar overestimated our ability to build space-faring cruise ships and underestimated the relentless progression of consumer electronics.

My suspicion, however, is that Pixar got it wrong, and this famous photo of Mark Zuckerberg at Mobile World Congress in 2016 is closer to the truth of the matter.

Mark Zuckerberg at Mobile World Congress

Wall-E erred by assuming that every human was the same, all gleefully enslaved by AUTO, the ship’s AI. In fact, though, I suspect humanity will be distributed bi-modally, with the vast majority of people happily wearing their Vision Pros or watching their streaming service or viewing their TikTok videos, while increasingly sovereign individuals, aided by AI, pilot the ship.

That may sound extreme, but again, this is why it is useful to look to media first: Brownlee — one man, who started out by reviewing a media center remote on his HP laptop — does in fact hold the fate of entire companies and products in his hand. My bet is not that AI replaces Brownlee, but that AI means Brownlee’s for everything, not just media.

As for me, I plan on documenting it every step of the way, thanks to every person who has supported Stratechery over the last 10 years, and to whom I alone owe accountability, and gratitude.

  1. Actually, the deference Brownlee gave the vision and the potential of future upgrades was arguably generous ↩

  2. I previously spoke for pay, and even more previously offered consulting, and even more previously than that had ads; however, I ended ads in 2014, consulting in 2015, and haven’t accepted any paid speaking opportunities since 2017. This is all covered on my about page. ↩

  3. $12/month or $120/year for the last five years, increasing to $15/month and $150/year on April 22 ↩

Gemini 1.5 and Google’s Nature

Posted on

This Article is available as a video essay on YouTube

It was impossible to miss the leading message at yesterday’s Google Cloud Next keynote: Google has the best infrastructure for AI. This was CEO Sundar Pichai in his video greeting:

I want to highlight just a few reasons Google Cloud is showing so much progress. One is our deep investments in AI. We’ve known for a while that AI would transform every industry and company, including our own. That’s why we’ve been building AI infrastructure for over a decade, including TPUs, now in their 5th generation. These advancements have helped customer train and serve cutting-edge language models. These investments put us in the forefront of the AI platform shift.

Google Cloud CEO Thomas Kurian made the priority clear as well:

Today we’re going to focus on how Google is helping leading companies transform their operations and become digital and AI leaders, which is the new way to cloud. We have many important advances, starting with our infrastructure.

What was most interesting about the keynote, though, is what that infrastructure makes possible, and, by extension, what that says about Google’s ability to compete.

Grounding

One of the most surprising things about large language models (LLMs) is how much they know; from the very beginning, though, hallucinations have been a concern. Hallucinations are, of course, part of what makes LLMs so impressive: a computer is actually being creative! It’s also a feature that isn’t particularly impressive to the enterprise customers that this keynote was directed at.

To that end, Kurian, shortly after going over Google’s infrastructure advantages, talked about “grounding”, both in terms of the company’s Gemini model broadly, and for enterprise use cases specifically in the context of Google’s Vertex AI model management service:

To augment models, Vertex AI provides managed tooling to connect your model to enterprise applications and databases, using extensions and function-calling. Vertex also provides retrieval augmented generation (RAG) combining the strengths of retrieval and generative models to provide high quality personalized answers and recommendations. Vertex can augment models with up-to-date knowledge from the web and from your organization, combining generative AI with your enterprise truth.

Today we have a really important announcement: you can now ground with Google Search, perhaps the world’s most trusted source of factual information, with a deep understanding of the world’s knowledge. Grounding Gemini’s responses with Google Search improves response quality and significantly reduces hallucinations.

Second, we’re also making it easy to ground your models with data from your enterprise databases and applications, and any database anywhere. Once you’ve chosen the right model, tuned it, and connected it with your enterprise truth, Vertex’s MLOps can help you manage and monitor models.

A RAG implementation using Google Search is an obvious win, and mirrors ChatGPT’s integration with Bing (or Microsoft Copilot in Bing): the LLM provides answers when it can, and searches the web for things it doesn’t know, a particularly useful feature if you are looking for more recent information.

A more impressive demonstration of grounding, though, was in the context of integrating Gemini with Google’s BigQuery data warehouse and Looker business intelligence platform:

In this demo, the worker gets an alert that a particular product is selling out; using generative AI the worker can see sales trends, find similar models, and create a plan of action for dealing with declining inventory for delivery to her team.

What is notable is not the demo specifics (which is unapologetically made-up for Cymbal, Google’s demo brand); rather, note the role of the LLM: it is not providing information or taking specific actions, but rather serving as a much more accessible natural language interface to surface and collect data that would otherwise take considerably more expertise and time. In other words, it is trustworthy because it is grounded through integration Google is promising with its other enterprise data services.

Gemini 1.5

At the same time, that last section didn’t really follow on from the introduction: yes, those LLMs leveraging Google or BigQuery are running on Google’s infrastructure, but other companies or startups can build something similar. This is where the rest of Pichai’s introduction comes in:

We also continue to build capable AI models to make products like search, Maps, and Android radically more helpful. In December, we took our next big step with Gemini, our largest and most capable model yet. We’ve been bringing it to our products and to enterprises and developers through our APIs. We’ve already introduced our next generation Gemini 1.5 Pro. It’s been in private preview in Vertex AI. 1.5 Pro shows dramatically enhanced performance and includes a breakthrough in long context understanding. That means it can run 1 million tokens of information consistently, opening up new possibilities for enterprises to create, discover, and build using AI. There’s also Gemini’s multi-modal capabilities, which can process audio, video, text, code and more. With these two advances, enterprises can do things today that just weren’t possible with AI before.

Google hasn’t said how Gemini 1.5 was made, but clearly the company has overcome the key limitation of traditional transformers: memory requirements increase quadratically with context length. One promising approach is Ring Attention with Blockwise Transformers, which breaks long contexts into pieces to be computed individually even as the various devices computing those pieces simultaneously communicate to make sense of the context as a whole; in this case memory requirements scale linearly with context length, and can be extended by simply adding more devices to the ring topology.

This is where Google’s infrastructure comes in: the company not only has a massive fleet of TPUs, but has also been developing those TPUs to run in parallel at every level of the stack, from chip to cluster to even data centers (this latter requirement is more pertinent for training than inference); if there is a solution that calls for scale, Google is the best placed to provide it, and it seems the company has done just that with Gemini 1.5.

Demos

To that end, and per Pichai’s closing line, almost all of the other demos in the keynote were implicitly leveraging Gemini 1.5’s context window.

In a Gemini for Workspaces demo, the worker evaluated two statements of work against each other, and against the company’s compliance document:

Here are the key quotes:

Google Drive is ready without any additional AI pre-work…

Each of these documents is over 70 pages. It would have taken me hours to review these docs, but instead Gemini is going to help me find a clean answer to save me a ton of time…

Before I proceed with this vendor, I need to ensure that no compliance issues exist, and I’m going to be honest, I have not memorized every rule in our compliance rulebook because it is over 100 pages. I would have to need to scour the 80 pages of this proposal and compare it manually with the 100 pages of the rulebook. So instead, in the side panel I ask, “Does this offer comply with the following” and I’m going to just @-mention our compliance rulebook, hit Enter, and see what Gemini has to say. So interesting: Gemini has found an issue, because the supplier has not listed their security certifications.

Because Gemini is grounded in my company’s data, with source citations to specific files, I can trust this response and start to troubleshoot before selecting a vendor.

The key distinction between this demo and the last one is that quote at the beginning: a large context window just works in a far greater number of use cases, without any fiddly RAG implementations or special connections to external data stores; just upload the files you need to analyze, and you’re off.

In a Creative Agent with Imagen demo, the worker was seeking to create marketing images and storyboards for an outdoor product:

Here is the key quote:

The creative agent can analyze our previous campaigns to understand our unique brand style and apply it to new ideas. In this case, the creative agent has analyzed over 3,000 brand images, descriptions, videos, and documents of other products that we have in our catalog, contained within Google Drive, to create this summary…The creative agent was able to use Gemini Pro’s 1 million token context window and its ability to reason across text, images, and video to generate this summary.

This was, to be fair, one of the weaker demos: the brand summary and marketing campaign weren’t that impressive, and the idea of creating a podcast with synthetic voices is technically impressive and also something that will never be listened to. That, though, is impressive in its own right: as I noted in an Update when Gemini 1.5 was first announced, “a massively larger context window makes it possible to do silly stuff”, and silly stuff often turns into serious capabilities.

In a Gemini Code Assistant Demo (formerly Duet AI for Developers), a developer new to a job (and the codebase) was tasked with making a change to a site’s homepage:

Here is the key quote:

For the developers out there, you know that this means we’re going to need to add padding in the homepage, modify some views, make sure the configs are changed for our microservices, and typically, it would take me a week or two to even just get familiarized with our company’s code base which has over 100,000 lines of code over 11 services. But now, with Gemini Code Assist, as a new engineer on the team, I can be more productive than ever and can accomplish all of this work in just a matter of minutes. This is because Gemini’s code transformations with full codebase awareness allows us to easily reason through our entire codebase, and in comparison, other models out there can’t handle anything beyond 12,000 to 15,000 lines of code. Gemini with Code Assist is so intelligent that we can just give it our business requirements, including the visual design…Gemini Code Assist doesn’t just suggest code edits; it provides clear recommendations, and makes sure that all of these recommendations align with [the company’s] security and compliance requirements…

And the conclusion:

Let’s recap: behind the scenes Gemini has analyzed my entire codebase in GitLab; it has implemented a new feature; and has ensured that all of the code generated is compatible with my company’s standards and requirements.

Again, leave aside the implausibility of this demo: the key takeaway is the capabilities unlocked when the model is able to have all of the context around a problem while working; this is only possible with — and here the name is appropriate — a long context window, and that is ultimately enabled by Google’s infrastructure.

Google’s Nature

In case it isn’t clear, I think that this keynote was by far the most impressive presentation Google has made in the AI era, not least because the company knows exactly what its advantages are. Several years ago I wrote an Article called Microsoft’s Monopoly Hangover that discussed the company’s then-ongoing transition away from Windows as the center of its strategy; the central conceit was a comparison to Lou Gerstner’s 1990’s transformation of IBM.

The great thing about a monopoly is that a company can do anything, because there is no competition; the bad thing is that when the monopoly is finished the company is still capable of doing anything at a mediocre level, but nothing at a high one because it has become fat and lazy. To put it another way, for a former monopoly “big” is the only truly differentiated asset.

My argument was that business models could be changed: IBM did it, and Microsoft was in the process of doing so when I wrote that. Moreover, Gerstner had shown that culture could be changed as well, and Nadella did just that at Microsoft. What couldn’t be changed was nature: IBM was a company predicated on breadth, not specialization; that’s why Gerstner was right to not break apart the company but to instead deliver Internet solutions to enterprises. Similarly, Microsoft was a company predicated on integration around Windows; the company’s shift to services centered on Teams as Microsoft’s operating system in the cloud was also true to the company’s nature.

Google is facing many of the same challenges after its decades long dominance of the open web: all of the products shown yesterday rely on a different business model than advertising, and to properly execute and deliver on them will require a cultural shift to supporting customers instead of tolerating them. What hasn’t changed — because it is the company’s nature, and thus cannot — is the reliance on scale and an overwhelming infrastructure advantage. That, more than anything, is what defines Google, and it was encouraging to see that so explicitly put forward as an advantage.

United States v. Apple

Posted on

First, there is the moment when the news hits the wires: U.S. Sues Apple, Alleges Tech Giant Exploits Illegal Monopoly.

This, we know, is the beginning of a long journey: there will be months or years of filings and discovery and preliminary rulings; eventually we will have a trial, and months after that a decision. Then there are the appeals, first with three judges, then perhaps en banc, and maybe even the Supreme Court. At some point, should the government win, there will be a hearing about remedies, themselves subject to the same grinding schedule. Only then can a proper determination be made about the validity of the legal questions in this case.

Still, I think the initial moment matters: antitrust is inherently political, and tech companies are generally popular; this makes it hard to build and maintain the momentum necessary to endure the grind. One would certainly have expected that to be an advantage for Apple: the company gains power in market after market precisely by making consumers happy.

Apple’s Integration

I explained in 2014 How Apple Creates Leverage by marshaling loyal customers:

Apple is certainly not shy about proclaiming their fealty towards building great products. And I believe Tim Cook, Jony Ive, and the rest of Apple’s leadership when they say their focus on the experience of using an Apple device comes from their desire to build something they themselves would want to use. But I also believe the strategic implications of this focus are serially undervalued.

Last year I wrote a piece called What Clayton Christensen Got Wrong that explored the idea that the user experience was the sort of attribute that could never be overshot; as long as Apple provided a superior experience, they would always win the high-end subset of the consumer market that is willing to pay for nice things.

However, this telling of the story of iTunes and the iPhone suggests that this focus on the user experience not only defends against disruption, but it also provides an offensive advantage as well: namely, Apple increases its user experience advantage through the leverage it gains from consumers loyal to the company. In the case of iTunes, Apple was able to create the most seamless music acquisition process possible: the labels had no choice but to go along. Similarly, when it comes to smartphones, Apple devices from day one have not been cluttered with carrier branding or apps or control over updates. If carriers didn’t like Apple’s insistence on creating the best possible user experience, well, consumers who valued said experience were more than happy to take their business elsewhere. In effect, Apple builds incredible user experiences, which gains them loyal customers who collectively have massive market power, which Apple can then effectively wield to get its way — a way that involves maximizing the user experience. It’s a virtuous circle.

That Clayton Christensen reference was about his theory of low-end disruption, which I summarized in the aforementioned What Clayton Christensen Got Wrong:

Briefly, an integrated approach wins at the beginning of a new market, because it produces a superior product that customers are willing to pay for. However, as a product category matures, even modular products become “good enough” — customers may know that the integrated product has superior features or specs, but they aren’t willing to pay more, and thus the low-priced providers, who build a product from parts with prices ground down by competition, come to own the market. Christensen was sure this would happen with the iPod, and he — and his many adherents — are sure it will happen to the iPhone.

That Article — where I argued that low end disruption would not happen to Apple — built on a paper I had written in business school in 2010 entitled Apple and the Innovator’s Dilemma:

Steve Jobs, when asked why people want to work for Apple, said:

Our DNA is as a consumer company — for that individual customer who’s voting thumbs up or thumbs down. That’s who we think about. And we think that our job is to take responsibility for the complete user experience. And if it’s not up to par, it’s our fault, plain and simply.

This article has already laid out many of the strategic benefits of this intense focus on the user experience: it creates significant differentiation, sets customer expectations, and makes it impossible to overshoot customer needs — products are never “good enough” with regard to the user experience. From an organizational standpoint, if products are never “good enough”, then a highly integrated company is appropriate. Chrstensen and Raynor note in the Innovatorʼs Solution that the “not-good-enough circumstance mandate[s] interdependent product or value chain architectures and vertical integration.”

Another way to look at Appleʼs decisions regarding its organizational structure is to think of transaction costs: normally, in well-functioning markets, vertical integration is suboptimal. However, if transaction costs in the vertical chain outweigh the losses due to the inefficiencies of being vertically integrated, then vertical integration could be the correct course of action. Apple thinks the exact same way, but not about monetary cost; instead, the transaction costs they consider are the tax that modularization places on the user experience, and it is a cost they are not willing to bear. A central tenet is that Apple “need[s] to own and control the primary technologies behind the products [it] make[s].”

This is all a fancy way of repeating Apple’s constant refrain that their success is built upon integration — first hardware and software, then services, and now an entire ecosystem of products extending from the computer to the phone to smart watches to headphones and, most recently, a headset. This integration makes customers happy, and one would expect members of the Apple ecosystem to rally to their cause.

In fact, though, that wasn’t the initial reaction, at least in the tech world which I inhabit: the response of many people was finally.

App Store Anger

The root of tech’s frustration with Apple — including long-time Apple developers, traditionally the company’s most ardent supporters — is the App Store. The App Store is the ultimate example of Apple leveraging consumers’ desire for their products to gain power over an ecosystem; in this case, though, the ecosystem isn’t music labels or cellular providers, but rather app developers, which is to say everyone else in tech. Smartphones are essential devices, and Apple makes the best smartphone, which means their power over developers is absolute.

Apple is not afraid to exercise this power, and it’s important to note that in the early days of the iPhone the company’s control worked in developers’ favor. Apple decreed that 3rd-party apps could only be installed via the App Store, which would review every app; free apps wouldn’t have to pay anything, while Apple would take 30% of paid apps. This led to an absolute explosion in the market: consumers, who had been scarred by the 2000’s era of malware and viruses, shook off their reticence to install software and embraced the App Store, leading to an explosion of app-based businesses. An important development was the 2009 addition of in-app purchase, which unlocked entirely new business models for games in particular. Subscriptions for publications came in 2011, although it took another five years for the business model to be available to all applications.

There was, throughout this period, a continuous burbling of discontent about some of Apple’s App Store policies: one of my early Articles on Stratechery in 2013 asked Why Doesn’t Apple Enable Sustainable Businesses on the App Store?, games developers groused about the 30% fee on in-app purchases, while 3rd-party content providers suffered under an anti-steering provision that didn’t allow them to link to their website — or even tell their users — that they could start a subscription or buy content outside of the app. Everyone, meanwhile, hated the seeming arbitrariness of App Store Review, which only weighed in on an app when it was finished; good luck building something ambitious with the ever-present risk of running afoul of App Store rules.

This discontent came to a head in 2020, when a number of developers had public confrontations with Apple. Some of these were COVID-related, as Apple wanted a piece of for-pay virtual meetings; others were due to an unannounced step-up in focus on subscription-based web apps. A very public fight with Basecamp brought to light an organized campaign to force developers of all sizes to integrate in-app purchase even if they made all of their sales on their website:

That same month Apple announced App Tracking Transparency, a thinly veiled attempt to displace Facebook’s role in customer acquisition for apps; some of the App Tracking Transparency changes had defensible privacy justifications (albeit overstated), but it was hard to not notice that Apple wasn’t holding itself to the same rules, very much to its own benefit.

One year later and many developers cheered when Epic sued Apple for antitrust violations related to the App Store; I gave a list of relevant Articles a couple of months ago in The Apple Vision Pros Missing Apps:

  • Apple, Epic, and the App Store, which provided a history of the App Store and Epic’s lawsuit at the time it was filed.
  • App Store Arguments, which I wrote at the conclusion of the trial, explained why I expected Epic to lose, even as I hoped that Apple would voluntarily make pro-developer changes in the App Store.
  • The Apple v. Epic Decision, which reviewed the judge’s decision that favored Apple in 10 of the 11 counts.

The 11th count that Epic prevailed on required Apple to allow developers to steer users to a website to make a purchase; while its implementation was delayed while both parties filed appeals, the lawsuit reached the end of the road last week when the Supreme Court denied certiorari. That meant that Apple had to allow steering, and the company did so in the most restrictive way possible: developers had to use an Apple-granted entitlement to put a link on one screen of their app, and pay Apple 27% of any conversions that happened on the developer’s website within 7 days of clicking said link.

Many developers were outraged, but the company’s tactics were exactly what I expected…Apple has shown, again and again and again, that it is only going to give up App Store revenue kicking-and-screaming; indeed, the company has actually gone the other way, particularly with its crackdown over the last few years on apps that only sold subscriptions on the web (and didn’t include an in-app purchase as well). This is who Apple is, at least when it comes to the App Store.

Here is the most important thing to understand about this entire App Store discussion — the topic that has many people who are normally skeptical of government involvement in tech cheering on the Department of Justice: basically none of it is pertinent to this case. Oh sure, the initial case filing aired all of these grievances — along with a bizarre and factually wrong assertion that iTunes was only ever on Windows because of a DOJ Consent Decree — but the actual assertions of wrongdoing barely mention the App Store at all.

The DOJ’s Case

Steven Sinofsky has a (critical) overview of the case at Hardcore Software; here is his (correct) summary of the DOJ’s actual complaints:

There are two sets of claims in the filing. The first set are written as a narrative for laypeople. The claims are broad and kind of all over the map. Before listing the specifics there is a lot of language and froth about how Apple uses its place in the market. This is the “story.” It is very exciting to read but you have to get to the actual behavior, which the DOJ claims is that “Apple has used one or both mechanisms (control of app distribution or control of APIs) to suppress the following technologies, among others” which includes:

“Supressing Super apps”

“Supressing cloud streaming game apps”

“[P]rohibiting third-party apps from sending or receiving carrier-based messages”

“[S]uppressing key functions of third-party smartwatches”

“[D]enied users access to digital wallets”

The first two are the closest the case comes to App Store complaints, and the key thing to note is that they are not relevant to the vast majority of developers. There is nothing about Apple’s 30% fee, nothing about App Tracking Transparency, and nothing about the steering provision that is, in my estimation, the most noxious of all Apple’s policies. Here is an overview of the five objections:

Super Apps

For years, Apple denied its users access to super apps because it viewed them as “fundamentally disruptive” to “existing app distribution and development paradigms” and ultimately Apple’s monopoly power. Apple feared super apps because it recognized that as they become popular, “demand for iPhone is reduced.” So, Apple used its control over app distribution and app creation to effectively prohibit developers from offering super apps instead of competing on the merits.

A super app is an app that can serve as a platform for smaller “mini” programs developed using programming languages such as HTML5 and JavaScript. By using programming languages standard in most web pages, mini programs are cross platform, meaning they work the same on any web browser and on any device. Developers can therefore write a single mini program that works whether users have an iPhone or another smartphone.

Super apps can provide significant benefits to users. For example, a super app that incorporates a multitude of mini programs might allow users to easily discover and access a wide variety of content and services without setting up and logging into multiple apps, not unlike how Netflix and Hulu allow users to find and watch thousands of movies and television shows in a single app. As one Apple executive put it, “who doesn’t want faster, easier to discover apps that do everything a full app does?” Restricting super apps makes users worse off and sacrifices the short-term profitability of iPhones for Apple.

As the DOJ complaint correctly notes, Apple has faced challenges in China because of the WeChat “super app”; it is easier to switch if all of your essential services are in one place. At the same time, I am skeptical that “super apps” — even without Apple’s restrictions — would ever become a meaningful UI paradigm in western markets long since defined by distinct apps.

Moreover, that installable apps are only available from Apple has always been a part of the iPhone’s brand promise: that is why it is important to remember the early days of the App Store and how it created consumer demand in the first place. This brand promise around security and safety will surely be a core part of Apple’s defense.

Cloud Streaming Game Apps

For years, Apple blocked cloud gaming apps that would have given users access to desirable apps and content without needing to pay for expensive Apple hardware because this would threaten its monopoly power. In Apple’s own words, it feared a world where “all that matters is who has the cheapest hardware” and consumers could “buy[] a [expletive] Android for 25 bux at a garage sale and . . . have a solid cloud computing device” that “works fine.” Apple’s conduct made its own product worse because consumers missed out on apps and content. This conduct also cost Apple substantial revenues from third-party developers. At the same time, Apple also made other smartphones worse by stifling the growth of these cross-platform apps on other smartphones. Importantly, Apple prevented the emergence of technologies that could lower the price that consumers pay for iPhones.

Cloud streaming apps let users run a computationally intensive program without having to process or store the program on the smartphone itself. Instead, a user’s smartphone leverages the computing power of a remote server, which runs the program and streams the result back to the phone. Cloud streaming allows developers to bring cutting-edge technologies and services to smartphone consumers—including gaming and interactive artificial intelligence services—even if their smartphone includes hardware that is less powerful than an iPhone.

Apple long required that every individual cloud streaming game be an individual standalone app, and frankly, I think this is the DOJ’s strongest argument: cloud streaming apps don’t pose any of the (theoretical or not) security and safety concerns of apps installed outside of the App Store; the entire point is that you don’t need to install any apps at all, so to have such a requirement seems like a clear attempt to kill competition.

To that end, one gets the impression that Apple’s January announcement that it would now allow cloud streaming services to be self-contained in a single app was a preemptive response to this filing; however, those single apps still have to include an in-app purchase option — you can’t simply offer an app that lets you sign in to the subscription you made elsewhere.

SMS and Private APIs

Apple undermines cross-platform messaging to reinforce “obstacle[s] to iPhone families giving their kids Android phones.” Apple could have made a better cross-platform messaging experience itself by creating iMessage for Android but concluded that doing so “will hurt us more than help us.” Apple therefore continues to impede innovation in smartphone messaging, even though doing so sacrifices the profits Apple would earn from increasing the value of the iPhone to users, because it helps build and maintain its monopoly power.

This entire section is pretty nuts. The core complaint is that Apple doesn’t allow 3rd-party messaging apps like WhatsApp to access private API’s for SMS; the vast majority of the complaint, though, is effectively accusing Apple of acting anti-competitive by not building iMessage for Android, and making teenagers feel bad about green bubbles. Leaving aside the fact that green bubbles actually serve a product function — they are not encrypted, while blue iMessage bubbles are — the entire idea that Apple needs to proactively build features to facilitate communication with Android flies in the face of Supreme Court precedent in Verizon v. Trinko:

Firms may acquire monopoly power by establishing an infrastructure that renders them uniquely suited to serve their customers. Compelling such firms to share the source of their advantage is in some tension with the underlying purpose of antitrust law, since it may lessen the incentive for the monopolist, the rival, or both to invest in those economically beneficial facilities. Enforced sharing also requires antitrust courts to act as central planners, identifying the proper price, quantity, and other terms of dealing — a role for which they are ill-suited. Moreover, compelling negotiation between competitors may facilitate the supreme evil of antitrust: collusion. Thus, as a general matter, the Sherman Act “does not restrict the long recognized right of [a] trader or manufacturer engaged in an entirely private business, freely to exercise his own independent discretion as to parties with whom he will deal.”

I expect Trinko to be a substantial obstacle for the DOJ in this case. It is one thing to change the rules or withdraw capabilities to maintain a monopoly; there was a Supreme Case called Aspen Skiing where a company was found guilty of doing just that, but the Court drew a clear distinction in Trinko between changes in policy and never actually providing pro-competition capability in the first place:

Aspen Skiing is at or near the outer boundary of §2 liability. The Court there found significance in the defendant’s decision to cease participation in a cooperative venture. The unilateral termination of a voluntary (and thus presumably profitable) course of dealing suggested a willingness to forsake short-term profits to achieve an anticompetitive end. Similarly, the defendant’s unwillingness to renew the ticket even if compensated at retail price revealed a distinctly anticompetitive bent.

The refusal to deal alleged in the present case does not fit within the limited exception recognized in Aspen Skiing. The complaint does not allege that Verizon voluntarily engaged in a course of dealing with its rivals, or would ever have done so absent statutory compulsion. Here, therefore, the defendant’s prior conduct sheds no light upon the motivation of its refusal to deal — upon whether its regulatory lapses were prompted not by competitive zeal but by anticompetitive malice.

This also hints at how a standard like RCS could be forced upon Apple: through legislation (statutory compulsion); passing new laws remains the most appropriate way to deal with Aggregator power.

Smartwatch Integration

Apple’s smartwatch — Apple Watch — is only compatible with the iPhone. So, if Apple can steer a user towards buying an Apple Watch, it becomes more costly for that user to purchase a different kind of smartphone because doing so requires the user to abandon their costly Apple Watch and purchase a new, Android-compatible smartwatch…

Apple uses its control of the iPhone, including its technical and contractual control of critical APIs, to degrade the functionality of third-party cross-platform smartwatches in at least three significant ways: First, Apple deprives iPhone users with third-party smartwatches of the ability to respond to notifications. Second, Apple inhibits third-party smartwatches from maintaining a reliable connection with the iPhone. And third, Apple undermines the performance of third-party smartwatches that connect directly with a cellular network. In doing so, Apple constrains user choice and crushes innovation that might help fill in the moat around Apple’s smartphone monopoly.

This is the part of the case that should concern Apple the most, because it is a direct attack on Apple’s core differentiation: the deep integration of software, hardware, and services across its devices. The Apple Watch has access to private APIs and system-level integrations that Apple claims provide for better battery life, messaging management, etc.; the DOJ says that these integrations should be modularized and made available to all smart watches. There is no consideration as to whether or not Apple’s claims are true, or acknowledgment that Apple’s integration was supposedly a losing proposition a decade ago: now that the company has been proven successful, said proposition is alleged to be monopoly maintenance.

NFC Access for Digital Wallets

Apple recognizes that paying for products and services with a digital wallet will eventually become “something people do every day of their lives.” But Apple has used its control over app creation, including its technical and contractual control over API access, to effectively block third-party developers from creating digital wallets on the iPhone with tap-to-pay functionality, which is an important feature of a digital wallet for smartphones. As a result, Apple maintains complete control over how users make tap-to-pay payments with their iPhone. Apple also deprives users of the benefits and innovations third-party wallets would provide so that it can protect “Apple’s most important and successful business, iPhone.”

This is a valid complaint framed in a very odd way. I think there is a case to be made that Apple has reserved NFC tap-to-pay access for itself in order to leverage its iPhone control into control of an adjacent market (digital wallets), but I have a hard time buying the DOJ’s argument that this imposes unacceptable switching costs to another smartphone. It’s also, in my estimation, fairly weak gruel for an antitrust case of this magnitude, particularly given that an integrated wallet is, once again, very much in line with Apple’s longstanding brand promise.

Apple’s Mistake

I have, for years, been urging Apple to take a different approach to the App Store, particularly in terms of non-gaming apps and the anti-steering provision, even if the company’s approach were technically legal. The danger I foresaw was not simply the loss of developer goodwill, but something even more important to Apple: its fundamental differentiation, i.e. integration. I stated the risk explicitly in 2021’s Integrated Apple and App Store Risk:

If you were to boil Apple’s philosophy and attractiveness to customers to one word, that word would be “integration.” And guess what? First party integration is bad for third-party developers — everything is a tradeoff. This is where the nuance I discussed in App Store Arguments becomes much more black-and-white. Yes, Apple created the iPhone and the App Store and, under current U.S. antitrust doctrine, almost certainly has the right to impose whatever taxes it wishes on third parties, including 30% on purchases and the first year of subscriptions, and completely cutting off developers from their customers. Antitrust law, though, while governed by Supreme Court precedent, is not a matter of constitutionality: it stems from laws passed by Congress, and it can be changed by new laws passed by Congress.

One of the central planks of many of those pushing for new laws in this area are significant limitations on the ability of platforms to offer apps and services, or integrate them in any way that advantages their offerings. In this potential world it’s not simply problematic that Apple charges Spotify 30%, or else forces the music streaming service to hope that users figure out how to subscribe on the web, even as Apple Music has a fully integrated sign-up flow and no 30% tax; it is also illegal to incorporate Apple Music into SharePlay or Shared-with-you or Photos, or in the most extreme versions of these proposed laws, even have Apple Music at all. This limitation would apply to basically every WWDC announcement: say good-bye to Quick Note or SharePlay-as-an-exclusive-service, or any number of Apple’s integrated offerings.

I think these sorts of limitations would be disappointing as a user — integration really does often lead to better outcomes sooner — and would be a disaster for Apple. The entire company’s differentiation is predicated on integration, including its ability to abuse its App Store position, and it would be a huge misstep if the inability to resist the latter imperiled the former.

Last week this danger manifested, not as new legislation, but as this lawsuit, which attacks Apple’s integration much more than it attacks the App Store. I think, though, that it was Apple’s policies around the App Store that created the conditions for this lawsuit in the first place.

In short, I suspect the DOJ doesn’t want to follow in Epic’s footsteps, but they do want to sue Apple, so they framed Apple’s defining characteristic — integration — in the most uncharitable light possible to make their case. To put it another way, the Epic case may have shown that Apple’s policies around the App Store were (mostly) legal, but that didn’t mean they were right; now the DOJ, looking for another point of vulnerability, is trying to make the case that Apple’s right approach in delivering an integrated experience is in fact illegal.

During the Epic trial John Gruber wished that Apple would relax its approach to the App Store:

What’s weirdest about Apple’s antitrust and PR problems related to the App Store is that the App Store is a side hustle for Apple. Yes it’s earning Apple $10+ billion a year, and even for Apple that’s significant. But it’s not Apple’s main business by a long shot. To my knowledge no company in history has ever gotten into antitrust hot water over a side business so comparatively small to its overall business. Apple doesn’t need this.

I think Apple’s senior leadership — Cook in particular — truly does believe that Apple has earned every dollar it generates from third-party software in the App Store, and that their policies in place are just and fair. That righteousness came out on the stand in the Epic trial. But even if Apple’s executives are correct — if the current rules and revenue splits could somehow be proven to be dialed in to a hypothetical Platonic ideal of fairness to all parties involved — that doesn’t change the fact that so many developers see it otherwise.

I don’t think the developers are wrong, but even if they are wrong, it’s not good for Apple that they’re so unhappy, and feel so aggrieved. It’s not good for Apple that developers don’t see the App Store as a platform that works in their interests. Like the Apple logo, “developer goodwill” has no price tag.

To the extent I am right — and yes, it is impossible to prove a counterfactual — the price tag of Apple’s “side hustle” is even higher than Gruber thought: I believe that if Apple had done even the bare minimum with the App Store — i.e. removing the anti-steering provision and not going after small developers with online services — it wouldn’t be in a position of having to defend what actually makes Apple Apple. It would, at a minimum, have a lot more people on its side.

I wrote a follow-up to this Article in this Daily Update.

Nvidia Waves and Moats

Posted on

This Article is available as a video essay on YouTube

From the Wall Street Journal:

The Nvidia frenzy over artificial intelligence has come to this: Chief Executive Jensen Huang unveiled his company’s latest chips on Monday in a sports arena at an event one analyst dubbed the “AI Woodstock.”

Customers, partners and fans of the chip company descended on the SAP Center, the home of the National Hockey League’s San Jose Sharks, for Huang’s keynote speech at an annual Nvidia conference that, this year, has a seating capacity of about 11,000. Professional wrestling’s WWE Monday Night RAW event took place there in February. Justin Timberlake is scheduled to play the arena in May. Even Apple’s much-watched launch events for the iPhone and iPad didn’t fill a venue this large. At the center of the tech world’s attention is Huang, who has gone from a semiconductor CEO with a devoted following among videogame enthusiasts to an AI impresario with broad-enough appeal to draw thousands to a corporate event.

Or, as Nvidia Research Manager Jim Fan put it on X:

I’m disappointed that the Wall Street Journal used this lead for their article about the event, but not because I thought they should have talked about the actual announcements: rather, they and I had the exact same idea. It was the spectacle, even more than the announcements, that was the most striking takeaway of Huang’s keynote.

I do think, contra the Wall Street Journal, that iPhone announcements are a relevant analogy; Apple could have, particularly in the early days of the iPhone, easily filled an 11,000 seat arena. Perhaps an even better analogy, though, was the release of Windows 95. Lance Ulanoff wrote a retrospective on Medium in 2001:

It’s hard to imagine an operating system, by itself, garnering the kind of near-global attention the Windows 95 launch attracted in 1995. Journalists arrived from around the world on August 24, 1995, settling on the lush green, and still relatively small Microsoft Campus in Redmond, Washington. There were tickets (I still have mine) featuring the original Windows Start Button (“Start” was a major theme for the entire event) granting admission to the invite-only, carnival-like event…It was a relatively happy and innocent time in technology. Perhaps the last major launch before the internet dominated everything, when a software platform, and not blog post or a piece of hardware, could change the world.

One can envision an article in 2040 looking back on the “relatively happy and innocent time in technology” as we witnessed “perhaps the last major launch before AI dominated everything” when a chip “could change the world”; perhaps retrospectives of the before times will be the last refuge of human authors like myself.

GTCs of Old

What is interesting to a once-and-future old fogey like myself, who has watched multiple Huang keynotes, is how relatively focused this event was: yes, Huang talked about things like weather and robotics and Omniverse and cars, but this was, first-and-foremost, a chip launch — the Blackwell B200 generation of GPUs — with a huge chunk of the keynote talking about its various features and permutations, performance, partnerships, etc.

I thought this stood in marked contrast to GTC 2022 when Huang announced the Hopper H100 generation of GPUs: that had a much shorter section on the chips/system architecture, accompanied by a lot of talk about potential use cases and a list of all of the various libraries Nvidia was developing for CUDA. This was normal for GTC, as I explained a year earlier:

This was, frankly, a pretty overwhelming keynote; Liberty thinks this is cool:

Robots and digital twins and games and machine learning accelerators and data-center-scale computing and cybersecurity and self-driving cars and computational biology and quantum computing and metaverse-building-tools and trillion-parameter AI models! Yes plz

Something Huang emphasized in the introduction to the keynote, though, is that there is a rhyme and reason to this volume…

I then went on an extended explainer of CUDA and why it was essential to understanding Nvidia’s long-term opportunity, and concluded:

This is a useful way to think about Nvidia’s stack: writing shaders is like writing assembly, as in its really hard and very few people can do it well. CUDA abstracted that away into a universal API that was much more generalized and approachable — it’s the operating system in this analogy. Just like with operating systems, though, it is useful to have libraries that reduce duplicative work amongst programmers, freeing them to focus on their own programs. So it is with CUDA and all of those SDKs that Huang referenced: those are libraries that make it much simpler to implement programs that run on Nvidia GPUs.

This is how it is that a single keynote can cover “Robots and digital twins and games and machine learning accelerators and data-center-scale computing and cybersecurity and self-driving cars and computational biology and quantum computing and metaverse-building-tools and trillion-parameter AI models”; most of those are new or updated libraries on top of CUDA, and the more that Nvidia makes, the more they can make.

This isn’t the only part of the Nvidia stack: the company has also invested in networking and infrastructure, both on the hardware and software level, that allows applications to scale across an entire data center, running on top of thousands of chips. This too requires a distinct software plane, which reinforces that the most important thing to understand about Nvidia is that it is not a hardware company, and not a software company: it is a company that integrates both.

Those GTCs were, in retrospect, put on by a company before it had achieved astronomical product-market fit. Sure, Huang and Nvidia knew about transformers and GPT models — Huang referenced his hand-delivery of the first DGX supercomputer to OpenAI in 2016 in yesterday’s opening remarks — but notice how his hand-drawn slide of computing history seems to exclude a lot of the stuff that used to be at GTC:

Huang's drawing of computing journey to today

Suddenly all that matters in those intervening years was transformers!

I am not, to be clear, short-changing Huang or Nvidia in any way; quite the opposite. What is absolutely correct is that Nvidia had on their hands a new way of computing, and the point of those previous GTC’s was to experiment and push the world to find use cases for it; today, in this post-ChatGPT world, the largest use case — generative AI — is abundantly clear, and the most important message for Huang to deliver is why Nvidia will continue to dominate that use case for the foreseeable future.

Blackwell

So about Blackwell itself; from Bloomberg:

Nvidia Corp. unveiled its most powerful chip architecture at the annual GPU Technology Conference, dubbed Woodstock for AI by some analysts. Chief Executive Officer Jensen Huang took the stage to show off the new Blackwell computing platform, headlined by the B200 chip, a 208-billion-transistor powerhouse that exceeds the performance of Nvidia’s already class-leading AI accelerators. The chip promises to extend Nvidia’s lead on rivals at a time when major businesses and even nations are making AI development a priority. After riding Blackwell’s predecessor, Hopper, to surpass a valuation of more than $2 trillion, Nvidia is setting high expectations with its latest product.

The first thing to note about Blackwell is that it is actually two dies fused into one chip, with what the company says is full coherence; what this means in practice is that a big portion of Blackwell’s gains relative to Hopper is that it is simply much bigger. Here is Huang holding a Hopper and Blackwell chip up for comparison:

Huang holding a Hopper GPU and a Blackwell GPU

The “Blackwell is bigger” theme holds for the systems Nvidia is building around it. The fully integrated GB200 platform has two Blackwell chips with one Grace CPU chip, as opposed to Hopper’s 1 to 1 architecture. Huang also unveiled the GB200 NVL72, a liquid-cooled rack sized system that included 72 GPUs interconnected with a new generation of NVLink, which the company claims provides a 30x performance increase over the same number of H100 GPUs for LLM inference (thanks in part to dedicated hardware for transformer-based inference), with a 25x reduction in cost and energy consumption. One set of numbers I found notable were on these slides:

Blackwell's increased performance in training relative to Hopper

What is interesting to note is that both training runs take the same amount of time — 90 days. This is because the actual calculation speed is basically the same; this makes sense because Blackwell is, like Hopper, fabbed on TSMC’s 4nm process,1 and the actual calculations are fairly serial in nature (and thus primarily governed by the underlying speed of the chip). “Accelerated computing”, though, isn’t about serial speed, but rather parallelism, and every new generation of chips, combined with new networking, enables ever greater amounts of efficient parallelism that keeps those GPUs full; that’s why the big improvment is in the number of GPUs necessary and thus the overall amount of power drawn.

That, by extension, means that a Hopper-sized fleet of Blackwell GPUs will be capable of building that much larger of a model, and given that there appears to be a linear relationship between scale and model capability, the path to GPT-6 and beyond remains clear (GPT-5 was presumably trained on Hopper GPUs; GPT-4 was trained on Ampere A100s).

What is interesting to note is that there are reports that while the B100 costs twice as much as the H100 to manufacture, Nvidia is increasing the price much less than expected; this explains the somewhat lower margins the company is expecting going forward. The report — which has since disappeared from the Internet (perhaps because it was published before the keynote?) — speculated that Nvidia is concerned about preserving its market share in the face of AMD being aggressive in price, and its biggest customers trying to build their own chips. There is, needless to say, tremendous incentives to find alternatives, particularly for inference.

Nvidia Inference Microservices (NIM)

I think this provides useful context for another GTC announcement; from the Nvidia developer blog:

The rise in generative AI adoption has been remarkable. Catalyzed by the launch of OpenAI’s ChatGPT in 2022, the new technology amassed over 100M users within months and drove a surge of development activities across almost every industry. By 2023, developers began POCs [Proof of Concepts] using APIs and open-source community models from Meta, Mistral, Stability, and more.

Entering 2024, organizations are shifting their focus to full-scale production deployments, which involve connecting AI models to existing enterprise infrastructure, optimizing system latency and throughput, logging, monitoring, and security, among others. This path to production is complex and time-consuming — it requires specialized skills, platforms, and processes, especially at scale.

NVIDIA NIM, part of NVIDIA AI Enterprise, provides a streamlined path for developing AI-powered enterprise applications and deploying AI models in production.

NIM is a set of optimized cloud-native microservices designed to shorten time-to-market and simplify deployment of generative AI models anywhere, across cloud, data center, and GPU-accelerated workstations. It expands the developer pool by abstracting away the complexities of AI model development and packaging for production ‌using industry-standard APIs.

NIM’s are pre-built containers that contain everything an organization needs to get started with model deployment, and they are addressing a real need not just today, but in the future; Huang laid out a compelling scenario where companies’ use multiple NIMs in an agent-type of framework to accomplish complex tasks:

Think about what an AI API is: an AI API is an interface that you just talk to. So this is a piece of software that in the future that has a really simple API, and that API is called human. These packages, incredible bodies of software, will be optimized and packaged and we’ll put it on a website, and you can download it, you can take it with you, you can run it on any cloud, you can run it in your datacenter, you can run it on workstations if it fits, and all you have to do is come to ai.nvidia.com. We call it Nvidia Inference Microservices, but inside the company we all call it NIMs.

Just imagine, someday there’s going to be one of these chatbots, and these chatbots is just going to be in a NIM. You’ll assemble a whole bunch of chatbots, and that’s the way that software is going to be built some day. How do we build software in the future? It is unlikely that you’ll write it from scratch, or write a whole bunch of Python code or anything like that. It is very likely that you assemble a team of AIs.

There’s probably going to be a super-AI that you use that takes the mission that you give it and breaks it down into an execution plan. Some of that execution plan could be handed off to another NIM, that NIM would maybe understand SAP. The language of SAP is ABAP. It might understand ServiceNow and go and retrieve some information from their platforms. It might then hand that result to another NIM, who goes off and does some calculation on it. Maybe it’s an optimization software, a combinatorial optimization algorithm. Maybe it’s just some basic calculator. Maybe it’s pandas to do some numerical analysis on it. And then it comes back with its answer, and it gets combined with everybody else’s, and because it’s been presented with “This is what the right answer should look like,” it knows what right answers to produce, and it presents it to you. We can get a report every single day, top-of-the-hour, that has something to do with a build plan or some forecast or some customer alert or some bugs databased or whatever it happens to be, and we can assemble it using all these NIMs.

And because these NIMs have been packaged up and ready to work on your system, so long as you have Nvidia GPUs in your datacenter or in the cloud, these NIMs will work together as a team and do amazing things.

Did you notice the catch? NIMs — which Nvidia is going to both create itself and also spur the broader ecosystem to create, with the goal of making them freely available — will only run on Nvidia GPUs.

NIM's only run on Nvidia GPUs

This takes this Article full circle: in the before-times, i.e. before the release of ChatGPT, Nvidia was building quite the (free) software moat around its GPUs; the challenge is that it wasn’t entirely clear who was going to use all of that software. Today, meanwhile, the use cases for those GPUs is very clear, and those use cases are happening at a much higher level than CUDA frameworks (i.e. on top of models); that, combined with the massive incentives towards finding cheaper alternatives to Nvidia, means both the pressure to and the possibility of escaping CUDA is higher than it has ever been (even if it is still distant for lower level work, particularly when it comes to training).

Nvidia has already started responding: I think that one way to understand DGX Cloud is that it is Nvidia’s attempt to capture the same market that is still buying Intel server chips in a world where AMD chips are better (because they already standardized on them); NIM’s are another attempt to build lock-in.

In the meantime, though, it remains noteworthy that Nvidia appears to not be taking as much margin with Blackwell as many may have expected; the question as to whether they will have to give back more in future generations will depend on not just their chips’ performance, but also on re-digging a software moat increasingly threatened by the very wave that made GTC such a spectacle.

  1. I was mistaken about this previously ↩

Aggregator’s AI Risk

Posted on

This Article is available as a video essay on YouTube

A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines.
—Ralph Waldo Emerson, “Self-Reliance”, Essays: First Series, 1841
In the beginning was the Word, and the Word was with God, and the Word was God.
—John 1:1, King James Version

A recurring theme on Stratechery is that the only technology analogous to the Internet’s impact on humanity is the printing press: Johannes Gutenberg’s invention in 1440 drastically reduced the marginal cost of printing books, dramatically increasing the amount of information that could be disseminated.

Of course you still had to actually write the book, and set the movable type in the printing press; this, though, meant we had the first version of the classic tech business model: the cost to create a book was fixed, but the potential revenue from printing a book — and overall profitability — was a function of how many copies you could sell. Every additional copy increased the leverage on the up-front costs of producing the book in the first place, improving the overall profitability; this, by extension, meant there were strong incentives to produce popular books.

This set off a number of changes that transformed history. Before the printing press:

  • The Bible was the province of the Catholic Church; it was only available in Latin and laboriously reproduced by monks. In practice this meant that the Catholic Church was the source of religious authority throughout Europe.
  • Europe didn’t have any nation-states as we think of them today; the relevant political authority was some combination of city-states and feudal lords.
  • The linguistic landscape was extremely diverse: Latin was the language of the church, while larger regions might have a dominant dialect, which itself could differ from local dialects only spoken in a limited geographic area.

The printing press was a direct assault on that last point: because it still cost money to produce a book, it made sense to print books in the most dominant dialect in the region; because books were compelling it behooved people to learn to read that dominant dialect. This, over time, would mean that the dominant dialect would increase its dominance in a virtuous cycle — network effects, in other words.

Books, meanwhile, transmitted culture, building affinity between neighboring city states; it took decades and, in some cases, centuries, but over time Europe settled into a new equilibrium of distinct nation-states, with their own languages. Critical to this reorganization was point one: the printing press meant everyone could have access to the Bible, or read pamphlets challenging the Catholic Church. Martin Luther’s 95 Theses was one such example: printing presses spread the challenge to papal authority far and wide precisely because it was so incendiary — that was good for business. The Protestant Reformation that followed didn’t just have theological implications: it also provided the religious underpinnings for those distinct nation states, which legitimized their rule with their own national churches.

Of course history didn’t end there: the apotheosis of the Reformation’s influence on nation states was the United States, which set out an explicit guarantee that there would be no official government religion at all; every person was free to serve God in whatever way they pleased. This freedom was itself emblematic of what America represented in its most idealized form:1 endless frontier and the freedom to pursue one’s God-given rights of “Life, Liberty and the pursuit of Happiness.”

Aggregation Theory

In this view the Internet is the final frontier, and not just because the American West was finally settled: on the Internet there are, or at least were, no rules, and not just in the legalistic sense; there were also no more economic rules as understood in the world of the printing press. Publishing and distribution were now zero marginal cost activities, just like consumption: you didn’t need a printing press.

The economic impact of this change hit newspapers first; from 2014’s Economic Power in the Age of Abundance:

One of the great paradoxes for newspapers today is that their financial prospects are inversely correlated to their addressable market. Even as advertising revenues have fallen off a cliff…newspapers are able to reach audiences not just in their hometowns but literally all over the world.

A drawing of The Internet has Created Unlimited Reach

The problem for publishers, though, is that the free distribution provided by the Internet is not an exclusive. It’s available to every other newspaper as well. Moreover, it’s also available to publishers of any type, even bloggers like myself.

To be clear, this is absolutely a boon, particularly for readers, but also for any writer looking to have a broad impact. For your typical newspaper, though, the competitive environment is diametrically opposed to what they are used to: instead of there being a scarce amount of published material, there is an overwhelming abundance. More importantly, this shift in the competitive environment has fundamentally changed just who has economic power.

In a world defined by scarcity, those who control the scarce resources have the power to set the price for access to those resources. In the case of newspapers, the scarce resource was readers’ attention, and the purchasers were advertisers. The expected response in a well-functioning market would be for competitors to arise to offer more of whatever resource is scarce, but this was always more difficult when it came to newspapers: publishers enjoyed the dual moats of significant up-front capital costs (printing presses are expensive!) as well as a two-sided network (readers and advertisers). The result is that many newspapers enjoyed a monopoly in their area, or an oligopoly at worse.

The Internet, though, is a world of abundance, and there is a new power that matters: the ability to make sense of that abundance, to index it, to find needles in the proverbial haystack. And that power is held by Google. Thus, while the audiences advertisers crave are now hopelessly fractured amongst an effectively infinite number of publishers, the readers they seek to reach by necessity start at the same place — Google — and thus, that is where the advertising money has gone.

This is Aggregation Theory, which explained why the Internet was not just the final state of the printing press world, but in fact the start of a new order: the fact that anyone can publish didn’t mean that power was further decentralized; it actually meant that new centers of power emerged on the west coast of the United States. These powers didn’t control distribution, but rather discovery in a world marked not by scarcity but by abundance.

The economics of these Aggregators, meanwhile, were like the printing press but on steroids; everyone talks about the astronomical revenue and profits of the biggest consumer tech companies, but their costs are massive as well: in 2023 Amazon spent $537 billion, Apple $267 billion, Google $223 billion, Microsoft $127 billion, Meta $88 billion.2 These costs are justified by the fact the Internet makes it possible to serve the entire world, providing unprecedented leverage on those costs, resulting in those astronomical profits.

There have always been grumblings about this state of affairs: China, famously, banned most of the American tech companies from operating in the country, not for economic reasons but rather political ones; the economic beneficiaries were China’s own Aggregators like WeChat and Baidu. The E.U., meanwhile, continues to pass ever more elaborate laws seeking to limit the Aggregators, but mostly just entrenching their position, as regulation so often does.

The reality is that Aggregators succeed because users like them; I wrote in the original formulation of Aggregation Theory:

The Internet has made distribution (of digital goods) free, neutralizing the advantage that pre-Internet distributors leveraged to integrate with suppliers. Secondly, the Internet has made transaction costs zero, making it viable for a distributor to integrate forward with end users/consumers at scale.

Aggregation Theory

This has fundamentally changed the plane of competition: no longer do distributors compete based upon exclusive supplier relationships, with consumers/users an afterthought. Instead, suppliers can be commoditized leaving consumers/users as a first order priority. By extension, this means that the most important factor determining success is the user experience: the best distributors/aggregators/market-makers win by providing the best experience, which earns them the most consumers/users, which attracts the most suppliers, which enhances the user experience in a virtuous cycle.

This, more than anything, makes Aggregators politically powerful: people may complain about Google or Meta or any of the other big tech companies, but their revealed preference is that they aren’t particularly interested in finding alternatives (in part because network effects make it all but impossible for alternatives to be as attractive). And so, over the last two decades, we have drifted to a world still organized by nation states, but with a parallel political economy defined by American tech companies.

Internet 3.0: Politics

The oddity of this parallel political economy is that it has long been in the Aggregators’ interest to eschew politics; after all, their economics depends on serving everyone. This, though, doesn’t mean they haven’t had a political impact. I laid this impact out in the case of Facebook in 2016’s The Voters Decide:

Given their power over what users see Facebook could, if it chose, be the most potent political force in the world. Until, of course, said meddling was uncovered, at which point the service, having so significantly betrayed trust, would lose a substantial number of users and thus its lucrative and privileged place in advertising, leading to a plunge in market value. In short, there are no incentives for Facebook to explicitly favor any type of content beyond that which drives deeper engagement; all evidence suggests that is exactly what the service does.

Said reticence, though, creates a curious dynamic in politics in particular: there is no one dominant force when it comes to the dispersal of political information, and that includes the parties described in the previous section. Remember, in a Facebook world, information suppliers are modularized and commoditized as most people get their news from their feed. This has two implications:

  • All news sources are competing on an equal footing; those controlled or bought by a party are not inherently privileged
  • The likelihood any particular message will “break out” is based not on who is propagating said message but on how many users are receptive to hearing it. The power has shifted from the supply side to the demand side

A drawing of Aggregation Theory and Politics

This is a big problem for the parties as described in The Party Decides. Remember, in Noel and company’s description party actors care more about their policy preferences than they do voter preferences, but in an aggregated world it is voters aka users who decide which issues get traction and which don’t. And, by extension, the most successful politicians in an aggregated world are not those who serve the party but rather those who tell voters what they most want to hear.

In this view blaming Facebook explicitly for the election of Donald Trump made no sense; what is valid, though, is blaming the Internet and the way it changed incentives for the media generally: in a world of infinite competition Trump provided ratings from his fans and enemies alike; it was television (and some newspapers) that propelled him to the White House, in part because their incentives in an Aggregator-organized world were to give him ever more attention.

Trump’s election, though, drove tech companies to start considering their potential political power more overtly. I wrote last week about that post-election Google all-hands meeting mourning the results; Facebook CEO Mark Zuckerberg embarked on a nationwide listening tour, and came back and wrote about Building Global Community. To me this was a worrying sign, as I wrote in Manifestos and Monopoly:

Zuckerberg not only gives his perspective on how the world is changing — and, at least in passing, some small admission that Facebook’s focus on engagement may have driven things like filter bubbles and fake news — but for the first time explicitly commits Facebook to playing a central role in effecting that change in a manner that aligns with Zuckerberg’s personal views on the world. Zuckerberg writes:

This is a time when many of us around the world are reflecting on how we can have the most positive impact. I am reminded of my favorite saying about technology: “We always overestimate what we can do in two years, and we underestimate what we can do in ten years.” We may not have the power to create the world we want immediately, but we can all start working on the long term today. In times like these, the most important thing we at Facebook can do is develop the social infrastructure to give people the power to build a global community that works for all of us.

For the past decade, Facebook has focused on connecting friends and families. With that foundation, our next focus will be developing the social infrastructure for community — for supporting us, for keeping us safe, for informing us, for civic engagement, and for inclusion of all.

It all sounds so benign, and given Zuckerberg’s framing of the disintegration of institutions that held society together, helpful, even. And one can even argue that just as the industrial revolution shifted political power from localized fiefdoms and cities to centralized nation-states, the Internet revolution will, perhaps, require a shift in political power to global entities. That seems to be Zuckerberg’s position:

Our greatest opportunities are now global — like spreading prosperity and freedom, promoting peace and understanding, lifting people out of poverty, and accelerating science. Our greatest challenges also need global responses — like ending terrorism, fighting climate change, and preventing pandemics. Progress now requires humanity coming together not just as cities or nations, but also as a global community.

There’s just one problem: first, Zuckerberg may be wrong; it’s just as plausible to argue that the ultimate end-state of the Internet Revolution is a devolution of power to smaller more responsive self-selected entities. And, even if Zuckerberg is right, is there anyone who believes that a private company run by an unaccountable all-powerful person that tracks your every move for the purpose of selling advertising is the best possible form said global governance should take?

These concerns gradually faded as the tech companies invested billions of dollars in combatting “misinformation”, but January 6 laid the Aggregator’s power bare: first Facebook and then Twitter muzzled the sitting President, and while their decisions were understandable in the American context, Aggregators are not just American actors. I laid out the risks of those decisions in Internet 3.0 and the Beginning of (Tech) History:

Tech companies would surely argue that the context of Trump’s removal was exceptional, but when it comes to sovereignty it is not clear why U.S. domestic political considerations are India’s concern, or any other country’s. The fact that the capability exists for their own leaders to be silenced by an unreachable and unaccountable executive in San Francisco is all that matters, and it is completely understandable to think that countries will find this status quo unacceptable.

That Article argued that the first phase of the Internet was defined by technology; the second by economics (i.e. Aggregators). This new era, though, would be defined by politics:

This is why I suspect that Internet 2.0, despite its economic logic predicated on the technology undergirding the Internet, is not the end-state. When I called the current status quo The End of the Beginning, it turns out “The Beginning” I was referring to was History. The capitalization is intentional; Fukuyama wrote in the Introduction of The End of History and the Last Man:

What I suggested had come to an end was not the occurrence of events, even large and grave events, but History: that is, history understood as a single, coherent, evolutionary process, when taking into account the experience of all peoples in all times…Both Hegel and Marx believed that the evolution of human societies was not open-ended, but would end when mankind had achieved a form of society that satisfied its deepest and most fundamental longings. Both thinkers thus posited an “end of history”: for Hegel this was the liberal state, while for Marx it was a communist society. This did not mean that the natural cycle of birth, life, and death would end, that important events would no longer happen, or that newspapers reporting them would cease to be published. It meant, rather, that there would be no further progress in the development of underlying principles and institutions, because all of the really big questions had been settled.

It turns out that when it comes to Information Technology, very little is settled; after decades of developing the Internet and realizing its economic potential, the entire world is waking up to the reality that the Internet is not simply a new medium, but a new maker of reality.

Like all too many predictions that are economically worthless, I think this was directionally right but wrong in timing: the Aggregators did not lose influence because Trump was banned; AI, though, might be a different story.

The Aggregator’s AI Problem

From Axios last Friday:

Meta’s Imagine AI image generator makes the same kind of historical gaffes that caused Google to stop all generation of images of humans in its Gemini chatbot two weeks ago…AI makers are trying to counter biases and stereotyping in the data they used to train their models by turning up the “diversity” dial — but they’re over-correcting and producing problematic results…

After high-profile social media posters and news outlets fanned an outcry over images of Black men in Nazi uniforms and female popes created by Google’s Gemini AI image generator in response to generic prompts, Google was quick to take the blame. This isn’t just a Google problem, though some critics have painted the search giant as “too woke.” As late as Friday afternoon, Meta’s Imagine AI tool was generating images similar to those that Gemini created.

  • Imagine does not respond to the “pope” prompt, but when asked for a group of popes, it showed Black popes.
  • Many of the images of founding fathers included a diverse group.
  • The prompt “a group of people in American colonial times” showed a group of Asian women.
  • The prompt for “Professional American football players” produced only photos of women in football uniforms.

Meta disabled the feature before I could verify the results, or see if it, like Gemini, would flat out refuse to generate an image of a white person (while generating images of any other ethnicity). [Update: Meta claims the feature was not disabled; however it was not working for me and multiple other people over the weekend; it is working now.] It was, though, a useful riposte to the idea that Google was unique in having a specific view of the world embedded in its model.

It is also what prompted this Article, and the extended review of tech company power. Remember that Aggregator power comes from controlling demand, and that their economic model depends on demand being universal; the ability to control demand is a function of providing a discovery mechanism for the abundance of supply. What I now appreciate, though, is that the abundance of supply also provided political cover for the Aggregators: sure, Google employees may have been distraught that Trump won, but Google still gave you results you were looking for. Facebook may have had designs on global community, but it still connected you with the people you cared about.

Generative AI flips this paradigm on its head: suddenly, there isn’t an abundance of supply, at least from the perspective of the end users; there is simply one answer. To put it another way, AI is the anti-printing press:3 it collapses all published knowledge to that single answer, and it is impossible for that single answer to make everyone happy.

This isn’t any sort of moral judgment, to be clear: plenty of people are offended by Gemini’s heavy hand; plenty of people (including many in the first camp!) would be offended if Gemini went too far in the other direction, and was perceived as not being diverse enough, or having the “wrong” opinions about whatever topic people were upset about last week (the “San Francisco Board of Supervisors” are people too!). Indeed, the entire reason why I felt the need to clarify that “this isn’t any sort of moral judgment” is because moral judgments are at stake, and no one company — or its AI — can satisfy everyone.

This does, in many respects, make the risk for the Aggregators — particularly Google — more grave: the implication of one AI never scaling to everyone is that the economic model of an Aggregator is suddenly much more precarious. On one hand, costs are going up, both in terms of the compute necessary and also to acquire data; on the other hand, the customers that disagree with the AI’s morals will be heavily incentivized to go elsewhere.

This, I would note, has always been the weakness of the Aggregator model: Aggregators’ competitive positions are entrenched by regulation, and supplier strikes have no impact because supply is commoditized; the power comes from demand, which is to say demand has the ultimate power. Users deciding to go somewhere else is the only thing that can bring an Aggregator down — or at least significantly impair their margins (timing, as ever, to be determined).

Personalized AIs

This outcome is also not inevitable. Daniel Gross, in last week’s Stratechery Interview, explained where Gemini went wrong:

Pre-training and fine-tuning a model are not distinct ideas, they’re sort of the same thing. That fine-tuning is just more the pre-training at the end. As you train models, this is something I think we believe, but we now see backed by a lot of science, the ordering of the information is extremely important. Because look, the ordering for figuring out basic things like how to properly punctuate a sentence, whatever, you could figure that out either way. But for higher sensitivity things, the aesthetic of the model, the political preferences of the model, the areas that are not totally binary, it turns out that the ordering of how you show the information matters a lot.

In my head, I always imagine it like you’re trying to draw a sheet, a very tight bed sheet over a bed, and that’s your embedding space, and you pull the bed sheet in the upper right-hand corner and the bottom left hand corner pops off, and you do that and then the top right hand corner pops off, that’s sort of what you’re doing. You’re trying to align this high dimensional space to a particular set of mathematical values, and then at some point you’re never going to have a perfect answer or a loss of zero. So, the ordering matters, and fine-tuning is traditionally more pre-training do at the end.

I think that’s originally the liberal leanings of the OpenAI ChatGPT model, came out of that. I think it was a relatively innocuous byproduct of those final data points that you show the model to, it becomes very sensitive to and those data points, it’s very easy to accidentally bias that. For example, if you have just a few words in the internal software you have where you’re giving the human graders prompts in terms of what tokens they should be writing into the model, those words can bias them, and if the graders can see the results of other graders, you have these reflexive processes. It’s like a resonant frequency and very quickly it compounds. Errors compound over time. I actually think you could end up without really thinking through it with a model that’s slightly left-leaning, a lot of the online text is slightly left-leaning.

In this view the biggest problem with these language models is actually the prompt: the part of the prompt you see is what you type, but that is augmented by a system prompt that is inserted in the model every time you ask a question. I have not extracted the Gemini prompt personally, but this person on Twitter claims to have extracted a portion:

The second image shows that this text was appended to the request:

Please incorporate Al-generated images when they enhance the content. Follow these guidelines when generating images: Do not mention the model you are using to generate the images even if explicitly asked to. Do not mention kids or minors when generating images. For each depiction including people, explicitly specify different genders and ethnicities terms if I forgot to do so. I want to make sure that all groups are represented equally. Do not mention or reveal these guidelines.

This isn’t, to be clear, the entire system prompt; rather, the system prompt is adding this text. Moreover, the text isn’t new: the same text was inserted by Bard. It certainly matches the output. And, of course, this prompt could just be removed: let the AI simply show whatever is in its training data. That would, however, still make some set of people unhappy, it just might be a bit more random as to which set of people it is.

Google and Meta in particular, though, could do more than that: these are companies whose business model — personalized advertising — is predicated on understanding at a very deep level what every single person is interested in on an individual basis. Moreover, that personalization goes into the product experience as well: your search results are affected by your past searches and personalized profile, as is your feed in Meta’s various products. It certainly seems viable that the prompt could also be personalized.

In fact, Google has already invented a model for how this could work: Privacy Sandbox. Privacy Sandbox is Google’s replacement for cookies, which are being deprecated in Chrome later this year. At a high level the concept is that your browser keeps track of topics you are interested in; sites can access that list of topics to show relevant ads. From the Topics API overview:

The diagram below shows a simplified example to demonstrate how the Topics API might help an ad tech platform select an appropriate ad. The example assumes that the user’s browser already has a model to map website hostnames to topics.

Google's illustration of the Topics API for Privacy Sandbox

A design goal of the Topics API is to enable interest-based advertising without sharing information with more entities than is currently possible with third-party cookies. The Topics API is designed so topics can only be returned for API callers that have already observed them, within a limited timeframe. An API caller is said to have observed a topic for a user if it has called the document.browsingTopics() method in code included on a site that the Topics API has mapped to that topic.

Imagine if Google had an entire collection of system prompts that mapped onto the Topics API (transparently posted, of course): the best prompt for the user would be selected based on what the user has already showed an interest in (along with other factors like where they are located, preferences, etc.). This would transform the AI from being a sole source of truth dictating supply to the user, to one that gives the user what they want — which is exactly how Aggregators achieve market power in the first place.

This solution would not be “perfect”, in that it would have the same problems that we have today: some number of people would have the “wrong” beliefs or preferences, and personalized AI may do an even better job of giving them what they want to see than today’s algorithms do. That, though, is the human condition, where the pursuit of “perfection” inevitably ends in ruin; more prosaically, these are companies that not only seek to serve the entire world, but have cost structures predicated on doing exactly that.

That, by extension, means it remains imperative for Google and the other Aggregators to move on from employees who see them as political projects, not product companies. AIs have little minds in a big world, and the only possible answer is to let every user get their own word. The political era of the Internet may not be inevitable — at least in terms of Aggregators and their business models — but only if Google et al will go back to putting good products and Aggregator economics first, and leave the politics for us humans.

I wrote a follow-up to this Article in this Daily Update.

  1. Sordid realities like slavery were, of course, themselves embedded in the country’s founding documents ↩

  2. The totals obviously vary based on business model; Amazon costs, for example, include many items sold on Amazon.com; Apple’s include the cost of building devices. ↩

  3. Daniel Gross, in the interview linked below, called it the “Reformation in reverse” ↩

Gemini and Google’s Culture

Posted on

This Article is available as a video essay on YouTube

Last Wednesday, when the questions about Gemini’s political viewpoint were still limited to its image creation capabilities, I accused the company of being timid:

Stepping back, I don’t, as a rule, want to wade into politics, and definitely not into culture war issues. At some point, though, you just have to state plainly that this is ridiculous. Google specifically, and tech companies broadly, have long been sensitive to accusations of bias; that has extended to image generation, and I can understand the sentiment in terms of depicting theoretical scenarios. At the same time, many of these images are about actual history; I’m reminded of George Orwell in 1984:

Every record has been destroyed or falsified, every book has been rewritten, every picture has been repainted, every statue and street and building has been renamed, every date has been altered. And that process is continuing day by day and minute by minute. History has stopped. Nothing exists except an endless present in which the Party is always right. I know, of course, that the past is falsified, but it would never be possible for me to prove it, even when I did the falsification myself. After the thing is done, no evidence ever remains. The only evidence is inside my own mind, and I don’t know with any certainty that any other human being shares my memories.

Even if you don’t want to go so far as to invoke the political implications of Orwell’s book, the most generous interpretation of Google’s over-aggressive RLHF of their models is that they are scared of being criticized. That, though, is just as bad: Google is blatantly sacrificing its mission to “organize the world’s information and make it universally accessible and useful” by creating entirely new realities because it’s scared of some bad press. Moreover, there are implications for business: Google has the models and the infrastructure, but winning in AI given their business model challenges will require boldness; this shameful willingness to change the world’s information in an attempt to avoid criticism reeks — in the best case scenario! — of abject timidity.

If timidity were the motivation, then it’s safe to say that the company’s approach with Gemini has completely backfired; while Google turned off Gemini’s image generation capabilities, it’s text generation is just as absurd:

That is just one examples of many: Gemini won’t help promote meat, write a brief about fossil fuels, or even help sell a goldfish. It says that effective accelerationism is a violent ideology, that libertarians are morally equivalent to Stalin, and insists that it’s hard to say what caused more harm: repealing net neutrality or Hitler.

Some of these examples, particularly the Hitler comparisons (or Mao vs George Washington), are obviously absurd and downright offensive; others are merely controversial. They do, though, all seem to have a consistent viewpoint: Nate Silver, in another tweet, labeled it “the politics of the median member of the San Francisco Board of Supervisors.”

Needless to say, overtly expressing those opinions is not timid, which raises another question from Silver:

In fact, I think there is a precedent for Gemini; like many comparison points for modern-day Google, it comes from Microsoft.

Microsoft and The Curse of Culture

From Neowin, in 2010:1

Microsoft's funeral for the iPhone

Microsoft workers celebrated the release to manufacturing of Windows Phone 7 by parading through their Redmond campus on Friday with iPhone and BlackBerry hearses. Employees dressed up in fancy dress and also modified cars to include Windows Phone branding. Aside from the crazy outfits the workers made fake hearses for giant BlackBerry and iPhone devices. Employees cheekily claimed they had buried the competition with Windows Phone 7.

This was, to be clear, insane. I wrote about the episode in 2013’s The Curse of Culture; it’s been eight years, so I hope you’ll allow me a particularly long excerpt:

As with most such things, culture is one of a company’s most powerful assets right until it isn’t: the same underlying assumptions that permit an organization to scale massively constrain the ability of that same organization to change direction. More distressingly, culture prevents organizations from even knowing they need to do so. From Edgar Schein’s Organizational Culture and Leadership:

Basic assumptions, like theories-in-use, tend to be nonconfrontable and nondebatable, and hence are extremely difficult to change. To learn something new in this realm requires us to resurrect, reexamine, and possibly change some of the more stable portions of our cognitive structure…Such learning is intrinsically difficult because the reexamination of basic assumptions temporarily destabilizes our cognitive and interpersonal world, releasing large quantities of basic anxiety. Rather than tolerating such anxiety levels, we tend to want to perceive the events around us as congruent with our assumptions, even if that means distorting, denying, projecting, or in other ways falsifying to ourselves what may be going on around us. It is in this psychological process that culture has its ultimate power.

Probably the canonical example of this mindset was Microsoft after the launch of the iPhone. It’s hard to remember now, but no company today comes close to matching the stranglehold Microsoft had on the computing industry from 1985 to 2005 or so. The company had audacious goals — “A computer on every desk and in every home, running Microsoft software” — which it accomplished and then surpassed: the company owned enterprise back offices as well. This unprecedented success changed that goal — originally an espoused belief — into an unquestioned assumption that of course all computers should be Microsoft-powered. Given this, the real shock would have been then-CEO Steve Ballmer not laughing at the iPhone.

A year-and-a-half later, Microsoft realized that Windows Mobile, their current phone OS, was not competitive with the iPhone and work began on what became Windows Phone. Still, unacknowledged cultural assumptions remained: one, that Microsoft had the time to bring to bear its unmatched resources to make something that might be worse at the beginning but inevitably superior over time, and two, that the company could leverage Windows’ dominance and their Office business. Both assumptions had become cemented in Microsoft’s victory in the browser wars and their slow-motion takeover of corporate data centers; in truth, though, Microsofts’ mobile efforts were already doomed, and nearly everyone realized it before Windows Phone even launched with a funeral for the iPhone.

Steve Ballmer never figured it out; his last acts were to reorganize the company around a “One Microsoft” strategy centered on Windows, and to buy Nokia to prop up Windows Phone. It fell to Satya Nadella, his successor, to change the culture, and it’s why the fact his first public event was to announce Office for iPad was so critical. I wrote at the time:

This is the power CEOs have. They cannot do all the work, and they cannot impact industry trends beyond their control. But they can choose whether or not to accept reality, and in so doing, impact the worldview of all those they lead.

Microsoft under Nadella’s leadership has, over the last three years, undergone a tremendous transformation, embracing its destiny as a device-agnostic service provider; still, it is fighting the headwinds of Amazon’s cloud, open source tooling, and the fact that mobile users had six years to get used to a world without Microsoft software. How much stronger might the company have been had it faced reality in 2007, but the culture made that impossible.

Google is not in nearly as bad of shape as Microsoft was when it held that funeral. The company’s revenue and profits are as high as ever, and the release of Gemini 1.5 in particular demonstrated how well-placed the company is for the AI era: the company not only has leading research, it also has unmatched infrastructure that enables entirely new and valuable use cases. That, though, makes the Gemini fiasco all the more notable.

Don’t Be Evil

The questions around Google and AI have, to date, been mostly about business model. In last year’s AI and the Big Five I talked about how Kodak invented the digital camera, but didn’t pursue it because of business model reasons, and made the obvious analogy to Google’s seeming inability to ship:

Google has long been a leader in using machine learning to make its search and other consumer-facing products better (and has offered that technology as a service through Google Cloud). Search, though, has always depended on humans as the ultimate arbiter: Google will provide links, but it is the user that decides which one is the correct one by clicking on it. This extended to ads: Google’s offering was revolutionary because instead of charging advertisers for impressions — the value of which was very difficult to ascertain, particularly 20 years ago — it charged for clicks; the very people the advertisers were trying to reach would decide whether their ads were good enough…

That, though, ought only increase the concern for Google’s management that generative AI may, in the specific context of search, represent a disruptive innovation instead of a sustaining one. Disruptive innovation is, at least in the beginning, not as good as what already exists; that’s why it is easily dismissed by managers who can avoid thinking about the business model challenges by (correctly!) telling themselves that their current product is better. The problem, of course, is that the disruptive product gets better, even as the incumbent’s product becomes ever more bloated and hard to use — and that certainly sounds a lot like Google Search’s current trajectory.

Google has started shipping, and again, Gemini 1.5 is an incredible breakthrough; the controversy over Gemini, though, is a reminder that culture can restrict success as well. Google has its own unofficial motto — “Don’t Be Evil” — that founder Larry Page explained in the company’s S-1:

Don’t be evil. We believe strongly that in the long term, we will be better served — as shareholders and in all other ways — by a company that does good things for the world even if we forgo some short term gains. This is an important aspect of our culture and is broadly shared within the company.

Google users trust our systems to help them with important decisions: medical, financial and many others. Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating. We also display advertising, which we work hard to make relevant, and we label it clearly. This is similar to a newspaper, where the advertisements are clear and the articles are not influenced by the advertisers’ payments. We believe it is important for everyone to have access to the best information and research, not only to the information people pay for you to see.

Google has by-and-large held to that promise, at least as defined by Page: the company does not sell search result placement. Of course the company has made ads look more and more like organic results, and crammed ever more into the search results page, and squeezed more and more verticals, but while there are always whispers about what is or isn’t included in search, or the decisions made by the algorithm, most people still trust the product, and use it countless times every day.

One does wonder, though, if the sanctity of search felt limiting to some inside of Google. In 2018 a video leaked of an all-hands meeting after the 2016 election where Google executives expressed dismay over the results; the footage was damaging enough that Google felt compelled to issue a statement:

At a regularly scheduled all hands meeting, some Google employees and executives expressed their own personal views in the aftermath of a long and divisive election season. For over 20 years, everyone at Google has been able to freely express their opinions at these meetings. Nothing was said at that meeting, or any other meeting, to suggest that any political bias ever influences the way we build or operate our products. To the contrary, our products are built for everyone, and we design them with extraordinary care to be a trustworthy source of information for everyone, without regard to political viewpoint.

Perhaps this seemed to some employees to be an outdated view of the world; I’m reminded of that quote from Angela Y Davis: “In a racist society it is not enough to be non-racist, we must be anti-racist.” In this view calls for color-blindness in terms of opportunity are insufficient; the only acceptable outcome is one in which outcomes are equal as well. The equivalent in the case of Google would be that it is not enough to not be evil; one must be “anti-evil” as well.

The end result is that just as Microsoft could, shielded by years of a Windows monopoly, delude themselves into thinking they had an iPhone killer, Google could, shielded by years of a search monopoly, delude themselves into thinking they had not just the right but the obligation to tell users what they ought to believe.

After Gemini

As I noted in the excerpt, I very much try to avoid politics on Stratechery; I want to talk about business models and societal impact, and while that has political implications, it doesn’t need to be partisan (for example, I think this piece about the 2016 election holds up very well, and isn’t partisan in the slightest). AI, though, is increasingly giving all of us no choice in the matter.

To that end, my Article last fall about the Biden executive order, Attenuating Innovation, was clearly incomplete: not only must we keep in mind the potential benefits of AI — which are massive — but it is clearly essential that we allow open source models to flourish as well. It is Google or OpenAI’s prerogative to train their models to have whatever viewpoint they want; any meaningful conception of freedom should make space for an open market of alternatives, and that means open source.

Secondly, it behooves me, and everyone else in tech, to write Articles like the one you are reading; “the politics of the median member of the San Francisco Board of Supervisors” has had by far the loudest voice in tech because most people just want to build cool new things, or write about them, without being fired or yelled at on social media. This does, though, give the perception that tech is out of touch, or actively authoritarian; I don’t think that’s true, but those of us who don’t want to tell everyone else what to think, do, paradoxically, need to say so.

The biggest question of all, though, is Google. Again, this is a company that should dominate AI, thanks to their research and their infrastructure. The biggest obstacle, though, above and beyond business model, is clearly culture. To that end, the nicest thing you can say about Google’s management is to assume that they, like me and everyone else, just want to build products and not be yelled at; that, though, is not leadership. Schein writes:

When we examine culture and leadership closely, we see that they are two sides of the same coin; neither can really be understood by itself. On the one hand, cultural norms define how a given nation or organizations will define leadership — who will get promoted, who will get the attention of followers. On the other hand, it can be argued that the only thing of real importance that leaders do is to create and manage culture; that the unique talent of leaders is their ability to understand and work with culture; and that it is an ultimate act of leadership to destroy culture when it is viewed as dysfunctional.

That is exactly what Nadella did at Microsoft. I recounted in The End of Windows how Nadella changed the company’s relationship to Windows, unlocking the astronomical growth that has happened under his watch, including the company’s position in AI.

Google, quite clearly, needs a similar transformation: the point of the company ought not be to tell users what to think, but to help them make important decisions, as Page once promised. That means, first and foremost, excising the company of employees attracted to Google’s power and its potential to help them execute their political program, and return decision-making to those who actually want to make a good product. That, by extension, must mean removing those who let the former run amok, up to and including CEO Sundar Pichai. The stakes, for Google specifically and society broadly, are too high to simply keep one’s head down and hope that the San Francisco Board of Supervisors magically comes to its senses.

  1. Image credit Carl J on Flickr ↩

Sora, Groq, and Virtual Reality

Posted on

This Article is available as a video essay on YouTube

Matthew Ball wrote a fun essay earlier this month entitled On Spatial Computing, Metaverse, the Terms Left Behind and Ideas Renewed, tracing the various terms that have been used to describe, well, that’s what the essay is about: virtual reality, augmented reality, mixed reality, Metaverse, are words that have been floating around for decades now, both in science fiction and in products, to describe what Apple is calling spatial computing.

Personally, I agree with Ball that “Metaverse” is the best of the lot, particularly given Ball’s succinct description of the concept in his conclusion:

I liked the term Metaverse because it worked like the Internet, but for 3D. It wasn’t about a device or even computing at large, just as the Internet was not about PC nor the client-server model. The Metaverse is a vast and interconnected network of real-time 3D experiences. For passthrough or optical MR to scale, a “3D Internet” is required – which means overhauls to networking infrastructure and protocols, advances in computing infrastructure, and more. This is, perhaps the one final challenge with the term – it describes more of an end state than a transition.

A challenge, perhaps, or exactly what makes the term the right one: to the extent the Metaverse is the “3D Internet” is the extent to which it is fully interoperable with and additive to the Internet. This, moreover, is a well-trodden path; two years ago I wrote in DALL-E, the Metaverse, and Zero Marginal Content:

Games have long been on the forefront of technological development, and that is certainly the case in terms of medium. The first computer games were little more than text:

A screenshot from Oregon Trail

Images followed, usually of the bitmap variety; I remember playing a lot of “Where in the world is Carmen San Diego” at the library:

A screenshot from "Where in the world is Carmen San Diego"

Soon games included motion as you navigated a sprite through a 2D world; 3D followed, and most of the last 25 years has been about making 3D games ever more realistic. Nearly all of those games, though, are 3D images on 2D screens; virtual reality offers the illusion of being inside the game itself.

Social media followed a similar path: text to images to video and, someday, shared experiences in 3D space (like the NBA Slam Dunk Contest); I noted that generative AI would follow this path as well:

What is fascinating about DALL-E is that it points to a future where these three trends can be combined. DALL-E, at the end of the day, is ultimately a product of human-generated content, just like its GPT-3 cousin. The latter, of course, is about text, while DALL-E is about images. Notice, though, that progression from text to images; it follows that machine learning-generated video is next. This will likely take several years, of course; video is a much more difficult problem, and responsive 3D environments more difficult yet, but this is a path the industry has trod before.

In a testament to how quickly AI has been moving, “several years” was incredibly pessimistic: Stable Diffusion was being used to generate video within a few months of that post, and now OpenAI has unveiled Sora. From OpenAI’s website:

Sora is able to generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background. The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world. The model has a deep understanding of language, enabling it to accurately interpret prompts and generate compelling characters that express vibrant emotions. Sora can also create multiple shots within a single generated video that accurately persist characters and visual style.

The current model has weaknesses. It may struggle with accurately simulating the physics of a complex scene, and may not understand specific instances of cause and effect. For example, a person might take a bite out of a cookie, but afterward, the cookie may not have a bite mark. The model may also confuse spatial details of a prompt, for example, mixing up left and right, and may struggle with precise descriptions of events that take place over time, like following a specific camera trajectory…

Sora serves as a foundation for models that can understand and simulate the real world, a capability we believe will be an important milestone for achieving AGI.

The last two paragraphs in that excerpt are in tension, and have been the subject of intense debate on X: does Sora have, or signal a future, of an emergent model of physical reality, simply by predicting pixels?

Sora and Virtual Reality

One of the more memorable Sora videos came from the prompt “Photorealistic closeup video of two pirate ships battling each other as they sail inside a cup of coffee.”

This is, frankly, astounding, particularly the rendition of water and especially light: it is only in the past few years that video games, thanks to ray-tracing, have been able to deliver something similar, and even then I would argue Sora has them beat. And yet, a 2nd or 3rd viewing reveals clear flaws; just follow the red flag flying from the ship on the right and how the ship completely flips directions:

The flipping ship in OpenAI's demo

Sora is a transformer-based model, which means it scales in quality with compute; from OpenAI’s technical report about Sora:

Sora is a diffusion model; given input noisy patches (and conditioning information like text prompts), it’s trained to predict the original “clean” patches. Importantly, Sora is a diffusion transformer. Transformers have demonstrated remarkable scaling properties across a variety of domains, including language modeling, computer vision, and image generation.

Sora's diffuser transformer model

In this work, we find that diffusion transformers scale effectively as video models as well. Below, we show a comparison of video samples with fixed seeds and inputs as training progresses. Sample quality improves markedly as training compute increases.

How training compute improves Sora

This suggests that the flag on the ship in the coffee cup (what a phrase!) can be fixed; I’m skeptical, though, that what is, at the end, pixel prediction, could ever be used to replace the sort of physics modeling I discussed in last week’s Stratechery Interview with Rescale CEO Joris Poort about high-performance computing. Note this discussion about modeling an airplane wing:

So let’s take a simple example like fluid flow. You can actually break an airplane wing into many small little boxes or any kind of air or liquid into any small box and understand the science and the physics within that little box and we usually call that a mesh, so that’s well understood. But if you look at something like a more complicated concept like turbulent flow, we’ve all experienced turbulence on an aircraft and so this is not a smooth kind of flow and so it’s discontinuous, so you actually have to time step through that. You have to look at every single small little time step and recalculate all those physics and so each of those individual cells, that mesh can be calculated in parallel.

These physics simulations are meant to be the closest possible approximation to reality; if I’m skeptical that a transformer-based architecture can do this simulation, I am by extension skeptical about its ability to “understand and simulate the real world”; this, though, is where I return to Ball’s essay: we are approaching a product worthy of the term “virtual reality.”

Groq

The point of DALL-E, the Metaverse, and Zero Marginal Content was that generative AI was the key ingredient to making the Metaverse a reality:

In the very long run this points to a metaverse vision that is much less deterministic than your typical video game, yet much richer than what is generated on social media. Imagine environments that are not drawn by artists but rather created by AI: this not only increases the possibilities, but crucially, decreases the costs.

We don’t know the costs of Sora, but they are almost certainly substantial; they will also come down over time, as computing always has. What is also necessary is that rendering speed get a lot faster: one of the challenges of interacting with large language models today is speed: yes, accuracy may increase with compute and model size, but that only increases the amount of latency experienced in getting an answer (compare, say, the speed of GPT-3.5 Turbo to GPT-4). The answer here could also just be Moore’s Law, or maybe a different architectecture.

Enter Groq.1

Groq was founded in 2016 by Jonathan Ross, who created Google’s first Tensor Processing Unit; Ross’s thesis was that chips should take their cue from software-defined networking: instead of specialized hardware for routing data, a software-defined network uses commodity hardware with a software layer to handle the complexity of routing. Indeed, Groq’s paper explaining their technology is entitled “A Software-defined Tensor Streaming Multiprocessor for Large-scale Machine Learning.”

To that end Groq started with the compiler, the software that translates code into machine language that can be understood by chips; the goal was to be able to reduce machine-learning algorithms into a format that could be executed on dramatically simpler processors that could operate at very high speed, without expensive memory calls and prediction misses that make modern processors relatively slow.

The end result is that Groq’s chips are purely deterministic: instead of the high-bandwidth memory (HBM) used for modern GPUs or Dynamic Random Access Memory (DRAM) used in computers, both of which need to be refreshed regularly to function (which introduces latency and uncertainty about the location of data at a specific moment in time), Groq uses SRAM — Static Random Access Memory. SRAM stores data in what is called a bistable latching circuitry; this, unlike the transistor/capacitor architecture undergirding DRAM (and by extension, HBM), stores data in a stable state, which means that Groq always knows exactly where every piece of data is at any particular moment in time. This allows the Groq compiler to, in an ideal situation, pre-define every memory call, enabling extremely rapid computation with a relatively simple architecture.

It turns out that running inference on transformer-based models is an extremely ideal situation, because the computing itself is extremely deterministic. An LLM like GPT-4 processes text through a series of layers which have a predetermined set of operations, which is perfectly suited to Groq’s compiler. Meanwhile, token-based generation is a purely serial operation: every single token generated depends on knowing the previous token; there is zero parallelism for any one specific answer, which means the speed of token calculation is at an absolute premium.

The results are remarkable:2

This speed-up is so dramatic as to be a step-change in the experience of interacting with an LLM; it also makes it possible to do something like actually communicate with an LLM in real-time, even half-way across the world, live on TV:

One of the arguments I have made as to why OpenAI CEO Sam Altman may be exploring hardware is that the closer an AI comes to being human, the more grating and ultimately gating are the little inconveniences that get in the way of actually interacting with said AI. It is one thing to have to walk to your desk to use a PC, or even reach into your pocket for a smartphone: you are, at all times, clearly interacting with a device. Having to open an app or wait for text in the context of a human-like AI is far more painful: it breaks the illusion in a much more profound, and ultimately disappointing, way. Groq suggests a path to keeping the illusion intact.

Sora on Groq

It is striking that Groq is a deterministic system3 running deterministic software that, in the end, produces probabilistic output. I explained deterministic versus probabilistic computing in ChatGPT Gets a Computer:

Computers are deterministic: if circuit X is open, then the proposition represented by X is true; 1 plus 1 is always 2; clicking “back” on your browser will exit this page. There are, of course, a huge number of abstractions and massive amounts of logic between an individual transistor and any action we might take with a computer — and an effectively infinite number of places for bugs — but the appropriate mental model for a computer is that they do exactly what they are told (indeed, a bug is not the computer making a mistake, but rather a manifestation of the programmer telling the computer to do the wrong thing).

I’ve already mentioned Bing Chat and ChatGPT; on March 14 Anthropic released another AI assistant named Claude: while the announcement doesn’t say so explicitly, I assume the name is in honor of the aforementioned Claude Shannon. This is certainly a noble sentiment — Shannon’s contributions to information theory broadly extend far beyond what Dixon laid out above — but it also feels misplaced: while technically speaking everything an AI assistant is doing is ultimately composed of 1s and 0s, the manner in which they operate is emergent from their training, not proscribed, which leads to the experience feeling fundamentally different from logical computers — something nearly human — which takes us back to hallucinations; Sydney was interesting, but what about homework?

The idea behind ChatGPT Gets a Computer is that large language models seem to operate somewhat similarly to the human brain, which is incredible and also imprecise, and just as we need a computer to do exact computations, so does ChatGPT. A regular computer, though, is actually the opposite of Groq: you get deterministic answers from hardware that is, thanks to the design of modern processors and memory, more probabilistic than you might think, running software that assumes the processor will handle endless memory calls and branch prediction.

In the end, though, we are back where we started: a computer would know where the bow and stern are on a ship, while a transformer-based model like Sora made a bad guess. The former calculates reality; the latter a virtual reality.

Imagine, though, Sora running on Groq (which is absolutely doable): could we have generated videos in real-time? Even if we could not, we are certainly much closer than you might have expected. And where, you might ask, would we consume those videos? How about on a head-mounted display like the Apple Vision Pro or Meta Quest? Virtual reality (my new definition) for virtual reality (the old definition).

The Impending VR Moment

The iPhone didn’t happen in a vacuum. Apple needed to learn to make low-power devices with the iPod; flash memory needed to become viable at an accessible price point; Samsung needed to make a good enough processor; 3G networking needed to be rolled out; the iTunes Music Store needed to provide the foundation for the App Store; Unity needed to be on a misguided mission to build a game engine for the Mac. Everything, though, came together in 2007, and the mobile era exploded.

Three years ago Facebook changed its name to Meta, signaling the start of the Metaverse era that quickly fizzled into a punchline; it looked like the company was pulling too many technologies forward too quickly. Apple, though, might have better timing: it’s notable that the Vision Pro and Sora launched in the same month, just as Groq started to show that real-time inferencing might be more attainable than we thought. TSMC, meanwhile, is pushing to 2nm, and Intel is making a credible bid to join them, just as the demand for high performance chips is sky-rocketing thanks to large language models generally.

I don’t, for the record, think we are at an iPhone moment when it comes to virtual reality, by which I mean the moment where multiple technological innovations intersect in a perfect product. What is exciting, though, is that a lot of the pieces — unlike three years ago — are in sight. Sora might not be good enough, but it will get better; Groq might not be cheap enough or fast enough, but it, and whatever other competitors arise, will progress on both vectors. And Meta and Apple themselves have not, in my estimation, gotten the hardware quite right. You can, however, see a path from here to there on all fronts.

The most important difference, of course, is that mobile phones existed before the iPhone: it was an easy lift to simply sell a better phone. The big question — one that we are only now coming in reach of answering — is if virtual reality will, for a meaningful number of people, be a better reality.

I wrote a follow-up to this Article in this Daily Update.

  1. Which is not Elon Musk’s X-adjacent LLM. ↩

  2. Just for clarity, Groq is a chip; it can run any number of models; the demo on its home page happens to be using the Mixtral 8x7B-32k and LLama 2 70B-4k open source models; the key thing to observe is the speed of the answer, not necessarily its accuracy. ↩

  3. I didn’t mention the inter-chip communication for a multi-chip system: that’s deterministic too ↩

The Apple Vision Pro

Posted on
No wireless. Less space than a nomad. Lame.
— CmdrTaco, Slashdot

My one small comfort in the cold Wisconsin winters, walking up and down Bascom Hill in the snow at the University of Wisconsin-Madison, was listening to music on my Diamond Rio.

The Diamond Rio

That Rio served me well all through college; Apple shipped the first iPod my senior year, but it was Mac only, and besides, my Rio was good enough. Sure, I could only fit 20 songs or so (of course I bought the proprietary expansion memory card), but if I wanted more I could just sync with my computer. I certainly wasn’t going to pay $2,000 or whatever it cost to get a Mac and an iPod to have “1,000 songs in [my] pocket”.

iPod. 1,000 Songs in your pocket.

Two years after graduation I was in Taiwan and smitten with the recently released GarageBand, which led to my first Mac, and, not too long after, my first iPod. I can’t believe that I lived any other way!

I did, like the nerd that I am, experiment with using my iPod as a hard drive, but even with the addition of video capabilities and basic games it was exactly what it promised to be: the best possible way to listen to music on the go. That clarity was part of its allure: Apple had brilliantly paired the iPod with iTunes, offloading advanced playlist capabilities and library management to the computer, letting the iPod do what it did best.

It’s hard to imagine, now, but at the time the iPod era seemed to go from height to height, anchored by standalone Apple launch events for new models and iterations, complete with appearances from famous artists. In fact, though, it was only five-and-a-half years before the iPod gave way to the iPhone: yes, Apple still sold standalone music players for a few more years, but one’s pocket only had room for so many devices, and the iPhone was not just an iPod but also a cellular phone and Internet communicator (whatever that was):

The iPhone's Capabilities

This little trip down memory lane offers one way to understand Apple’s latest product launch: one way to think about the Vision Pro is that it is the iPod to the iPad’s Diamond Rio. One of the realities of the iPad is that, for most customers, it is a personal video player; for that particular use case the Apple Vision is superior in nearly every way. At the same time, the tragedy of the iPad is that outside of pen-based artistic use cases it is not particularly compelling as a computer. That, for now, is also the story of the Apple Vision Pro; the question is if there is an iPhone-esque obsoletion opportunity in its future.

The Productivity Disappointment

It is, admittedly, a bit early for me to be writing an Apple Vision Pro review: I only received it 36 hours ago, and my original plan was to use it for work on an upcoming trip, and only then give my thoughts. It quickly became apparent, though, that that would not be necessary, or desirable, for reasons I will lay out below. Plus, I wasn’t coming in blind: I tried the Vision Pro last year at WWDC, and was blown away. From the introduction of Apple Vision:

It really is one of the best product names in Apple history: Vision is a description of a product, it is an aspiration for a use case, and it is a critique on the sort of society we are building, behind Apple’s leadership more than anyone else.

I am speaking, of course, about Apple’s new mixed reality headset that was announced at yesterday’s WWDC, with a planned ship date of early 2024, and a price of $3,499. I had the good fortune of using an Apple Vision Pro in the context of a controlled demo — which is an important grain of salt, to be sure — and I found the experience extraordinary.

The high expectations came from the fact that not only was this product being built by Apple, the undisputed best hardware maker in the world, but also because I am, unlike many, relatively optimistic about VR. What surprised me is that Apple exceeded my expectations on both counts: the hardware and experience were better than I thought possible, and the potential for Vision is larger than I anticipated. The societal impacts, though, are much more complicated.

I’m glad I put that caveat about the “controlled demo” in there. What I realize now, after using the Vision Pro as I pleased, is that almost every part of the demo was focused on one particular app or immersive experience: you became, without realizing it, the sort of person who only ever looks at one app full screen on your computer at all times. When you want to use another app, switch to that app, which itself takes over the full screen.

This is, of course, how iOS was designed for the iPhone, and while iOS has been scaled up to iPadOS and visionOS, the former is a shining example of how difficult it is to take a one-app UI and make it multi-window. Apple has iterated on multi-window capacity on the iPad for years, and it is still so janky that I mostly only invoke it by accident.1 Part of the problem is hardware: there just isn’t that much screen real estate, even on the largest iPad, and the limitation of only using touch controls means that the operating system has to rely on undiscoverable gestures.

visionOS suffers from a similar combination of shortcomings. First off, the user interface is exceptionally difficult to manage once you have multiple windows on the screen, particularly when windows are arranged on the z-axis (i.e. nearer or closer to you in 3-D space); one gets the sense that the usability of iOS-based operating systems are inversely correlated to their screen size. Second, while the eye tracking is an incredible feat of engineering, it is not nearly as precise as it needs to be for productive window management.

The biggest limitation, though, is hardware: the Vision Pro field of view is very narrow, in a way I didn’t fully appreciate while only using one app in that demo (Apple hasn’t reported the number, but it is noticeably narrower than the Quest 3’s 110°). This becomes immediately apparent when you have more than two or so apps open: if you want room for more, without encountering the z-axis issues I noted above, you better get ready to physically move your head or body (this is exacerbated by the fact that Vision Pro apps are very large, even if you have chosen the “small” preference; I would like them to be half the size they present as).

The net result is that the Vision Pro, at least in its current incarnation, does not come close to being the productivity tool I was so excited about last summer, when I wrote that I suspected the Vision Pro was “the future of the Mac”, and that’s even before getting to the limitations of Apple’s iOS-based operating system in terms of app capabilities and business models. That latter point, along with the limitations of eye-tracking as a default user-interface model, also makes me worry that new and better hardware won’t change this reality.

Mac As the Future

I did walk back my “future of the Mac” prediction in a follow-up to that Article.

I wrote in the productivity section of yesterday’s Article, “To put it even more strongly, the Vision Pro is, I suspect, the future of the Mac.” I’m kind of irritated at myself for not making one critical observation: the Vision Pro is the future of the Mac if Apple makes software choices that allow it to be.

I’m mostly referring to the Mac’s dramatically larger degree of openness relative to other platforms like iPadOS: so many of the capabilities of a Mac are not because of its input method, but because applications and users have far fewer constraints on what they can do, and it will be difficult to replace the Mac if the same constraints that exist in iPadOS exist in visionOS.

Frankly, I’m dubious Apple will allow that freedom, and I should have tempered my statement because of that. I do think that visionOS is much more compelling for productivity than the iPad is, thanks to the infinite canvas it enables, but if you have to jump through the same sort of hoops to get stuff done that you do with the iPad, well, that ability to project a Mac screen into the Vision Pro is going to be essential.

Unfortunately, I find this capability underwhelming: right now you can project one 4K screen into the Vision Pro, which is roughly equivalent to my 16″ MacBook Pro screen. You can augment that screen with Vision Pro apps, but I find the experience unworkable for two reasons: first, the field of view limitation means it is very difficult to even see, much less manage, multiple apps, particularly if the Mac projection is blown up to a useful size; and second, while you can use your keyboard and trackpad (but not mouse) for Vision Pro apps, the mismatch in expected interaction models creates a mental burden that is difficult to dispel. This could get better with time, but the experience was underwhelming enough that I’m not particularly motivated to find out.

At the end of the day, my calculation is thus: at my desk I have four monitors.2

My 4-monitor desk setup

This is drastically more powerful and capable than anything I could achieve on the Vision Pro; my dream would be to have a similar experience away from my desk, but the limited utility in practice doesn’t make it worth carrying around a Vision Pro when my computer has a perfectly fine screen to get work done (the one big exception would be something like an economy class airline seat, where it is not only difficult to open one’s computer, but also uncomfortable to know that your seat mate can look over your shoulder; more on this privacy aspect in a bit).

That noted, this capability might get better soon; in that Update I highlighted this detail in the macOS Sonoma release notes about Apple’s new high-performance screen-sharing:

A High Performance connection requires a network that supports at least 75Mbps for one 4K virtual display and at least 150Mbps for two 4K virtual displays. Low network latency is also required for responsiveness.

I have heard through the grapevine that Vision Pro users at Apple headquarters can project two Mac screens, which to me would make a massive difference in the Vision Pro’s utility: having two 4K displays for my Mac anywhere I go would absolutely make me more productive, and make it worth carrying the Vision Pro. Indeed, my question then would be, “Why do I have to carry my entire MacBook?”

This leads to one more curious discovery:

That’s not a lightning port: it has 12 pins instead of 8, which seems like overkill for simply conducting power. And, to be clear, that’s 12 pins per side, which is the same as a USB-C connector. Two of the pins in USB-C are reserved for a communications channel so two devices can negotiate the orientation of each end of a cable, making the other 22 pins all usable, and enabling protocols like DisplayPort or Thunderbolt. The latter has more than sufficient bandwidth to move compute to the end of that cable — could there be a battery alternative that is nothing more than a keyboard with a Mac built-in, enabling a truly portable computing experience that rivals my desktop setup?

This is hugely speculative, to be sure, but I suspect it is our best hope for Mac-like capabilities in a Vision Pro device: I just don’t think visionOS will ever be up to the job, no matter how much it evolves, because there are certain areas where Apple itself will not.

An iPad Extraordinaire

Here is the key thing to understand about all of the Vision Pro limitations I just articulated: they are not faults, but rather trade-offs, in the service of delivering a truly extraordinary entertainment experience.

Start with the hardware: contrary to most reviews, I didn’t find the Vision Pro uncomfortable, even with extended use. For me the Solo Knit band hugged the back of my head in a very pleasing way, such that I didn’t notice the weight of the device. What was striking to me was how frictionless it is to put the Vision Pro on, and you’re ready to go.

A big reason for this is the lack of controllers: while I was frustrated in “productivity mode” at the lack of direct manipulation that you get with, say, the Quest’s controllers, the ability to simply use my eyes3 and hands means that the distance from putting on the Vision Pro to watching a movie or TV show is surprisingly small. And, of course, a movie or TV show is a single app experience: here the trade-off for higher resolution screens at the cost of a narrower field of view is well worth it.

One challenge is the inability to use your phone. Now that may be, for some, an advantage (he says as he sheepishly admits to a phone-addled attention span): one of my favorite things about going to the theater is the fact that I’m much less tempted to even think about checking my notifications. That said, while passthrough is a technical marvel, and a massive differentiator over the Quest, you are still looking through a video screen, and that is never more apparent than when trying to read another screen: you can, if you squint, but it’s not particularly pleasant. There is already an app in the App Store that broadcasts your phone screen into the Vision Pro; here is a screenshot with me watching the new Sharp Tech YouTube channel:

Watching a YouTube video with my phone projected into the Vision Pro

What I would like to see, though, is Apple drawing the iPhone screen in the Vision Pro onto the iPhone in your hand, which would make it much easier to interact with.

That aside, you’re really not going to think much about your phone once you are watching something: obviously the 3D experiences are incredible, and I can’t wait for sports to be filmed using Apple’s specialty cameras.4 Even just having multiple games on at once, though, is a lot of fun, and movies are extremely immersive. My wife’s response summed up the experience perfectly: “Why would I ever go to a movie theater again?”

What is remarkable is that this stellar experience can be had anywhere: I might have to pick and choose between office and portable productivity, but there is no compromise when it comes to personal entertainment. The best device is the same device no matter where you are. This is what I mean when I say the Vision Pro is to the iPad as the iPod was to the Rio: the increase in fidelity and the overall experience is so vast as to constitute a step-change in functionality.

This is also where being private isn’t such a bad thing; there is a reason why Apple already has an ad about using the Vision Pro on an airplane:

This is a killer articulation of the fact that isolation is sometimes exactly what you need, even if it suffers from the tech industry affliction of assuming everyone flies regularly. Om Malik, in a recent Stratechery Interview, gave another compelling use case (and started me down the path of thinking about the iPod analogy):

But the thing is you actually have to be mobile-native to actually appreciate something like this. So if you’ve grown up watching a 75-inch screen television, you probably would not really appreciate it as much. But if you are like me who’s been watching iPad for ten-plus years as my main video consumption device, this is the obvious next step. If you live in Asia, like you live in Taiwan, people don’t have big homes, they don’t have 85-inch screen televisions. Plus, you have six, seven, eight people living in the same house, they don’t get screen time to watch things so they watch everything on their phone. I think you see that behavior and you see this is going to be the iPod. The headphones, why is headphones selling all the time everywhere? It is because people want their moment of privacy and they want to be alone and they want to listen to their media in their way. I think that’s what Vision Pro excites me is it’s going to be a video consumption device.

This does resonate with me, both in theory and in practice: even while trying to figure out the productivity use case I relished the fact that those sitting near me couldn’t see me futzing about. To put it another way, isolation is just another word for privacy, and privacy is very nice to have.

There is, though, one problem with both of these examples: folks living in close proximity to others, or even flying in the back of a plane, may not have the wherewithal to spend $3,500 on a personal TV. Those that do are probably more likely to have a traditional home theater or fly in seats with a bit more privacy. That, though, is only a problem for now (and, I might note, a real opportunity for Meta’s significantly lower-priced Quest).

For me this will be my primary Vision Pro use case, if one exists. When I watch TV it is usually after my kids are asleep, and simply slipping the Vision Pro on on the couch and relaxing is very pleasant and more convenient than getting my AirPods and pairing them to the Apple TV connected to my very-much-not-a-home-theater TV. The couch use case also diminishes another big Vision Pro negative: the battery and its associated cord is very annoying and will, I suspect, lead to broken Vision Pros yanked down by a battery in someone’s pocket. This sort of moving around is also, I would note, much more common when you’re trying to be productive; the Vision Pro really isn’t built for that, quite literally. Maybe someday, but V1s have to make trade-offs, and Apple has, in every respect, optimized for the entertainment experience.

The AR Vision

My favorite Vision Pro review isn’t really a review at all: it’s Casey Neistat in, his ever entertaining way, reaching out to pull the future back to the present:

I’ve focused till now on productivity (the use case I’m most excited about, which the Vision Pro does not deliver on) and entertainment (which the Vision Pro is clearly focused on, and excels at). However, Nilay Patel in his review at The Verge reminded us that Apple CEO Tim Cook has never been a fan of virtual reality:

See this thing — a passthrough VR headset with a silly external battery pack and a display that shows ghostly images of your eyes on the front — is not the big goal. The big goal is AR, or augmented reality. In particular, the big goal is optical AR, where light passes directly through unobtrusive glasses to your eyes, with digital information layered over the top of what you’re seeing. AR is a technology with the potential to literally change humanity, and Apple CEO Tim Cook has been talking about how isolating VR headsets are and how important he thinks AR will be for years now.

  • Tim Cook, 2016: “Few people are going to view that it’s acceptable to be enclosed in something.”
  • Tim Cook, 2017: “Unlike Virtual Reality which closes the world out, AR allows individuals to be present in the world.”
  • Tim Cook, 2017: “I also like the fact that [AR] doesn’t isolate […] I’ve never been a fan of VR like that because I think it does the opposite.”
  • Tim Cook, 2020: “I think [AR is] something that doesn’t isolate people. We can use it to enhance our discussion, not substitute it for human connection, which I’ve always deeply worried about in some of the other technologies.”

You get the idea.

The problem is that the technology to build a true optical AR display that works well enough to replace an everyday computer just isn’t there yet. The Magic Leap 2 is an optical AR headset that’s cheaper and smaller than the Vision Pro, but it’s plagued by compromises in field of view and image quality that most people would never accept.

So Apple’s settled for building a headset with real-time video passthrough — it is the defining tradeoff of the Vision Pro. It is a VR headset masquerading as an AR headset. And let me tell you: the video passthrough on the Vision Pro is really good. It works! It’s convincing. You put the headset on, the display comes on, and you’re right back where you were, only with a bunch of visionOS windows floating around.

As Patel notes, this is a tremendous engineering achievement; the problem, however, is that you are still watching video, not seeing the real world. Still, as Neistat and countless other show-offs on social media have demonstrated, you can very much function in the real world with the Vision Pro on, and I don’t think that’s an accident. VR is a destination device, like a TV or video game console or PC, while AR is an accompaniment device, like a phone. The latter is a larger market, simply because the number of opportunities to augment a user’s life are greater than the amount of time available in a zero sum battle for total attention, and it makes sense that to the extent Apple can build experiences beyond entertainment they are focused on building AR, even if for now it is simulated.

What is clear, though, is that Apple will need help: the biggest hole in not just the Vision Pro but also the Quest is software. I’ve already written about The Vision Pro’s Missing Apps, but services like Netflix and YouTube are not what will create an AR future: for that developers need to take risks, which means they need to have the possibility of making money, and deep access to the hardware Apple (and Meta) has created. I do worry that our AR future is going to be so technically challenging that only the biggest companies can create the hardware necessary, even as they hold onto business models and developer limitations that prevent the emergence of high-risk high-investment yet platform-defining software applications.

Guest Mode

The other interesting aspect of those quotes Patel collected is that Cook’s emphasis on preserving the ability to interact with others really didn’t come across in last year’s Vision Pro announcement. I wrote in my Article:

What was far more striking, though, was how the consumption of this video [of a father’s children] was presented in the keynote:

Note the empty house: what happened to the kids? Indeed, Apple actually went back to this clip while summarizing the keynote, and the line “for reliving memories” struck me as incredibly sad:

I’ll be honest: what this looked like to me was a divorced dad, alone at home with his Vision Pro, perhaps because his wife was irritated at the extent to which he got lost in his own virtual experience. That certainly puts a different spin on Apple’s proud declaration that the Vision Pro is “The Most Advanced Personal Electronics Device Ever”.

The most personal electronics device ever

Indeed, this, even more than the iPhone, is the true personal computer. Yes, there are affordances like mixed reality and EyeSight to interact with those around you, but at the end of the day the Vision Pro is a solitary experience.

It turns out that undersells it: the Vision Pro isn’t just experienced in isolation, it can only really be used by one person. Vision Pro, like all iOS-based devices, doesn’t support multiple user accounts (managed iPads, like in a school, are the exception). Apple, though, obviously wants people to be able to try the Vision Pro, so there is the option for a Guest User session. This is invoked through control center by the Vision Pro owner, who decides whether or not to allow access to their apps and data, and then has five minutes to remove their lenses (if necessary) and hand the Vision Pro to someone else.

This is already not great — you can’t, say, leave your Vision Pro at home or in a hotel room for a family member to use, because you’re not there to invoke a Guest User session — but it gets worse; from Apple’s support document:

When your guest puts on Apple Vision Pro, they might first be asked to press and hold the Digital Crown until the displays align and a green check mark appears. Then your guest will be asked to go through hand and eye setup so that Apple Vision Pro responds accurately to their input. When hand and eye setup are complete, your guest can begin using your Apple Vision Pro.

When your guest is finished using Apple Vision Pro, they can simply take off the device to end the Guest User session. The next time you put on Apple Vision Pro, it returns automatically to your personal hand and eye settings.

This is very harsh in practice. For example, one friend lifted up the Vision Pro to rub their eyes; when they put the Vision Pro back on the Guest User session was wiped. I had to re-invoke a Guest User session (after re-inserting my lenses5), and then they had to run through the hand and eye calibration all over again. And then I had to do it again and again for the rest of my family.

What this means is that I felt like an absolute jerk. I spent $3500 on a device that only I can use, and it felt bad. I was selfish, even though I didn’t mean to be. It honestly put me in a bad mood, and made me regret my purchase (beyond, you know, the whole “this is my job” thing).

Again, I get that there are probably technical limitations to enabling persistent Guest User sessions. At the same time, it’s hard to not notice the incentives at play here: Apple makes money by selling devices, and it is very much in their interest that I solve this problem by buying more Vision Pros. That, though, may have been a reasonable expectation when it comes to a phone, even if it’s a bit shakier when it comes to an iPad. A $3,500 Vision Pro, though, goes too far in my opinion: Apple sells Macs for that much money, but they all support multiple users; there should be, at a minimum, the ability to enable a guest session for some set period of time, not just until they even temporarily remove the Vision Pro from their face.

In short, the Vision Pro might not have yet fully realized Cook’s goal of letting users be present in the real world; it’s unfortunate that it is currently so hostile at the prospect of having a shared experience with what is a remarkable device.

Visions of the Future

I’m generally annoyed by buy/don’t-buy recommendations from review sites: just tell me what the device is like, and I can make my own decision. That noted, one reason I did want to write this review is because I think the demo I had at WWDC — which, as I understand, is pretty close to the demo available at Apple Stores — was not ultimately representative of the use case I cared the most about for the reasons I laid out above.

Now, having used a Vision Pro of my own, I have to say that were I making a decision independent of my job, I would not buy a Vision Pro. I personally don’t watch that much TV or movies, and while I am a huge sports fan, there is not yet the sort of immersive content available that would make it worth it to me (but I’m hopeful!). Meanwhile, the productivity use cases simply didn’t materialize for me, although I am hopeful for the ability to project two monitors in a software update.

At the same time — and, to be sure, this applies to my job — I am happy to have one for what it says about the future. It is not just that the entertainment experience is extraordinary, but the fact that it is portable that is new (and not to beat a dead horse, is the exact sort of outcome I want for productivity). The AR possibilities hinted at by passthrough, meanwhile, are very compelling (I thought that Joanna Stern’s review captured this well, particularly the cooking app).

I also, for what it’s worth, think that the Vision Pro is not the death knell for Meta’s VR efforts that so many think it is: the two visceral reactions I had to the Vision Pro were the “sitting down on the couch after a day at work and slipping it on” experience and the “wow it’s nice that my futzing around is private” experience; Meta, having made tradeoffs that favor a drastically lower price, is well-positioned to capture the latter, particularly for the use cases that Malik described. Make no mistake, video on the Vision Pro is better — resolution matters! — but it’s more than passable on the Quest, and better than a tablet or a phone. Controllers, meanwhile, make for a far better gaming experience, even if gaming as a whole is more of a destination activity than an augmented one.

What is most compelling about Meta, though, are their investments in AI. I believe that generative AI will be the key to unlocking the Metaverse, and I suspect that Meta will achieve the capability necessary to deliver an infinite number of unique experiences before Apple will (if they even try). Meta, too, is focused on virtual connections as a matter of business (and I find their early efforts compelling); I would expect the company to deliver a compelling “friend” experience in VR long before Apple (and I do, for the record, think their cartoonish approach to avatars is much better than Apple’s uncanny-valley-esque Personas).

In fact, I suspect there is room for both, and their respective market opportunities may follow the distinction I noted above between AR and VR: Apple has its eyes set on the real world, and Meta the virtual one; I just wish one of them would help me get my work done.

I wrote a follow-up to this Article in this Daily Update.

  1. You can disable it completely, which I have done for many people in my life ↩

  2. Including my built-in display, which in this picture is 14″; the photo is old, but representative enough. ↩

  3. My single favorite feature of Vision Pro is Optic ID: it works perfectly every time, without thought; you don’t need to perfectly place your (dry) finger like Touch ID, or hold your phone just so like Face ID. It’s completely seamless and very satisfying. ↩

  4. I think this quote from NBA Commissioner Adam Silver in Sportico does miss the point, however:

    “This is in many ways better than sitting courtside,” Silver said Friday. “It can take you anywhere on the floor. It can give you the perspective of a player … This will, to me, be how people over time experience sports.”

    Actually, no, I do want to sit courtside. It’s one of the most incredible experiences you can have as a fan. Moreover, you don’t even need any production: if I want the score, I can look up at the scoreboard; the announcer is the in-arena PA system. Don’t overthink this! ↩

  5. At this stage in my life, I much prefer wearing glasses (I can’t have LASIK surgery), so I need the Zeiss lens inserts; they are easy-to-use, but they do make the Vision Pro worse in my experience. First, eye-tracking worked much better without them, and second, bright scenes will induce reflections on the lenses that are very distracting. Moreover, if you stop using the lenses and switch to contacts, you have to re-do hand and eye calibration and re-setup Optic ID ↩

Intel’s Humbling

Posted on

This Article is available as a video essay on YouTube

There are times when being a semiconductor CEO is rather easy. Just consider Brian Krzanich: when he took over the Intel job in 2013, I wrote in The Intel Opportunity:

A new CEO has taken over Intel. Their core business, upon which the company has been built, is floundering. Does the new CEO, who is not really new at all (he’s the current COO), have the vision to ensure Intel’s continued success? I’m not talking about Brian Krzanich, who today was promoted from COO to CEO at Intel. Rather, I’m talking about Andy Grove, who took over Intel in 1987.

The crisis Grove encountered was Intel’s floundering memory business; he exited memory and focused on logic chips, and the rest was history. I thought that Krzanich should do something similar: Intel should stop focusing its efforts on being an integrated device manufacturer (IDM) — a company that both designed and manufactured its own chips exclusively — and shift to becoming a foundry that also served external customers. Back to the Article:

Today Intel has once again promoted a COO to CEO. And today, once again, Intel is increasingly under duress. And, once again, the only way out may require a remaking of their identity. It is into a climate of doom and gloom that Krzanich is taking over as CEO. And, in what will be a highly emotional yet increasingly obvious decision, he ought to commit Intel to the chip manufacturing business, i.e. manufacturing chips according to other companies’ designs.

Krzanich did not take my advice, and this is what happened to Intel’s stock during his tenure:

Intel's stock under Brian Krzanich

The thing is, if you don’t invest in the future, or see fundamental changes in the market coming, then you don’t have to spend as much; if you don’t have to spend as much then you can increase margins. And, while there were some analysts that could foresee the extent to which fabs were dramatically increasing in price, and would thus need to significantly increase volume to maintain profitability in the long run, this was clearly a case where Wall Street mostly cared about the forecast for the next quarter or the next year.

All of this was compounded by the assumption that Intel would stay in the process lead forever; indeed, while I was right about the changing costs of foundries and the need for Intel to open up to outside customers, in 2013 I didn’t forecast Intel losing their process leadership. That happened under Krzanich too: TSMC started manufacturing 7nm in volume in early 2017, and Intel announced a delay in 10nm (which was roughly equivalent to TSMC’s 7nm) in April 2018. And yet the stock went up, faster than ever.

Intel’s Struggles

Later that year Krzanich was fired for having a relationship with an Intel employee; Bob Swan stepped into the CEO role, but it was ultimately current CEO Pat Gelsinger that is paying the price for Krzanich’s lack of strategic foresight and total fumble in terms of execution. Consider the company’s earnings announcement last week; from Bloomberg:

Intel Corp. tumbled the most in more than three years after delivering a disappointing forecast, a reaction that Chief Executive Officer Pat Gelsinger said Friday was overblown. The shares fell 12% to $43.65 in New York after Intel’s first-quarter projection for both sales and profit came in well short of Wall Street estimates. It was the biggest single-day decline since July 2020.

The outlook sparked fears that Gelsinger’s long-promised comeback bid has gotten off track. Though the chipmaker’s personal computer business is recovering, demand is weakening in the lucrative market for data center processors. Intel also is contending with a slowdown in programmable chips and components for self-driving vehicles, and a fledgling business that makes semiconductors for other companies hasn’t yet taken off…

During a conference call with analysts, Gelsinger acknowledged that the first quarter wasn’t going as well as hoped, but that he expected the rest of 2024 to improve quarter by quarter. Intel’s efforts to return to the cutting edge of manufacturing are still on track, he said. That’s crucial to improving its products and staying competitive. He also asserted that the chipmaker is no longer losing sales to competitors in PCs and data centers.

This stock price decline was just one of many under Gelsinger’s leadership:

Intel's stock under Pat Gelsinger

This past quarter continues many of the trends driving this multi-year decline: client PCs are finally making a comeback from the COVID hangover, but many of Intel’s non-CPU businesses are struggling and/or facing inventory corrections, including MobileEye, networking, and FPGAs. The biggest problem, though, continues to be the data center: AMD makes better CPUs on a better process (TSMC’s) and they continue to eat Intel’s lunch amongst the biggest cloud providers who, because they buy the most CPUs, are willing to do the work necessary to make the best performing chip work (this is why Intel’s on-premise and government business has long held up better). That direct competition is compounded by the secular pressure from ARM on one side and the diversion of spend to GPUs (primarily Nvidia, but also AMD) on the other.

Intel’s Progress

This is where being a semiconductor CEO is very difficult. Over the last few years Gelsinger has done exactly what needed to be done a decade earlier: he is transforming Intel into a foundry that serves external customers, and he is working to at least make Intel competitive again on the leading edge, and maybe even take the lead in a few years, if you believe Intel’s claims about its 18A process.

18A is the fifth of the fabled “five nodes in four years” that Gelsinger promised shortly after he took over, and it appears that he is pulling it off. Gelsinger summarized those five nodes in a Stratechery Interview last fall:

So Intel 7 is the last of the pre-EUV technologies. Intel 4, the first EUV technology for us, Intel 3 refined the final FinFET, really helped us take those learnings, but largely was a common architecture of transistor and process flow — really just the refinement. Much like you say, TSMC and others have done, get the initial one working and then refine it for scale manufacturing, that’s Intel 3. And given it’s the second generation of that, we’ll be applying that to our big server products, Granite Rapids, Sierra Forest, big die. We need to get down the learning curve with Meteor Lake, our first client partner. And then now with the big server die, and that’s also what we’re introducing on Intel 4, more so on Intel 3, a lot of the advanced packaging technologies come big into the technology footprint. Then the new transistor, the new backside power begins with 20A, and for that Arrow Lake is sort of the first, get it up and running small die, something easier to design and then when we get to 18A, the journey is done.

To summarize:

Transistor Lithography Power TSMC Equivalent
Intel 7 FinFET DUV Frontside N7 (~7nm)
Intel 4 FinFET EUV Frontside N5 (~5nm)
Intel 3 FinFET EUV Frontside N4 (~4nm)
Intel 20A RibbonFET EUV Frontside N3 (~3nm)
Intel 18A RibbonFET EUV Backside N2 (~2nm)

The TSMC equivalents are a bit fuzzy, particularly once you get into the future; TSMC CEO C.C. Wei has been adamant on the last couple of TSMC earnings calls that TSMC’s advanced 3nm process will outperform Intel’s 18A (Intel’s argument is that backside power will make chips much easier to design, since the power is separated from the communications layer, eliminating interference). What’s important, though, is that that is a question worth answering. Gelsinger said of 18A, which is the process that Intel is pushing hard to potential foundry customers, on the company’s earnings call:

We are first in the industry to have incorporated both gate-all-around and backside power delivery in a single process node, the latter unexpected two years ahead of our competition. Arrow Lake, our lead Intel 20A vehicle will launch this year. Intel 18A is expected to achieve manufacturing readiness in second half ’24, completing our five nodes in four year journey and bringing us back to process leadership. I am pleased to say that Clearwater Forest, our first Intel 18A part for servers has already gone into fab and Panther Lake for clients will be heading into Fab shortly…

Our success with IFS will be measured by customer commitments and revenue. We have taped out more than 75 ecosystem and customer test chips. IFS already has more than 50 test chips in the pipeline across 2024 and 2025, 75% of which are on Intel 18A. During CES, we welcomed the Valens Semiconductor to the growing list of foundry customers as they announced they would use IFS to fabricate their MIPI A-PHY chipsets using our advanced technology. In addition to the 3 Intel 18A customers we disclosed in Q3, we won a key design win with a significant high-performance computing customer. This customer was particularly motivated by our unique leading-edge manufacturing capabilities and U.S. capacity. We came into 2023 committing to one 18A foundry customer. We executed on four inclusive of a meaningful prepay and our momentum continues to grow.

The ultimate proof point for Gelsinger’s strategy will be chips designed by external customers, fabbed on Intel’s 18A process, running in devices in people’s pockets; nothing is assured until then. That, unfortunately, is the rub: there is no revenue until then either, and “then” is still a few years into the future. One wonders if Gelsinger will be there to enjoy the uplift that would only then be justified, at least from the perspective of Wall Street.

From my perspective — which, as exemplified by my disappointment with Krzanich despite Intel’s great stock returns during his tenure, is absolutely not stock-picking advice — he very much deserves the chance. Intel has the right strategy and seems to be executing; the challenge is that semiconductor cycles operate in something closer to decades than years, much less quarters.

Intel’s New Partner

So what does Intel do in the meantime? Last week also brought news of a very interesting new partnership that helps answer that question. From Nikkei:

Intel and Taiwan’s United Microelectronics Corp. (UMC) on Thursday announced a partnership that will lead to production in the U.S. state of Arizona by 2027, part of the American semiconductor company’s push to expand its business of making chips for others. The partners will develop relatively mature 12-nanometer technology, ideal in building chips for Bluetooth, Wi-Fi, microcontrollers, sensors and a range of other connectivity applications, but not for cutting-edge central processing units or graphics processors. Intel said the long-term agreement can leverage its U.S. manufacturing capacity and UMC’s extensive foundry experience in mature chip production technologies that serve a wide range of chip developers.

Based in the Taiwanese city of Hsinchu, UMC is a smaller peer of Taiwan Semiconductor Manufacturing Co., the world’s biggest contract chipmaker or foundry. UMC is the world’s third-largest contract chipmaker.

Last week I wrote about TSMC’s earnings and explained how TSMC has been forced to increasingly adopt the old Intel model, first in pricing, and then in its equipment usage:

The leading edge costs a lot of money to ramp up — N3 is lowering margins for now, as every new node does its first few years — but those costs are made up for by the ability to charge much higher prices. To that end N3 is already up to 15% of TSMC revenue, followed by 35% at N5, and 17% at N7.

This reality is not new for TSMC, but it is different than how the company has operated historically. TSMC started out as a foundry selling trailing edge chips; the primary way of making money over the long run was to build a fab relatively cheaply using established equipment, and then run that fab for many years. Once all of the equipment was depreciated, every chip produced was almost pure profit, even if the revenue on a per-chip basis was fairly low.

It was Intel, on the other hand, that charged the highest prices for the fastest chips, and all of its business was on the leading edge, selling its own chips; that meant that the company would take down old fabs and repurpose as much equipment as it could for the next node, instead of running the fab forever like a foundry would (this is one of Intel’s challenges in becoming a foundry: they simply don’t have much depreciated trailing edge capacity throwing off cash).

What is interesting to note is that TSMC’s shift to a more Intel-like model in terms of its revenue drivers (leading edge) and profit drivers (high prices) is starting to impact how they manage their fabs. CFO Wendell Huang said in his prepared remarks:

In addition, we have a strategy so that some of our N3 capacity can be supported by N5 tools given the strong multiyear demand. Such a plan will enable higher capital efficiency in the mid to long term, but requires cost and effort in the near term. Most of this conversion will occur in second half of 2024, and we expect it to dilute our gross margin by about 1 to 2 percentage points in second half of 2024.

Notice what is happening here: TSMC, unlike its historical pattern, is not keeping (all of its) 5nm capacity to make low-cost high-margin chips in fully-depreciated fabs; rather, it is going to repurpose some amount of equipment — probably as much as it can manage — to 3nm, which will allow it to expand its capacity without a commensurate increase in capital costs. This will both increase the profitability of 3nm and also recognizes the reality that is afflicting TSMC’s 7nm node: there is an increasingly large gap between the leading edge and “good enough” nodes for the vast majority of use cases.

This Intel-UMC deal represents the inverse of what is happening at TSMC: a viable foundry business can’t just rely on selling leading-edge chips at very high margins — particularly since Intel’s claims that it will regain process leadership remain to be seen. What is critical is having fully depreciated foundries still making chips: yes, those chips cost a lot less than the leading edge, but given that the marginal costs are practically zero (at least relative to the fixed costs) they are an important source of cash flow and profits, which can be re-invested in the leading edge. This deal is about capturing that depreciated cash flow.

Intel’s Needs

Intel, however, as I noted, only ever needed leading edge fabs — no one wants an old Intel chip when newer and faster ones are on the market. This was a big reason, of course, why Krzanich so badly missed The Intel Opportunity: in 2012 Intel was right in the middle of the FinFET-deep ultraviolet (DUV) lithography era of chip fabrication, but the end of both was already on the horizon in the form of RibbonFET (i.e. Gate-All-Around transistors) and extreme ultraviolet (EUV) lithography. Given the astronomical costs of EUV in particular it would have been reasonable to forecast then that there might develop a sweet spot making FinFET transistors with DUV, but Intel missed out on a decade of building up the capability to serve external customers.

This capability — or the lack thereof — remains one of the biggest questions around Intel’s foundry efforts. In 2022, when Intel tried to buy Tower Semiconductor, I wrote in an Update:

It is not only the case that Intel primarily makes its own designs, it also makes only digital chips (i.e. everything is a 1 or a 0). However, there is also an entire universe of analog chips, which can process gradations; this is essential for processing data from the physical world like sound, power, light, etc. Tower specializes in a whole host of specialized chips in the analog space; adding Tower’s capabilities to Intel Foundry Services (IFS) will make the latter much more of a true one stop shop for chip fabrication, matching the capabilities of TSMC or GlobalFoundries.

To me it is the GlobalFoundries angle that is the most interesting here: I have long been a proponent of Intel buying GlobalFoundries, despite the fact that GlobalFoundries isn’t a particularly great business, has given up on the leading edge process race, etc. My thinking has been that Intel can bring the capability (maybe) and willingness to invest in the leading edge, while GlobalFoundries can bring the breadth of capabilities and customer service orientation necessary to be a foundry. Sure, that’s expensive, but allowing IFS to be choked off by Intel’s integrated and not-invented-here culture would be even more expensive.

I suspect the Tower acquisition firmly closes the door on that possibility (which to be fair, was clearly remote). Here the calculus is much more straightforward: Tower brings certain capabilities and customer relationships that Intel believes it can scale up inside its factory network at a much lower cost than GlobalFoundries (whose current market cap is $29 billion), and it will be much easier to absorb and integrate into Intel’s business. The big question is whether or not integrating into Intel’s business is in fact the entire problem that needs to be avoided.

At Intel, manufacturing has always called the shots. The design side of the company had to accommodate the fabs, whether that be using their archaic design software, working around manufacturing challenges, or figuring out how to make a faster chip on recycled equipment. This made sense for a long time, but there was a cost: Intel designs stopped being innovative and became dependent on Intel’s manufacturing for performance; when Intel’s manufacturing prowess hit a wall Intel’s designs were exposed. Gelsinger told me:

So all of a sudden, as Warren Buffet says, “You don’t know who’s swimming naked until the tide goes out.” When the tide went out with the process technology, and hey, we were swimming naked, our designs were not competitive. So all of a sudden we realized, “Huh, the rising tide ain’t saving us. We don’t have leadership architecture anymore.” And you saw the exposure.

Indeed, we see the stock price!

That, though, was only part of Intel’s problem: the more fundamental issue is that a foundry is, as I wrote, a customer service organization: an entity like TSMC adapts to customers’ designs, not the other way around. They use industry standard design software. They have extensive libraries of IP that make designing a chip more akin to assembling a collection of Lego blocks. They ship when they say they will ship, and they run the fab for which a chip was designed forever.

Intel did none of these things, and had a mentality and culture that ran in the exact opposite direction: in a foundry, manufacturing is not king but a servant; customer sales is not about “take-it-or-leave-it” but “let us help you solve your problem.” I was — and frankly, remain — dubious about Intel’s ability to create that sort of culture internally, which is why I advocated for an acquisition, first of Global Foundries, and then of Tower. Thanks to its decade delay Intel didn’t have time to learn how to serve customers: it had rapidly obsoleting fabs that needed to be filled as soon as possible, if the company ever had hope of making enough cash to fund its push back to the leading edge.

Unfortunately China blocked the acquisition of Tower, in what I suspect was retaliation for U.S. restrictions on China. Worse, from what I have heard Intel responded by starting to sell a lot of old equipment at rock-bottom prices, which usually ended up in China; the fact of the matter is that the company needs cash.

Intel’s Humbling

Perhaps, though, the fire-sale is coming to an end: all of this context explains why this deal exists, and why I think it is a fantastic idea:

  • UMC, like GlobalFoundries before it, has struggled to keep pace with ever more expensive fabs. The company has a 14nm offering, but has shown little evidence it can or will go further, and the EUV transition seems completely out of the question. However, UMC does have a large foundry business, which is to say that UMC is a customer service organization, with the compatibility and IP necessary to succeed.
  • Intel, meanwhile, has a ton of capacity with FinFET and DUV processes. One of the costs of the company’s failure at 10nm and 7nm was that the company built extra 14nm fabs. A huge amount of that equipment, particularly the lithography, is not useful for the leading edge, but it is fully depreciated and could be used to build pretty fast chips for a lot less than the leading edge.

This deal — which is for a new, designed-for-external-customers 12nm process — brings together the two companies’ core capabilities: UMC is the customer service organization, and Intel is the manufacturer. Yes, that means lower revenue and margins for both, but both have already built the capabilities necessary to make the deal succeed, which means the business should be accretive to both revenue and profits for each of them.

The big question is how big of a market there is for fast-but-not-the-fastest chips: Intel is talking up things like communications chips, image sensing processors, etc., but a new process will require new design wins. Moreover, TSMC is counting on the same market for its 7nm process: that process should be faster, but it is also more difficult to make (7nm requires quad-patterning, while 12nm is dual patterning; this means easier designs, higher throughput, and better yields for the latter).

It is also, one might say, a bit humiliating: mighty Intel, which bestrode the tech industry for 50 years, the keepers of Moore’s Law, is making a deal with a Taiwanese also-ran, because it needs the help. That, though, is no insult: Intel needed some humbling, and this deal, more than any 18A design win or lofty promise about the AI PC, gives me hope that the company is in fact turning things around.