Good morning,

On Tuesday’s episode of Sharp Tech we discussed how Meta might monetize WhatsApp, more on the Apple car and Apple’s AI plans, and an accidental deep dive on podcasts. Meanwhile, on Dithering John and I discussed the rumor that Apple might include Gemini in iOS.

On to the update:

Microsoft and Inflection AI

From Bloomberg:

Microsoft Corp. has named Mustafa Suleyman head of its consumer artificial intelligence business, hiring most of the staff from his Inflection AI startup as the software giant seeks to fend off Alphabet Inc.’s Google in the fiercely contested market for AI products. Suleyman, who co-founded Google’s DeepMind, will report to Chief Executive Officer Satya Nadella and oversee a range of projects, such as integrating an AI Copilot into Windows and adding conversational elements to the Bing search engine. His hiring will put Microsoft’s consumer AI work under one leader for the first time.

Inflection, a rival of Microsoft’s key AI partner OpenAI, is shifting to selling AI software to businesses but will continue operating its Pi consumer chatbot business for now. Karén Simonyan, Inflection’s co-founder, will join Microsoft as chief scientist for the new consumer AI group.

This is a bizarre move that only makes sense once you collect a number of different details in one place.

First, from Inflection’s website:

Our plan going forward is to lean into our AI studio business, where custom generative AI models are crafted, tested and fine tuned for commercial customers. Our success at training, tailoring and improving the performance of large AI models makes us uniquely well placed to be the AI platform for businesses around the world.

As part of this, we’re thrilled to announce that we will now host Inflection-2.5 on Microsoft Azure helping us get it into the hands of creators everywhere. We’ll also be ensuring it comes to other cloud hosting platforms in the near future. The API itself isn’t available today, but will be up and running very soon. To sign up for early access and help us test it, please register your interest here. Between API access and select high-level partnerships with great customers, our AIs can now spread to even larger new user bases while helping put cutting-edge AI capabilities in the hands of thousands of developers.

Second, from Forbes:

Microsoft is not taking any equity position in Inflection AI as part of the shift, a second source added, nor will it be acquiring any of Inflection’s intellectual property. Inflection, Microsoft and Suleyman declined to comment beyond their respective posts. In a statement posted to LinkedIn, that professional network’s cofounder Reid Hoffman — also Inflection’s third cofounder and a board director — wrote: “This agreement with Microsoft means that all of Inflection’s investors will have a good outcome today, and I anticipate good future upside.”

Of note: Microsoft was already an investor in Inflection AI before Tuesday. But that “good outcome” didn’t include investors receiving any cash from the company in the form of a partial exit, a source with knowledge said. Hoffman may have been instead alluding to a financial arrangement, previously undisclosed by Microsoft and Inflection and first reported by Forbes, as part of its license of Inflection’s API, multiple sources said. By that arrangement, Inflection investors could be made whole on their committed capital, though not immediately, they added.

Inflection will focus now on providing an application programming interface, or API, to other businesses — a major shift for a company that had focused until recently on Pi, its chatbot alternative to OpenAI’s ChatGPT and Anthropic’s Claude. It will host that API on Microsoft’s cloud service as well as others, the source added. In its own post, Inflection claimed that PI had “millions” of weekly users. Those people would continue to have access, at least for now, as the company claimed “there will be no immediate changes to the service.”

Finally, from The Information:

Microsoft’s decision to hire most of the staff of Inflection AI, including co-founders Mustafa Suleyman and Karén Simonyan, leaves the would-be OpenAI rival a shell of its former self. But Inflection has arranged an unusual deal for investors that should take the sting out of that blow.

At the same time it revealed the staff departures, Inflection disclosed a licensing deal with Microsoft to make its models available for sale on Microsoft’s Azure, and said other cloud platforms would follow. It has also told investors they will fully recoup their investment, and more, as a result of the licensing agreement, according to a person briefed on the arrangement. Inflection has raised over $1.5 billion from investors including Microsoft and Greylock.

My apologies for all of the excerpts, but like I said, I think it’s the only way to piece together what is going on here. Start from the top, with Microsoft. I wrote last month after the company made an investment into Mistral:

The fact the investment exists is notable: this is a very logical response by Microsoft to last fall’s upheaval at OpenAI. Microsoft is making massive investments in both Azure and in its products around AI that it doesn’t control, and the best alternative to not owning the AI undergirding both is to hedge its bets by increasing the number of suppliers. The important thing — which was revealed during that episode — is that Microsoft has leverage because it owns GPUs…

Microsoft’s products are still being built around OpenAI, to be clear; the short-term outcome of this agreement is that Mistral will be available to developers as a part of Azure’s Models-as-a-Service. Building that out, though, also moves Microsoft down the road to switching over, should the need some day arise.

There’s one solution that is even better than having a small investment in a model maker: employing your own entire model-making team, along with a license to a GPT-4 class model that they already trained. This, I think, is the most important takeaway: Microsoft is very serious about hedging their OpenAI bet.

Inflection Oddities

This brings me to Inflection AI, and what was, in retrospect, a very odd fundraise last summer. From Forbes in late June 2023:

Less than two months after the launch of their first chatbot Pi, artificial intelligence startup Inflection AI and CEO Mustafa Suleyman have raised $1.3 billion in new funding. Microsoft, Nvidia and three of tech’s most influential billionaires led the investment in the Palo Alto-based startup launched in early 2022. LinkedIn cofounder Reid Hoffman, Microsoft cofounder Bill Gates and former Google CEO Eric Schmidt all personally invested, with Nvidia the sole new investor among the group. The new funding values Inflection at $4 billion, according to a source with knowledge of the transaction…

Some details of Inflection’s new deal with Microsoft and Nvidia are, like Suleyman’s iceberg, still largely out of view. He declined to provide a breakdown of how much of the $1.3 billion raised included cash equivalents (such as computing credits) but said that a “very, very large chunk” was in dollars. “We have all the cash we need to run and operate,” he added…

What is clear: the round significantly deepens Inflection’s ties with Microsoft and Nvidia, two key partners in the AI race. Microsoft, also a major investor in OpenAI, is Inflection’s cloud computing partner; Nvidia, meanwhile, has been working closely with Inflection on the deployment of its flagship H100 graphics processing unit (GPU), the current gold standard for AI training and powering large language models like OpenAI’s GPT-3. Nvidia worked closely with Inflection and service provider CoreWeave to co-develop Inflection’s current H100 cluster; Inflection paused its own work for Nvidia to run a recent test that Nvidia announced this week had set records on eight tests of current AI model training benchmarks, completing a benchmark based on GPT-3 in less than 11 minutes.

That test, which matched the computational power of training a model that took an estimated three to six months to develop, ran on Inflection’s 3,584 H100 GPUs already in service, Suleyman noted. But in the wake of this funding round and partnership, Inflection’s growing horsepower is about to get a turbo-charge. Nvidia and CoreWeave (which helps physically deploy the GPUs) are now in the process of helping Inflection install many thousands more. Once fully operational, Inflection’s new cluster will run 22,000 H100s.

Everything about this is weird. Start with the fact that Microsoft is Inflection’s cloud computing partner, and Inflection raised over a billion dollars, mostly in cash, to buy 22,000 H100s that will actually be managed by CoreWeave, who is not Microsoft. Part of this puzzle is explained by another piece of news from last June; from CNBC:

CNBC has learned from people with knowledge of the matter that Microsoft has agreed to spend potentially billions of dollars over multiple years on cloud computing infrastructure from startup CoreWeave, which announced Wednesday that it raised $200 million. That financing comes just over a month after the company attained a valuation of $2 billion.

CoreWeave sells simplified access to Nvidia’s graphics processing units, or GPUs, which are considered the best available on the market for running AI models. Microsoft signed the CoreWeave deal earlier this year in order to ensure that OpenAI, which operates the viral ChatGPT chatbot, will have adequate computing power going forward, said one of the people, who asked not to be named due to confidentiality. OpenAI relies on Microsoft’s Azure cloud infrastructure for its hefty compute needs…

CoreWeave’s announced funding on Wednesday from hedge fund Magnetar Capital was an extension of a $221 million round in April. Nvidia invested $100 million in the prior financing, Intrator said. CoreWeave was founded in 2017 and has 160 employees.

Nvidia is the key figure, here: one of the reasons why it was such a surprise that Nvidia did not raise prices on Blackwell as much as expected is that demand still outstrips supply for the H100. GPUs, though, are not sold at an auction: rather, Nvidia decides who gets what, and my suspicion is that this leverage was key to building out DGX Cloud. From an Update in November:

What AWS also remains strongly committed to is IaaS for AI; that’s actually where Selipsky started with his AI segment, and he wasted no time in bringing out the biggest gun of all: Nvidia CEO Jensen Huang. Of course that was hardly an exclusive appearance: Huang has now appeared at all three of Google, Microsoft, and Amazon’s cloud keynotes in a matter of months. And, once again, in addition to announcements about the cloud provider deploying Nvidia GPUs, there was also the news that said cloud provider would be a host for Nvidia’s DGX Cloud — a competing PaaS owned and operated by Nvidia, running on everyone else’s cloud.

In short, if cloud providers wanted Nvidia GPUs, my personal belief is they needed to agree to host DGX Cloud (this has not been reported, to be clear). That, though, isn’t the only game Nvidia is playing with GPU allocation: the company has made it a point to sell a decent amount to startups like CoreWeave in which, conveniently enough, it is an investor; CoreWeave then turns around and sells access to those GPUs to the same cloud providers who, despite hosting DGX Cloud, still can’t get enough GPUs. Oh, and Inflection: Nvidia was invested in a startup that somehow decided to buy 22,000 H100s.

This plan was definitely pretty nuts: 22,000 H100s were way more GPUs than Inflection needed to train its models, and unlike OpenAI, the company didn’t really have an API story; did the startup really think that Pi was going to become that popular? And, even if it did, those H100s are a depreciating asset that would need to be replaced at some point; it simply never made sense for a consumer-facing startup with uncertain longterm usage to be buying its own GPUs, at least in a normal market — this is the exact reason why cloud computing is so compelling! Again, though, this isn’t a normal market: despite the fact Nvidia just announced the B100, H100s are still very much in demand, and maybe Inflection thought they couldn’t get enough otherwise?

Regardless, one of the biggest open questions is the fate of those 22,000 GPUs. Here Inflection’s announcement that their latest model will be hosted by Azure going forward is notable; my guess is that that license fee Microsoft is paying isn’t just for the model, but also for the GPUs, which we already know that Microsoft doesn’t have enough of. Or, to put it another way, the reason why Hoffman can express confidence that Inflection’s investors will be made whole is because they are invested in an entity that has access to an asset that is actually more valuable if it isn’t being used by the company they nominally invested in.

The Acquisition That Isn’t

All of this conjecture assumes that Inflection was a failed product, which seems reasonable; here are SimilarWeb’s traffic estimates for February:

These relatively sluggish usage numbers would be much more bearable if Inflection was renting the capacity to serve them, given that cloud computing is usage-based. In the context of massive fixed investments in GPUs, though, that lack of usage is catastrophic. Inflection is a failed company.

Here again, though, we come to the issue of a normally functioning market, or lack thereof. The obvious outcome for a company like Inflection is to be acquired: the company has valuable employees, some useful IP, and, apparently, some very valuable GPUs. The problem, though, is that tech broadly is not a normally functioning market, thanks to the FTC: the assumption in big tech is that any acquisition will lead to an FTC lawsuit — Meta got sued for buying a VR exercise app! — and while it is likely that said lawsuit would be won, it would take years to resolve.

This hesitance was exactly what I warned about in 2020’s First, Do No Harm, which was written in response to the FTC’s plan to more closely examine small-scale acquisitions:

It’s important to note that the sort of acquisitions the FTC is looking at almost certainly fall predominantly in this third group. Acquihires of failed startups is arguably the most tangible way that big tech companies contribute to Silicon Valley’s durable startup culture: there is more reason for entrepreneurs, early employees, and investors to take a chance on new ideas because the big tech companies provide a backstop.

Another way to consider these benefits, meanwhile, is to think about a world where acquisitions by large tech companies are severely constricted or banned:

• New technology would be diffused far more slowly (as the new startup scales), if at all (if the startup goes out of business).
• The amount of investment in risky technologies without obvious avenues to go-to-market would decrease, simply because it would be far less likely that investors would earn a return even if the technology worked.
• The risk of working for a startup would increase significantly, both because the startup would be less likely to succeed and also because the failure scenario is unemployment.

There are structural reasons why Big Tech looks poised to dominate AI, including the importance of cloud infrastructure, data, and existing distribution channels. It’s notable, though, that a huge amount of the investment into AI startups, particularly the ambitious ones actually building models, is also coming from Big Tech: they can de-risk those investments by providing credits instead of dollars as Microsoft has, or by double-dipping on GPU sales like Nvidia is. Traditional investors, meanwhile, don’t have a clear exit if acquisitions are off the table.

Big Tech, though, is making it work anyways: Microsoft is definitely getting the Inflection team and model, and maybe also the GPUs. Meanwhile, the one traditional venture firm on Inflection’s cap table was Greylock, where Reid Hoffman, the only Inflection co-founder not going to work at Microsoft, is partner. Hoffman, of course, doesn’t need to go to Microsoft because he, conveniently enough, is already on the behemoth’s board, which means he was perfectly placed to engineer a scenario where Greylock and the other investors are poised to get their money back for a failed startup.

In other words, this is an acquisition in everything but name, just in the most convoluted way possible; the FTC can pat itself on the back, I guess, while Microsoft probably pays less than they might have otherwise, and every other would-be entrepreneur checks if their investors are sitting on any big company boards, and if they can make any irrational GPU purchases to make the relationship worth the trouble.

---

This Update will be available as a podcast later today. To receive it in your podcast player, visit Stratechery.

The Stratechery Update is intended for a single recipient, but occasional forwarding is totally fine! If you would like to order multiple subscriptions for your team with a group discount (minimum 5), please contact me directly.

Thanks for being a subscriber, and have a great day!