AWS, MongoDB, and the Economic Realities of Open Source

In 1999, music industry revenue in the United States peaked at $14.6 billion (all numbers are from the RIAA). It is important to be precise, though, about what was being sold:

  • $12.8 billion was from the sale of CDs
  • $1.1 billion was from the sale of cassettes
  • $378 million was from the sale of music videos on physical media
  • $222.4 million was from the sale of CD singles

In short, the music industry was primarily selling plastic discs in jewel cases; the music encoded on those discs was a means of differentiating those pieces of plastic from other ones, but music itself was not being sold.

This may sounds like a stupid distinction, but it explains what happened after that peak:

U.S. music industry sales over time

Music industry revenue plummeted, even as the distribution and availability of music skyrocketed: the issue is that people were no longer buying plastic discs, which is what the music industry was selling; they were simply downloading music directly.

Selling Convenience

The problem is that recorded music has always been worthless: once a recording is made, it can be copied endlessly, which means the supply is effectively infinite; it follows that to capture value from a recording depends on the imposition of scarcity. That is exactly what plastic discs were: a finite supply of a physical good differentiated by their being the most convenient way to get music. Pirating MP3s from sites like Napster or its descendants, though, was even more convenient — and cheaper.

As you can see from the chart, the industry started to stabilize in 2010, and in 2016 returned to growth; 2018 looks to be up around 10% from 2017’s $8.7 billion number, and it seems likely the industry will pass that 1999 peak in the not-too-distant future.

What happened is that the music industry — prodded in large part by Spotify, and then Apple — found something new to sell. No, they are still not selling music; in fact, they are beating piracy at its own game: the music industry is selling convenience. Get nearly any piece of recorded music ever made, for a mere $10/month.

DocumentDB (with MongoDB compatibility)

Last week, from the AWS blog:

Today we are launching Amazon DocumentDB (with MongoDB compatibility), a fast, scalable, and highly available document database that is designed to be compatible with your existing MongoDB applications and tools. Amazon DocumentDB uses a purpose-built SSD-based storage layer, with 6x replication across 3 separate Availability Zones. The storage layer is distributed, fault-tolerant, and self-healing, giving you the the performance, scalability, and availability needed to run production-scale MongoDB workloads.

The specifics of MongoDB and now DocumentDB are not particularly important to this article; basically, MongoDB created a type of database that is more flexible and better suited to large1 amounts of both structured and unstructured data, making it useful for large scale applications that traditional relational databases were never designed to accommodate.

And now you can run it on AWS. Kind of.

Open Source Licensing

Like an increasing number of such projects, MongoDB is open source…or it was anyways. MongoDB Inc., a venture-backed company that IPO’d in October, 2017, made its core database server product available under the GNU Affero General Public License (AGPL).2

AGPL is a close relative of the GPL, the copyleft license created by Richard Stallman. “Copyleft” means that the license allows for the free distribution, use, and modification of copyrighted material (in this case software), with the stipulation that those same rights extend to all derivative works; that means that any project built using GPL code must itself have a GPL license. This is in contrast to “permissive” open source licenses that allow others to use the copyrighted material however they wish, without a stipulation that derivative works also be open-sourced. AGPL extended the GPL to apply to software accessed over a network; since the software is only being used, not copied, the GPL would not triggered, but the end result is even more onerous than the GPL.

Both GPL and especially AGPL tend to be very problematic for companies: Apple, for example, does not allow software licensed with the GPL on the App Store, because the App Store requires that apps be licensed for a single user; apps with permissive licenses are fine — their license can be replaced — but the GPL, once applied, cannot be removed. AGPL is worse, because its provisions are triggered by users simply using the software; that’s why Google bans its use internally. The company notes in its open source documentation:

The license places restrictions on software used over a network which are extremely difficult for Google to comply with. Using AGPL software requires that anything it links to must also be licensed under the AGPL. Even if you think you aren’t linking to anything important, it still presents a huge risk to Google because of how integrated much of our code is. The risks heavily outweigh the benefits.

There is one addendum to the policy:

In some cases, we may have alternative licenses available for AGPL licensed code.

This is MongoDB’s business.3

MongoDB’s Business Model

MongoDB explained in their S-1:

We believe we have a highly differentiated business model that combines the developer mindshare and adoption benefits of open source with the economic benefits of a proprietary software subscription business model. To encourage developer usage, familiarity and adoption of our platform, we offer Community Server as an open source offering, analogous to a “freemium” offering. Community Server is a free-to-download version of our database that does not include all of the features of our commercial platform. This allows developers to evaluate our platform in a frictionless manner, which we believe has contributed to our platform’s popularity among developers and driven enterprise adoption of our subscription offering…

Unlike software companies built around third-party open source projects, we own the intellectual property of our offerings since we are the creators of the software, enabling our proprietary software subscription business model…Our primary subscription package is MongoDB Enterprise Advanced, our comprehensive offering for enterprise customers that can be run in the cloud, on-premise or in a hybrid environment. MongoDB Enterprise Advanced includes our proprietary database server, advanced security, enterprise management capabilities, our graphical user interface, analytics integrations, technical support and a commercial license to our platform. We also offer MongoDB Atlas, our cloud hosted database-as-a-service, or DBaaS, offering that includes comprehensive infrastructure and management of our Community Server offering.

Basically, MongoDB sells three things on top of its open source database server:

  • Additional tools for enterprise companies to implement MongoDB
  • A hosted service for smaller companies to use MongoDB
  • Legal certainty

The importance of this last one can not be overstated: MongoDB’s enterprise version and hosted service are not governed by the AGPL — or, as of late last year, a new MongoDB-created license called the Server Side Public License (SSPL). The SSPL is like the AGPL on steroids: it compels companies selling MongoDB-as-a-service to not only open-source their modifications, but also open-source their entire stack.4

What AWS Sells

The largest company selling software-as-a-service is, of course, Amazon. That, though, does not mean that Amazon is selling “software.” The reality is that software is no different than music: it is infinitely reproducible, and thus, in isolation, worth nothing.

Instead, the value of software is typically realized in three ways:

  • First is hardware. The most famous example is the iPhone, which is the only way to obtain iOS, but there are countless other examples.
  • Second is licenses. This was Microsoft’s core business for decades: licenses sold to OEMs (for the consumer market) or to companies directly (for the enterprise market). Indeed, there is a bit of irony in that both Microsoft and open source, for all their historical opposition to each other, both depended on copyright, strong legal regimes, and companies doing the right thing.
  • Third is software-as-a-service. This is Microsoft’s new model, as well as Amazon’s, and almost all new enterprise software companies.5 In this case what is being sold is not the software per se, but rather the utility of the software: the company doing the selling does everything else, including making the software available reliably.

With that in mind, read again what AWS announced last week:

The storage layer is distributed, fault-tolerant, and self-healing, giving you the the performance, scalability, and availability needed to run production-scale MongoDB workloads.

AWS is not selling MongoDB: what they are selling is “performance, scalability, and availability.” DocumentDB is just one particular area of many where those benefits are manifested on AWS.

Make no mistake: these benefits are valuable. There is a secular shift in enterprise computing moving to the cloud, not because it is necessarily cheaper (although costs are more closely aligned to usage), but because performance, scalability, and availability are hard problems that have little to do with the core competency and point of differentiation of most companies.

Those are, though, the core competency of AWS, which can bring unmatched scale to bear on solving them: by effectively operating the servers for millions of customers Amazon can apply more resources to all of those issues than any one company could on its own, as well as develop its own customer architecture, from datacenter software down to custom chips (and drive a hard bargain for hardware from suppliers like Intel).

The result is that “performance, scalability, and availability” is a tremendously attractive business: the more customers AWS has not only drive that much more recurring revenue, but also deepen AWS’ moat by allowing the company to bring that many more resources to bear on ever more obscure use cases, making AWS that much more attractive to new customers. Microsoft is competing but is a distant second; Google is even further behind. In fact, even MongoDB’s managed service runs on the three giants: it simply makes no sense to go it alone.

The Open Source Conundrum

Thus we have arrived at a conundrum for open source companies:

  • MongoDB leveraged open source to gain mindshare.
  • MongoDB Inc. built a successful company selling additional tools for enterprises to run MongoDB.
  • More and more enterprises don’t want to run their own software: they want to hire AWS (or Microsoft6 or Google) to run it for them, because they value performance, scalability, and availability.

This leaves MongoDB Inc. not unlike the record companies after the advent of downloads: what they sold was not software but rather the tools that made that software usable, but those tools are increasingly obsolete as computing moves to the cloud. And now AWS is selling what enterprises really want.

Worse, because AWS doesn’t have access to MongoDB (it is only matching the API) it only supports MongoDB 3.6; the current version is 4.0.5. It is possible that if AWS’ service becomes popular MongoDB will effectively stagnate: sure, you can get a better version from MongoDB Inc., but then you have to manage it yourself or go the effort to tie in all of your AWS services with MongoDB’s offering (then again, the potential for differentiation may be MongoDB’s salvation, and an important lesson for other companies).

Not that permissive licensing would necessarily help: Redis Labs offers its Redis database under a permissive license; that means that AWS’ offering is usually up-to-date, which is good for Redis development, but doesn’t help Redis Labs make any money. That compelled Redis Labs to change the licensing on its add-on modules to add the “Commons Clause”; this compels service providers to pay for their use, effectively making them proprietary software.

It’s hard to not be sympathetic to MongoDB Inc. and Redis Labs: both spent a lot of money and effort building their products, and now Amazon is making money off of them. But that’s the thing: Amazon isn’t making money by selling software, they are making money by providing a service that enterprises value, and both MongoDB and Redis are popular in large part because they were open source to begin with.

Economic Realities and the Future

Little of what I wrote is new to folks in the open source community: the debate over the impact of cloud services on open source has been a strident one for a while now. I think, though, that the debate gets sidetracked by (understandable) discussions about “fairness” and what AWS supposedly owes open source. Yes, companies like MongoDB Inc. and Redis Labs worked hard, and yes, AWS is largely built on open source, but the world is governed by economic realities, not subjective judgments of fairness.

And that is why I started with music: it wasn’t necessarily “fair” that music industry sales plummeted, and yes, companies like Apple with its iPod business made billions off of piracy. The only reality that mattered, though, was that music itself, thanks to its infinite reproducibility, was as pure a commodity as there could be.

It’s the same situation with software: bits on a disk are fundamentally free — just ask Richard Stallman. In his seminal essay Why Software Should Be Free Stallman wrote:7

A copy of a program has nearly zero marginal cost (and you can pay this cost by doing the work yourself), so in a free market, it would have nearly zero price. A license fee is a significant disincentive to use the program. If a widely useful program is proprietary, far fewer people will use it.

It is easy to show that the total contribution of a program to society is reduced by assigning an owner to it. Each potential user of the program, faced with the need to pay to use it, may choose to pay, or may forego use of the program. When a user chooses to pay, this is a zero-sum transfer of wealth between two parties. But each time someone chooses to forego use of the program, this harms that person without benefiting anyone. The sum of negative numbers and zeros must be negative.

But this does not reduce the amount of work it takes to develop the program. As a result, the efficiency of the whole process, in delivered user satisfaction per hour of work, is reduced.

This tradeoff is inescapable, and it is fair to wonder if the golden age of VC-funded open source companies will start to fade (although not open source generally). The monetization model depends on the friction of on-premise software; once cloud computing is dominant, the economic model is much more challenging.

That, though, should give pause to AWS, Microsoft, and Google. It is hard to imagine them ever paying for open source software, but at the same time, writing (public-facing) software isn’t necessarily the core competency of their cloud businesses. They too have benefited from open-source companies: they provide the means by which their performance, scalability, and availability are realized. Right now everyone is winning: simply following economic realities could, in the long run, mean everyone is worse off.

I wrote a follow-up to this article in this Daily Update.

  1. “Mongo” comes from the word humongous [↩︎]
  2. I’m sorry, but this next bit is going to be dry; bear with me please [↩︎]
  3. To be clear, I’m not saying that Google has a license; rather, that MongoDB offers alternative licenses [↩︎]
  4. I’m not going to get into the SSPL, but it is very controversial: many detractors argue it is not an open source license because it does not abide by the freedom to run a program for any purpose, and it may not not be enforceable [↩︎]
  5. I’m using software-as-a-service as an umbrella term for infrastructure-as-a-service and platform-as-a-service [↩︎]
  6. Which, by the way, has its own MongoDB compatible offering [↩︎]
  7. To be clear, I don’t agree with Stallman on a whole host of things; that doesn’t diminish his importance as a thinker or influence on the industry, though, or his insights on the nature of software [↩︎]

Apple’s Errors

As rare as last week’s Apple revenue warning from CEO Tim Cook may have been — the company last issued a revenue warning in June 2002 — the company has had other bad quarters in the iPhone era. Look no further than the stock chart:

Apple's stock price in the iPhone era

Three troughs stand out:

  • In fiscal year 2013 (the iPhone 5 cycle),1 Apple’s year-over-year revenue growth slowed to 18%, then 11%, 1%, and 4%; this was a dramatic slowdown from 73%, 59%, 23%, and 27% the year before. Worse, net income growth actually went negative (0%, -18%, -22%, -9%) thanks to a significant drop in margin.
  • In fiscal year 2016 (the iPhone 6S cycle), Apple’s year-over-year revenue growth went negative (2%, -13%, -15%, -9%); again, net income was worse (2%, -22%, -27%, -19%), thanks in part to a $2 billion inventory write-off.
  • This year does project to be the worst first quarter of all three: a -5% revenue decline, and -1% net income decline; this decline comes after last quarter’s announcement that Apple would no longer disclose unit sales, which precipitated the current slide in the stock price.

What makes this quarter seem so much worse was both the already negative sentiment surrounding the shift in Apple’s reporting (the presumption being the company wanted to hide declining unit sales), and also the fact that Apple’s management forecast was so off: here is CFO Luca Maestri on last quarter’s earnings call:

As we move ahead into the December quarter, I’d like to review our outlook, which includes the types of forward-looking information that Nancy referred to at the beginning of the call. We have the strongest lineup ever as we enter the holiday season and we expect revenue to be between $89 billion and $93 billion, a new all-time record.

In fact, the only record, such that there was, was the size of the miss. So what went wrong?

On Confirmation Bias

If you will forgive a brief aside, this article requires a few very large caveats: first, Apple has not yet released its final quarter numbers, had its earnings call, or filed it’s 10-Q; there is a lot of information still to come.

Secondly, thanks in part to the lack of information, this miss is catnip for confirmation bias: everyone has their pet theory about what Apple is doing wrong or how they will ultimately fail, and it has been striking the degree to which this revenue warning has been breezily adapted to show that said critics were right all along (never mind that many of those critics trotted out the exact same explanations in 2013 and 2016).

Third, well, I happen to think that I am right as well: I believe that Apple’s management made three critical errors in their forecast for this last quarter that were predictable precisely because they had made the same errors before — errors that I wrote about at the time. In other words, I am very much susceptible to confirmation bias as well.

That noted, if indeed I am right, then that is good news for Apple: I suspect the company is in better shape than the last week of hysteria suggests.

Error 1: China and ‘S’ Cycles

The most important takeaway from the revenue warning is that the vast majority of the problem in Apple’s forecast comes from Greater China. From Cook’s letter:

While we anticipated some challenges in key emerging markets, we did not foresee the magnitude of the economic deceleration, particularly in Greater China. In fact, most of our revenue shortfall to our guidance, and over 100 percent of our year-over-year worldwide revenue decline, occurred in Greater China across iPhone, Mac and iPad…

The problem was specifically around the iPhone:

Lower than anticipated iPhone revenue, primarily in Greater China, accounts for all of our revenue shortfall to our guidance and for much more than our entire year-over-year revenue decline. In fact, categories outside of iPhone (Services, Mac, iPad, Wearables/Home/Accessories) combined to grow almost 19 percent year-over-year.

That this exact quarter would be challenging for Apple is exactly what I predicted in May 2017 in Apple’s China Problem; specifically:

  • In most of the world, Apple is differentiated first-and-foremost by its integration between hardware and software; the company has a “monopoly” on iOS, which allows it to sell its hardware at much higher prices than the competition.
  • However, in China iOS is much less of a lock-in, thanks to the dominance of cross-platform Chinese-specific services, particularly WeChat (WeChat, while the most important factor, is not the only one: indeed, given that Android in China is specifically tuned to the Chinese market by Chinese OEMs, iOS is if anything a hindrance).
  • The net result is that Apple in China competes not on the basis of integration, but rather on the attractiveness of its hardware; in other words, Apple is, to far greater degree in China than anywhere else, simply another OEM.

I wrote at the time:

For the day-to-day lives of Chinese there is no penalty to switching away from an iPhone. Unsurprisingly, in stark contrast to the rest of the world, according to a report earlier this year only 50% of iPhone users who bought another phone in 2016 stayed with Apple:

This is still better than the competition, but compared to the 80%+ retention rate Apple enjoys in the rest of the world, it is shockingly low, and the result is that the iPhone has slid down China’s sales rankings: iPhone sales were only 9.6% of the market last year, behind local Chinese brands like Oppo, Huawei and Vivo. All of those companies sold high-end phones of their own; the issue isn’t that Apple was too expensive, it’s that the iPhone 6S and 7 were simply too boring.

At the end I concluded that Apple’s next phone — what turned out to be the iPhone X — would return the company to growth in China, and so it did: Apple was up 11%, 21%, 19%, and 16% last fiscal year, after declining by double digits six of the previous eight quarters. The other half of that prediction, though, was that the next ‘S’ model, with only component upgrades in the same form factor, would struggle; that is exactly what appears to have happened.

To be sure, there are absolutely other issues in China, particularly the country’s significant economic slowdown as well as the possibility of anti-U.S. company sentiment thanks to the ongoing trade war and the arrest of Huawei’s CFO. I strongly suspect, though, that those macroeconomic factors made what would have been a tough quarter for Apple regardless that much worse; to put it another way, Apple is far more exposed to challenging macroeconomic conditions in China than they are elsewhere thanks to their relative lack of a moat.

There are two adjustments Apple needs to make to avoid this error in the future: first, and most obviously, the company needs to be far more pessimistic with regard to its China forecasts in ‘S’ model years. Second, management needs to appreciate that the plane of competition in China is different than the rest of the world: the company is a luxury brand, but only in terms of hardware. If anything, iOS in China needs to cater more to the local market; as far as hardware, perhaps it is time for the ‘S’ strategy to be retired.

Error 2: Non-Flagship iPhones

There is one complicating factor in the last piece of analysis: Apple’s provided their last forecast on November 1, a full month into the quarter; did they not see this massive China miss coming? Perhaps the economy simply crashed in the last two months?

That may be true, but I don’t think it is the entire explanation; Apple also tripped itself up with the staggered release of iPhones both this year and last:

  • In September 2017 (FY2017 Q4) Apple released the non-flagship iPhones 8 and 8 Plus
  • In November 2017 (FY2018 Q1) Apple released the flagship iPhone X
  • In September 2018 (FY2018 Q4) Apple released the flagship iPhones XS and XS Max
  • In (late) October 2018 (FY2019 Q1) Apple released the iPhone XR

This schedule resulted in two blind spots for Apple: first, the company’s FY2018 Q4 results in China were almost certainly artificially high, thanks to Error 1. Of course the iPhone XS should have a strong year-over-year comp to the iPhone 8, but that comp was likely particularly extreme in China.

Second, to the extent that iPhone XS sales slowed in October, Apple likely expected the iPhone XR to pick up the slack;2 I strongly suspect the XR failed to live up to expectations.

This too, though, should have been predictable: sure, from a feature perspective the XR seemed remarkably competitive with the XS, but we have ample evidence that iPhone buyers want the best possible iPhone. After this year’s iPhone keynote I wrote:

There is, of course, the question of cannibalism: if the XR is so great, why spend $250 more on an XS, or $350 more for the giant XS Max? This is where the iPhone X lesson matters. Last year’s iPhone 8 was a great phone too, with the same A11 processor as the iPhone X, a high quality LCD screen like the iPhone XR, and a premium aluminum-and-glass case (and 3D Touch!). It also had Touch ID and a more familiar interface, both arguably advantages in their own right, and the Plus size that so many people preferred.

It didn’t matter: Apple’s best customers, not just those who buy an iPhone every year, but also those whose only two alternatives are “my current once-flagship iPhone” or “the new flagship iPhone” are motivated first-and-foremost by having the best; price is a secondary concern. That is why the iPhone X was the best-selling smartphone, and the iPhone 8 — which launched two months before the iPhone X — a footnote.

It remains to be seen the extent to which this is the case globally, but the market where having the flagship matters most has always been China. iPhone XS sales slowing and not being picked up by the just-launched XR certainly explain the timing of the missed forecast.

Error 3: iPhone Destiny

This gets at the third error made by Apple management, and arguably the most concerning: the assumption that iPhone growth is inevitable.

This was seen most clearly during the iPhone 6 cycle, when Cook insisted on earnings call after earnings call — I documented his statements in this Daily Update — that Apple’s record-breaking sales were not an abnormally large number people buying new iPhones sooner than they would have thanks to the large screen. In fact, it turned out that is exactly what was happening, which is why 6S sales were such a disappointment. I concluded:3

I know I’m kind of harping on this point, but in fact I find any possible explanation for this inconsistency very troubling: either Cook was purposely overselling the upgrade narrative last year, which would not only be duplicitous but also dumb, given that he would only be setting up Apple for a fall this cycle; or even as late as last year Cook was out of touch with how the iPhone upgrade cycle actually works, or how it may have changed over time.

I strongly suspect it is the latter explanation, and while that is concerning, it’s also understandable; the implication of my ongoing contention that the iPhone has now picked all of the “low-hanging fruit” of growth is that iPhone growth had multiple causes: certainly the inherent quality and new features of each annual iPhone model played a role, but an arguably bigger factor was simply distribution — getting the iPhone onto more carriers in more countries. Indeed, I strongly suspect the predictable impact of increased distribution helps explain why Apple’s earnings forecasts were so eerily accurate for so many years.

Apple, though, has been a lot less accurate for the last five quarters: the iPhone 6 sold better than they expected in nearly every quarter, and now the iPhone 6S is selling worse (note the Maestri forecast errors I highlighted above); along similar lines, Apple seems to have underestimated iPhone SE demand to a significant degree. Ultimately, while I think Apple still has the advantage when it comes selling people their second (or third or nth) smartphones, what I think we are seeing is that it’s a lot more difficult to determine when exactly that sale will occur, and Apple itself is only now coming to grips with that.

This gets at the rest of the miss — the non-China parts, especially. Cook cited the end of carrier subsidies (largely an old story at this point, to be fair), a stronger U.S. dollar, and customers getting new batteries instead of new iPhones. It’s a bit of a hodgepodge with one primary takeaway: convincing customers to upgrade “good enough” phones is both challenging and unpredictable, and Apple can’t simply assume it will happen at the rate it has previously.

Reasons for Optimism

The good news for Apple is that, to the extent these errors really were predictable, there is nothing structurally different about the company’s competitive position today versus six months ago, when the current stock slide began.

  • The next iPhone hardware revision should sell better in China, simply by virtue of being new (and the implication of it being easy to switch away from iOS is that it’s easy to switch back).
  • Customers still prefer Apple’s flagship iPhones, no matter how expensive they are.
  • Headwinds like currency and battery replacement programs will go away, and phones, thanks to their centrality in people’s lives as well as the greater likelihood of harm, will always have a faster replacement cycle than PCs.

Meanwhile, the company’s Services business continues to grow, along with its installed base (including in China); the company is clearly putting more strategic emphasis in this area, effectively abandoning also-ran hardware products like HomePod and Apple TV to increase the reach of its services. I would expect significant announcements in this area through 2019.

That is not to say the company is finished with hardware: wearables are a huge area of growth, as both AirPods and Apple Watch are big successes, and it seems likely that an augmented reality product is coming in the next few years. Nothing will match the iPhone, but that’s ok; the sky is not falling, only the stock.

I wrote a follow-up to this article in this Daily Update.

  1. Keep in mind that Apple’s fiscal years start on October 1st [↩︎]
  2. The company had less than a week’s worth of data about how the XR was selling by the time management made its forecast [↩︎]
  3. I first made the case for abandoning the ‘S’ strategy in this Daily Update, writing:

    To that end, I do question how much longer Apple can afford to stick with the ‘S’ strategy. Again, big screens were such an important feature that it’s difficult to take away too much from the 6S’ poor year-over-year comparisons, but it seems reasonable to wonder if the structural expansions that increased the iPhone’s addressable market papered over the fundamentally weaker value proposition presented by the ‘S’ lines. To put it another way, one could argue the ‘S’ lines are introducing a holiday quarter-like dynamic into Apple’s earnings but on a two-year basis: we may not really tell know the iPhone is doing until a completely new model that will drive upgrades comes out.

    In fact, if you look at a two-year comparison, Apple’s revenue last quarter was up 7%, a perfectly acceptable result [↩︎]

Holiday/Vacation Break: Weeks of December 24 and December 31

Stratechery is taking a holiday and vacation break the weeks of December 24 and December 31. There will be no Weekly Article or Daily Updates. The Daily Update will resume on January 7.

All new subscriptions made since November 26, including during this break, will have two weeks added to their subscriptions. See you in 2019!

The 2018 Stratechery Year in Review

In last year’s Stratechery Year in Review I noted that the predominant theme was the impact of tech on society; perhaps unsurprisingly, the dominant theme in 2018 was tech and regulation.

This year I wrote 139 Daily Updates (including tomorrow) and 40 Weekly Articles, and, as per tradition, today I summarize the most popular and most important posts of the year; the question of regulating tech generally and Facebook’s foibles specifically figure prominently. Frankly, I plan on making an effort to spend more time elsewhere next year.

In March I also launched Stratechery 4.0, with dramatically improved search; easier access to the archives via Concepts, Companies, and Topics; as well as a visual refresh and new logo.1

You can find previous Stratechery Years in Review here: 2017 | 2016 | 2015 | 2014 | 2013

A drawing of Amazon's Positioning as Healthcare Middleman

Here is the 2018 list.

The Five Most-Viewed Articles

Stratechery once again had record traffic in 2018, and although none of the articles matched the juggernaut that was 2017’s Amazon’s New Customer, the first two articles on this list were the second and third-most popular articles all-time.

  1. The End of Windows — The Windows division no longer exists at Microsoft, marking the end to a four-year process of changing Microsoft’s culture.
  2. Amazon Health — Amazon Health doesn’t seem like much now, but there are hints it could be the ultimate application of Aggregation Theory.
  3. Lessons From Spotify — Spotify has a marginal cost problem, but while the cause is unique to Spotify, the challenges are more applicable than it seems.
  4. The Bill Gates Line — Understanding the differences between aggregators and platforms matters both for companies interacting with them and regulators considering antitrust.
  5. Amazon Go and the Future — Amazon Go exemplifies how Amazon is building its monopoly in three ways: horizontally, vertically, and financially. Plus, why automation is worth being optimistic about.

A drawing of The Moat Map

The Evolution of Aggregation Theory

I am very pleased at the way I fleshed out Aggregation Theory this year, both in theoretical terms and also in its importance when it comes to thinking about regulation.

  • Zillow, Aggregation, and Integration explores why owning the customer relationship isn’t enough: an Aggregator also needs a way to transform a value chain. See also the afore-linked Lessons From Spotify.
  • Tech’s Two Philosophies, The Moat Map, and the afore-linked The Bill Gates Line explore the differences between platforms and Aggregators: the former facilitates a connection between suppliers and users, while the latter intermediates it. This distinction is critical for regulators.
  • Aggregators and Jobs-to-be-Done filled in a missing piece of Aggregation Theory: when I say that Aggregators win by having the best user experience, that means they are the best and most all-encompassing option to get a job done.

A drawing of Platform Businesses Attract Customers by Third Parties

Tech and Regulation

There were three broad subcategories when it came to tech and regulation: the tendency of regulation to entrench incumbents, antitrust, and Facebook. Lots and lots of Facebook.

Regulation:

  • Open, Closed, and Privacy — Just as encryption is only viable on closed systems, so it is that increased privacy regulations will only entrench walled gardens. That should affect thinking on regulation.
  • The European Union Versus the Internet — The EU is back to regulating tech companies, and getting the Internet wrong in the process. That, though, helps illuminate an approach that could work.
  • Data Factories — Facebook and Google and other advertising businesses are data factories, and regulation will be most effective if it lets users look inside.

A drawing of The Facebook Data Factory

Antitrust:

  • The European Commission Versus Android — Examining the history of Android explains why the European Commission may be right to fine Google for its actions around Android, even as the reasoning feels off.
  • Antitrust, the App Store, and Apple — Apple’s case before the Supreme Court is about standing; Apple has a strong case. That, though, doesn’t mean the App Store isn’t a monopoly — and that Apple isn’t increasingly predicated on rent-seeking.
  • The State of Technology at the End of 2018 — The State of Technology, at least in the enterprise space, is strong; consumer tech is another story, and it is time to question the dominance of big companies like Google.

A drawing of How The Internet Undid Microsoft's Platform Advantage

Facebook:

Facebook at the center of data exchange

The Old Guard

This year several companies from the first generation of tech had their day in the sun, in many cases because of acquisitions.

  • The Cost of Developers — Microsoft paid a lot for GitHub, because it had to pay directly for access to developers. It doesn’t have the leverage of users the way that Apple does on the App Store.
  • Intel and the Danger of Integration — Intel is in an increasingly bad position in part because it has been captive to its integrated model. Or, you could simply say they were disrupted.
  • IBM’s Old Playbook — IBM has bought Red Hat in an attempt to recreate its success in the 90s; it’s not clear, though, that the company or the market is the same.
  • The Experience Economy — SAP’s acquisition of Qualtrics shows how the shift in technology has changed business; it is a perfect example of using the Internet to one’s advantage.
  • Apple’s Middle Age — For Apple, hitting middle age means a strategy primarily focused on monetizing its existing customers. It makes sense, but one wonders what happens next (See also, The iPhone Franchise and Apple’s Social Network).

Integrated intel was competing with a competitive modular ecosystem

More Analysis

There was still room for Stratechery staples: analysis of industries and companies in the context of the bigger picture.

A drawing of The Asymptote Version of the User Experience

The Year in Daily Updates

This was another strong year for the Daily Update, which continued its trend towards being one fully-fledged topic with three areas of focus; the difference between Weekly Articles and Daily Updates is smaller than ever (you should subscribe!). Here are some of my favorites:

A drawing of Apple, Microsoft, Google, and Facebook on tech's compass
A special Daily Update drawing from Platforms Versus Aggregators, What About Amazon?, Walmart Buys Flipkart

I also conducted three interviews for The Daily Update:


I can’t say it enough: I am so grateful to Stratechery’s readers and especially subscribers for making all of these posts possible. I wish all of you a Merry Christmas and Happy New Year, and I’m looking forward to a great 2019!

  1. And merchandise! [↩︎]

The State of Technology at the End of 2018

This article is a bit of an annual tradition: in mid-December I summarize the state of technology,1 and appropriately enough, this year’s edition coincides with a tech executive testifying in front of Congress. This time the executive was Sundar Pichai, the CEO of Google, and on the surface, it was more of the same; Casey Newton wrote:

From time to time the entire technology press corps gets together on Twitter, spends several hours live-tweeting the same event, and then writes a series of blog posts about how nothing important happened. This event is known as a Congressional hearing, and today we witnessed our final one of the year.

Newton’s pithy summary, though, missed one essential part of the script: the Twitterati complaining about just how stupid members of Congress are:

It’s hard to deny Ohanian the point: Congressman Lamar Smith’s line of questioning was — and I swear this is exactly what I wrote in my notes as I watched the hearing — “freaking delusional”.2

Smith Versus Pichai

Congressman Smith, like many of his Republican colleagues, was concerned about Google being biased against Conservatives;3 Congressman Smith stated:

Google has revolutionized the world, though not entirely in the way I expected. Americans deserve the facts objectively reported. The muting of conservative voices by Internet platforms has intensified, especially during the Presidency of Donald Trump. More than 90% of all Internet searches take place on Google or YouTube and they are curating what we see. Google has long faced criticism for manipulating search results to censor Conservatives. Organizations have had pro-Trump content tagged as hate speech or had content reduced in search results. Enforcement of immigration laws has been tagged as hate speech as well. Such actions pose a grave threat to our democratic form of government. PJ Media found 96% of search results for Trump were from liberal media outlets. In fact, not a single right-leaning site appeared on the first page of results. This doesn’t happen by accident, but is baked into the algorithm. Google’s algorithms…It will require a herculean effort in senior management to change the political bias now programmed into the company’s culture.

Pichai, as he did throughout the hearing, explained that Google did not manipulate search results for partisan ends, and that it would it not be in their business interest to do so.

This is, to be clear, correct: Google’s business is perhaps the most perfect example of a capital-intensive tech company there has ever been. The company spends huge amounts of money on research-and-development and back-end infrastructure for the sake of offering services and advertisements that have zero marginal costs. It follows, then, that the company is heavily incentivized to serve as many users as possible; being purposely biased against approximately 50% of them would be illogical.

Congressman Smith, though, was not convinced, leading to the exchange Ohanian highlighted:

Congressman Smith: To my knowledge, you have never sanctioned any employee for any type of manipulating the search results whatsoever. Is that the case?

Pichai: It’s not possible for an individual employee to manipulate the search results. We have a robust framework including many steps in the process.

Congressman Smith: I disagree. I think they can manipulate the process.

I mean, what are you supposed to say to that? Any person that works at Google — indeed, any person that has worked in any technology company of even the slightest scale — knows that it would be impossible for a rogue employee to manipulate search results. Good luck, though, convincing Congressman Smith.

Google’s Impregnability

Still, as a thought experiment, suppose Congressman Smith were right, and that Google’s search results, whether via managerial decree, general employee bias, or rogue employee, were gamed to disfavor Conservatives. The solution seems clear: create a competitor to serve the part of the market that is dissatisfied with Google. After all, this is a company that made $110 billion in revenue and $27 billion in pre-tax income;4 big profits mean a big opportunity for competitors, right? So what is Congressman Smith complaining about?

The issue, of course, is that Google is, at least for a while (and more on this in a bit), impregnable: the company is an Aggregator with positive feedback loops everywhere:

  • A superior search product earns users, leading to more data and more supply that leads to better results, earning more users.
  • Superior ad inventory that attracts advertisers, leading to more data that, when combined with aggregated users, leads to more inventory that is (justifiably) more expensive than alternatives, resulting in outsized revenue and profits.
  • Outsized revenue and profits make it possible to acquire complementary companies (like DoubleClick), new sources of growth (like YouTube), and invest massively in research and development (for products like Android and TensorFlow), all of which serve to accelerate the first two feedback loops.

The result is that consumers — whatever their political affiliation or feelings about bias — use Google because it is the best option, and, for all of Google’s technical brilliance, its insurmountable “bestness” is, at this point in the company’s history, more due to the frictionless structure of the Internet, with its zero distribution and transaction costs that make it possible for a company to achieve Google’s insurmountable scale, than it is due to any sort of unique innovation.

The Cost of Dominance

But still, so what? Google offers tremendously valuable services for no direct cost to consumers. What’s the problem? It is certainly hard for the American antitrust community to find any, thanks to the consumer welfare standard: Google is not raising prices for consumers, they are lowering them, in basically every market they enter.

The question that must be asked, though, is at what cost? This year’s set of Congressional hearings suggest that one casualty is any sort of effective government oversight: a lack of competition for not just Google but also Facebook, particularly in terms of digital advertising, combined with an antitrust philosophy stripped of even a shred of suspicion about sheer size and the political and economic power that inevitably follow, means politicians are left with little recourse other than vague references to regulations that will only entrench the two consumer tech giants at best, and Smith-style conspiracy theories at worst (and, I’d note, it is not as if Progressives are thrilled about Google and Facebook’s content moderation policies and algorithms either).

Equally concerning is the innovation that is not happening: venture investment in seed rounds and initial follow-ons is down considerably, and numerous studies (like here, here, and here) show that most of the decline is in the consumer space — i.e. the domain of Google, Facebook, Amazon, and Apple (I wrote about Apple’s problematic App Store two weeks ago).

Moreover, the VC-backed tech industry knows better than anyone that this is not because large companies, with their top-down decision-making, are inherently better at innovation. The goal of venture capital is to make multiple bets on ideas with extremely uncertain outcomes, because the best way to figure out what works is to let the market decide, not mid-level managers. That strategy, though, isn’t nearly as successful if the market isn’t functioning correctly.

In contrast, consider the enterprise software market: here the Internet has very much lived up to its billing, unleashing a torrent of innovative companies made possible by cloud computing, that are challenging lumbering incumbents up-and-down their product lines. And, to their credit, some of those incumbents, like Microsoft, are responding in kind, dramatically overhauling their core strategies and releasing new products and services that are innovative in their own right. Small wonder both venture capital investment and the IPO market are dominated by these enterprise startups: functioning markets have positive feedback loops of their own.

The State of Technology

This, then, is the state of technology in 2018: the enterprise market is thriving, and the consumer market is stagnant, dominated by the “innovations” that a few large behemoths deign to develop for consumers (probably by ripping off a smaller company). Meanwhile a backlash is brewing on both sides of the political spectrum, but with no immediately viable outlet through competition or antitrust action, the politics surrounding technology simply becomes ever more rancid.

Still, some might argue, this moment may soon pass: just look at Microsoft. I praised them above for their new-found competitiveness, driven by the fundamental shift wrought by the combination of cloud computing and mobile that obviated their PC monopoly-based business model. Surely Google’s dominance will soon pass, just like Microsoft’s did, right?

I’m not so sure.

The Internet Age

The single most important factor in the loosening of Microsoft’s monopoly was the Internet. Suddenly applications could be run and data could be stored in a way that was independent of the underlying operating system, undoing Microsoft’s platform lock-in.

How the Internet undid Microsoft's platform advantage

This didn’t affect Microsoft immediately — people were already accustomed to buying PCs (although it is not solely because of Steve Jobs’ return that the Mac’s fortunes increased in line with Internet penetration) — but it created an ecosystem that made a device like the iPhone, with its groundbreaking browsing capabilities, immediately useful in a way it wouldn’t have been otherwise. That attracted consumers, which attracted developers to the App Store, and the rest is history.

That story extended to enterprise: not only were more and more line-of-business applications delivered via the cloud, but new companies providing services that competed with Microsoft were far quicker to support mobile, providing a compelling reason to switch, unwrapping Microsoft’s bundle and opening the door to new companies of all types.

Again, though, all of this was because of the Internet, a paradigm shift that I have repeatedly likened to the Industrial Revolution in the profound impact I expect it to have when all is said-and-done. How often, though, do such paradigm shifts actually happen? Yes, the Internet saved the industry from Microsoft, but are we so sure another Internet-level shift, one that will upend Google’s dominance, is on the horizon?5 And how much foregone innovation and political dysfunction are we willing to suffer in the meantime?

I wrote a follow-up to this article in this Daily Update.

  1. Here is 2014, 2015, and 2016; I skipped it last year in order to cover Disney’s acquisition of 21st Century Fox [↩︎]
  2. OK, fine, I might have used a slightly different adjective [↩︎]
  3. The capitalization is intentional, in reference to the distinct American political movement [↩︎]
  4. Google took a special charge related to the recent tax law last year, artificially lowering its net income to $12.6 billion [↩︎]
  5. Yes, the blockchain is fundamentally interesting, particularly its decentralized nature, along with the idea of digital scarcity; I suspect, though, that when and if blockchain applications achieve meaningful use cases they will be in areas fundamentally different than Google and Facebook, which are predicated on attractive user experiences [↩︎]

Aggregators and Jobs-to-be-Done

There were two developments in the scooter space over the last week. First, Bird announced the Bird Platform for entrepreneurs to start their own scooter services; from TechCrunch:

The company will provide the independent operators with scooters, which they are given free rein to brand as they please, as well as access to the company’s marketplace of chargers and mechanics, in exchange for 20 percent of the cost of each ride. Bird says fleet managers, which may be independent entrepreneurs or local mom and pop bike rental shops, for example, can also collect and charge the scooters themselves.

There is an optimistic view of Bird Platform: fleet operators on the Bird platform will allow the company to expand more rapidly than it would otherwise, even while Bird continues to (mostly) own the customer relationship (fleet operators will get their own app, but scooters will also appear in the Bird app). There is also a pessimistic view: Bird is offloading the risk involved in owning and managing scooters because their costs are unsustainably high, and moats unsustainably shallow.

Then, a few days later, The Information reported that Uber was exploring the possibility of buying either Bird or Lime, their primary competitor:

Uber, which already holds a minority stake in Lime, is evaluating both Bird and Lime as it looks to expand further into the fast-growing market for electric scooter services. A deal with either Bird or Lime could be reached before the end of the year, one of the people said, though there still is a possibility neither will happen. While financial terms of the talks couldn’t be learned, Bird was valued at $2 billion in its previous fundraising round, while Lime’s last valuation was $1.1 billion. Both have also been trying to raise more money at much higher valuations in recent months.

There are still big questions about the financial viability of scooter rental services. For ride-hailing companies, the hope is that they and bike-rental services can be used to handle shorter trips in dense cities, though it is possible they could eat into their core car-based businesses.

Once again, there are two ways to view this news; start with the pessimistic take in The Information (later reported by the Financial Times): dockless scooters are eating into traditional ride-sharing, which means Uber is interested in buying one of the leading scooter-sharing companies so that the company is at least cannibalizing itself. More optimistically, Uber is where scooter-sharing should have been all along.

Uber’s Job-to-be-Done

Theodore Levitt, the former Harvard Business School professor and editor of the Harvard Business Review, famously said “People don’t want to buy a quarter-inch drill. They want a quarter-inch hole.” The idea, which is at the core of well-known innovation frameworks like Outcome-Driven Innovation and Jobs-to-be-Done, is that effective customer segmentation relies not on easily measurable attributes like demographics or location — much less product features and prices — but rather on a deeper understanding of what the consumer is trying to accomplish.

With this approach it quickly becomes obvious that, for all of the differences in their underlying businesses, Uber and the scooter companies are doing the same “job”: transporting users to a desired destination. Sure, the means are different — human-driven cars versus dockless scooters — which trickles down into the core mechanics and defensibility of their business models, but customers don’t care about all that: they just want to get to where they want to be.

There was a time when the customer’s point of view might not have mattered quite so much; it used to be the case that success depended on controlling the supply of a good or service, or owning the distribution channels through which goods or services flow. The difference with the Internet — and it is a difference that, thanks to smartphones, very much affects real world goods like cars and scooters — is that goods and services can, at least in theory, reach anyone. Distribution is free, and in markets where supply is plentiful, value accrues to the companies that own demand — that is, those that have the most end users thanks to their superior user experience; I call them Aggregators.

Aggregators and the User Experience

I mentioned the importance of the user experience in my original formulation of Aggregation Theory:

The fundamental disruption of the Internet has been to turn this dynamic on its head. First, the Internet has made distribution (of digital goods) free, neutralizing the advantage that pre-Internet distributors leveraged to integrate with suppliers. Secondly, the Internet has made transaction costs zero, making it viable for a distributor to integrate forward with end users/consumers at scale.

This has fundamentally changed the plane of competition: no longer do distributors compete based upon exclusive supplier relationships, with consumers/users an afterthought. Instead, suppliers can be commoditized leaving consumers/users as a first order priority. By extension, this means that the most important factor determining success is the user experience: the best distributors/aggregators/market-makers win by providing the best experience, which earns them the most consumers/users, which attracts the most suppliers, which enhances the user experience in a virtuous cycle.

What, though, makes for a good user experience? I have always been careful to distinguish between user interface and user experience: for example, many people find Facebook’s user interface to be confusing, but that is only one aspect of the user experience; another aspect is whether or not your friends or family are on the service, and here Facebook’s overall user experience is very strong indeed.

This distinction underscores the importance of the virtuous cycle characteristic of all Aggregators: new suppliers — whether they be drivers on Uber, products on Amazon, content on Google or Facebook, shows on Netflix, apps on the App Store, etc. — attracted to the platform by the existing userbase enhance the user experience, even though have nothing to do with the user interface. Still, the user experience of what?

Here the question of Uber and scooters perhaps provides some insight: the “user experience” for Uber is just how well the service does at transporting people where they wish to go. Along those lines, think about the virtuous cycle I’ve described between supply and demand and its impact on the user experience: the more riders there are, the more drivers come onto the platform (both in the short-term through higher prices and the long-term through reliable demand); the more drivers there are, the more reliable Uber is as a transportation service, increasing demand, and so it goes.

Note, then, the similarities between my summary of the Uber user experience — how well the service does at transporting people where they wish to go — and my earlier description of the “job” that Uber does — transporting users to a desired destination. In short, the “user experience” that propels Aggregators is how well they do the “job” customers need in the space in which they compete. Or, to put it another way, if you want to know where in a value chain an Aggregator is likely to form, figure out what and where the “job” is.

Aggregators and Jobs-to-be-Done

Consider examples from Aggregators of all types:

  • Google is the best at doing the job of answering questions; that is how they gained their initial userbase, which attracted content suppliers of all types to formulate their content for Google, making Google even better at its job. Indeed, when it comes to answering questions, it is striking the degree to which Google has improved not simply in general queries but also in vertically-specific ones, thanks in large part to content suppliers willingly tuning their content to Google’s specifications.
  • Facebook started as the best place to find your friends and family, but, over time, evolved into being the best place to waste time. The former created an obvious virtuous cycle — more friends and family meant more users meant more friends and family — but so did the latter: more users meant more content suppliers eager for eyeballs, which made it that much more alluring a place to waste time, and for longer (some call this engagement).
  • Amazon is the best place to buy things; the company famously started by having more books than anyone, which attracted customers, which made it possible for the company to offer more products, and later merchants, which attracted more customers, eventually evolving into the hybrid store/merchant platform that Amazon.com is today.
  • Netflix is the best place to watch TV. The company gained its initial streaming userbase by licensing Starz’ 11,000 title movie library; while Starz’ effective library size was one (whatever was showing on the Starz channel), Netflix’s was 11,000. That attracted users, which gave Netflix the funding (and prospect for future funding, realized through debt) to buy more shows, attracting more users, providing funding to eventually make their own shows, attracting more users still.
  • YouTube is the best place to find video content about basically anything (for better or worse); the site started by being the only online video site willing to show copyrighted material, which attracted users, which attracted more video makers, and eventually, copyright owners themselves.

All of these jobs are quite straightforward and understandable: answer questions, waste time, buy things, watch TV, find videos; the Aggregators are the ones that do the job the best, both initially through some valuable insight, technical (Google) or otherwise (Facebook), and presently thanks to the virtuous cycle that followed. The user experience is the quality of the job done.

The Expansionary Nature of Aggregators

To return to Uber, this formulation argues strongly for the sort of acquisition under discussion: a car isn’t always the best means of transportation, which means that Uber is not doing the job customers ask it as well as it can. And, on the flipside, a scooter company on its own doesn’t exactly fit the bill either; add in the fundamental indefensibility of dockless scooters and Uber’s very large userbase looks like a truly differentiating asset.1

Another company worth considering is Airbnb: the home-sharing startup has resolutely stuck to, well, shared homes. One wonders, though, if “find a shared home” is the job customers are asking Airbnb to do; I suspect the better answer is “find a place to stay.” Doing that job well, though, means including hotels in Airbnb’s listings, a step the company has so far declined to make at scale.2 I can understand the reticence: shared homes are what the service is known for; then again, Uber was once thought of as being for black cars, and Amazon for books.

More generally, that virtuous cycle characteristic of Aggregators, where more users attract more suppliers which attract more users, is likely most important in terms of the breadth with which a job is done. By doing more of a job, an Aggregator attracts more marginal users, which attract more suppliers on the edges of a space, which expands what jobs can be done for what users. In concrete terms, Amazon started by selling things to book buyers, then expanded to selling things to CD buyers, until it now sells everything to everyone; the job-to-be-done, though, was only ever selling things.

This helps explain why it is there are a few large companies that dominate their space: Aggregators don’t simply get better at what they were already good at, they expand their purview into the broadest possible definition of their job. Google, for example, was once thought to be under threat by vertical search alternatives; it turns out vertical search alternatives were under threat by an expansionary Google. The only exception is shopping and Amazon, but frankly, that is a different job anyways; it seems job-to-be-done defines not only the aggregation opportunity, but its limit as well.

I wrote a follow-up to this article in this Daily Update.

  1. Of course, lots of deals that make sense don’t happen, and this one, thanks to the sky-high valuations of both Uber and the scooter companies, will be particularly challenging [↩︎]
  2. The company has added some boutique hotels [↩︎]