Why Web Pages Suck

John Gruber had strong words about Apple news site iMore:1

I love iMore. I think they’re the best staff covering Apple today, and their content is great. But count me in with Nick Heer — their website is shit-ass. Rene Ritchie’s response acknowledges the problem, but a web page like that — Rene’s 537-word all-text response — should not weigh 14 MB.1.

It’s not just the download size, long initial page load time, and the ads that cover valuable screen real estate as fixed elements. The fact that these JavaScript trackers hit the network for a full-minute after the page has completed loaded is downright criminal. Advertising should have minimal effect on page load times and device battery life. Advertising should be respectful of the user’s time, attention, and battery life. The industry has gluttonously gone the other way. iMore is not the exception — they’re the norm. 10+ MB page sizes, minute-long network access, third-party networks tracking you across unrelated websites — those things are all par for the course today, even when serving pages to mobile devices. Even on a site like iMore, staffed by good people who truly have deep respect for their readers.

It’s that last line that should give Gruber, or anyone else complaining about crappy websites, pause. After all, if iMore respects their readers, the only alternative explanation is that their development team is incompetent. Unless, of course, iMore, along with the vast majority of ad-supported sites on the web, has basically no choice in the matter.

Advertisers and the Early Web

Gruber talks about publishers and readers, but if you begin with the premise that web pages need to be free,2 then the list of stakeholders for most websites is incomplete without the inclusion of advertisers. After all, they’re the ones that pay the bills.

Back when the web first became an important medium it actually wasn’t particularly great for advertisers. In the pre-Internet days an advertiser could buy ads in the local paper, perhaps a TV spot or two, put up a billboard and call it a day, confident they were reaching their entire target segment as well as they could. To do the same with web ads, on the other hand, required somehow knowing what websites your target customers were visiting, which could number in the hundreds or thousands, and then buying ads on those websites and crossing your fingers your customers actually saw your ad (although you would never know if they did). It was a mess.

The problem was one of scale, and in two dimensions:

  • There were way more places to advertise than before, which sounds great in theory but actually stunk in practice, because who has the time and resources to deal with hundreds or thousands of different ad sales teams?
  • Any one website only knew what its visitors were interested in on that particular website, which meant the targeting ability a website could sell to advertisers was scarcely better than a physical newspaper selling a spot in the Sports section to a gym

The result is that as late as 2010, when Mary Meeker for the first time used the following slide in her annual Internet Trends report, the Internet’s share of advertising was significantly less than its share of attention:

meeker2010

Ad networks and programmatic advertising, though, changed everything.

The Rise of Ad Networks

Ad networks solved both scale problems:

  • Instead of buying ads on a plethora of (relatively-speaking) little websites, advertisers could centralize their buying with an ad network that promised to place their ads across said plethora.
  • By virtue of having ads — and their associated trackers — across the aforementioned plethora of sites, networks could get a much richer picture of individual visitors giving advertisers the ability to much more finely target their ads.

The way it actually works is a little complicated: unlike print ads, which were delivered days ahead of time and inserted along with editorial copy before going to press, ads today are delivered “programmatically”. The process is actually kind of amazing, and consists of several different pieces (my reference to “ad networks” has been a bit simplistic):

  • When a user requests a URL, the publisher checks to see if they have any directly sold ads available (because of the scale problems noted above, fewer and fewer publishers have fewer and fewer directly-sold ads; advertisers just aren’t interested)
  • If they don’t, the publisher asks an ad exchange for an ad
  • The ad exchange, which has built up a profile of the user across all the different sites where the ad exchange is used, sends the (anonymized) user profile and website description to a variety of demand-side platforms (DSPs) (which actually sell the ads)
  • The DSPs examine the user profile and website description and a host of other factors and offers up the price they are willing to pay to serve an ad to the user
  • The ad exchange selects the highest price, retrieves the ad, and sends it to the publisher to display

All of this happens on a just-in-time basis, and you can see why advertisers love it: to a greater extent than ever before they are reaching exactly who they want to reach at the most efficient price possible. The result has been a huge increase in advertising on the Internet; look at Meeker’s equivalent slide from 2015:

meeker2015

Advertisers’ strong preference for programmatic advertising is why it’s so problematic to only discuss publishers and users when it comes to the state of ad-supported web pages: if advertisers are only spending money — and a lot of it — on programmatic advertising, then it follows that the only way for publishers to make money is to use programmatic advertising.

The Modularization of Publishing

From publishers’ perspective, the fixed cost of a printing press not only provided a moat from competition, it also meant that publishers displayed ads on their terms. To use the Conservation of Attractive Profits model that I discussed last week, publishers were exceptionally profitable for having integrated content and ads in this way:

print

As the description of programmatic advertising should make clear, though, that is no longer the case. Ad spots are effectively black boxes from the publisher perspective, and direct windows to the user from the ad network’s perspective. This has both modularized content and moved ad networks closer to users:

internet

Here’s the simple truth: if you’re competing in a modular market, as today’s publishers are, profits are slim at best, and you generally take what you can get from a revenue perspective. To put it another way, publishers today have about as much bargaining power as do Uber drivers, and we’ve seen how that has gone.

So What Now?

To this point I’ve discussed ad networks from the advertisers’ perspective; Gruber’s critique, though, was that of a user: he is absolutely correct that the price of efficiency for advertisers is the user experience of the reader. The problem for publishers, though, is that dollars and cents — which come from advertisers — are a far more scarce resource than are page views, leaving publishers with a binary choice: provide a great user experience and go out of business, or muddle along with all of the baggage that relying on advertising networks entails.3

Again, the solution is not that publishers should try harder to have better ads. The New York Times, arguably the gold standard when it comes to both brand and quality impressions, noted in its annual report:

Digital advertising networks and exchanges, real-time bidding and other programmatic buying channels that allow advertisers to buy audiences at scale are also playing a more significant role in the advertising marketplace and causing downward pricing pressure.

If the New York Times cannot resist programmatic advertising, what chance does iMore or the vast majority of online publications have? If anything this puts Facebook’s Instant Articles initiative in a far more positive light: the social network is offering to not only improve the user experience by displaying articles instantly — thanks, primarily, to the lack of programmatic advertising cruft — but also to help monetize said content by selling ads against it and sharing 70%, backed by profile data that is far superior to even the ad networks.

Indeed, arguably the biggest takeaway should be that the chief objection to Facebook’s offer — that publishers are giving up their independence — is a red herring. Publishers are already slaves to the ad networks, and their primary decision at this point is which master — ad networks or Facebook — is preferable?

This too provides additional context to Apple’s new News app, which looks an awful lot like Facebook’s offer: publishers put their articles in a common repository that is monetized collectively through iAd, thus achieving advertising scale (and, it should be noted, more user data than Apple’s rhetoric would suggest). Apple, of course — and this is what prompted this entire discussion — is providing not only a carrot but a stick. Gruber concludes:

With Safari Content Blockers, Apple is poised to allow users to fight back. Apple has zeroed in on what we need: not a way to block ads per se, but a way to block obnoxious JavaScript code. A reckoning is coming.

It absolutely is, as I noted when Facebook’s Instant Articles launched. The future for most publishers is likely that of pure content production only, save for the few — like Gruber — who are destination sites capable of selling native advertising in stream (or selling subscriptions, like this site). What is very much in question is exactly how users will feel when they finally get what they claim they wish for.

  1. I excerpted most of Gruber’s post — his writing is impressively tight! — with permission []
  2. The experience of the vast majority of publishers is that readers will not pay for content; the exception are sites like this one that keep costs extremely low and focus on primarily analysis, not original reporting []
  3. Advertisers and ad networks, unfortunately, don’t really have an incentive to improve the user experience; there is an effectively unlimited amount of inventory on the web []