Virulent Word of Mouse

July 11, 2011

Did one invention lead to the decline of newspapers?

Filed under: Considering topical questions,Internet economics,Short observations — Shane Greenstein @ 7:50 pm

Did one invention lead to the decline of newspapers? What is economic myth and what is true?

Don’t get me wrong. The decline of newspapers is NOT an economic myth. The business continues to lose more revenue each year, and so does other advertising-supported media, such as magazines. Much of this happened in the last decade. It is  unclear if executives at any newspaper have any good strategic choices. None of this is a secret.

But one invention and one firm did not produce this outcome. The typical story blames Google and Internet search, and does so a little too blithely. There is a grain of truth in the common story, but it misses a lot. While it is correct that the firm to profit the most from this trend is Google, Google alone did not kill the newspaper.

This post is not an apology for Google. Rather, it presents a broader economic history than typically found in public discussion. As more issues get debated in public it important not to premise decisions on a distorted view of how we got here. 

More than one invention

What is killing newspapers? It resulted from the accumulation of several inventions and several firms.

First of all, let’s get something out of the way. News papers are suffering from two problems, loss of readers and loss of advertisers. The former can cause the latter, and has to some extent, but loss of advertising can happen for other reasons as well, and did.

Craigslist is probably more responsible for the decline in revenue than any other factor. Their ads are much cheaper. Lots of people use them. Craigslist had a direct and significant effect on ad prices in newspapers. If you are curious, there is some recent good research documenting the effect Craigslist had on newspapers by Rob Seamens and Feng Zhu.

By the way, so did Monster.com and all the other sites for job seekers. Classified ads for jobs used to be a big part of what newspapers provided.

There is a lot I could say about loss of classified ads others than through Craigslist, but since is the short version of this story, I will not. Just remember that classified ads used to be an easy source of revenue and are not anymore.

Now that that is out of the way, let’s focus on other topics, specifically, on ad-supported search on the web. That has taken BOTH readers and ads away from newspapers.

Ad-supported search is easy to misunderstand. Ad-supported-search arose as a result of several trends, and these should not be conflated.

That gets us to Google. As is well known, Google sells ads by auction, where the “auction” is really a bidding game between advertisers for keywords. Advertisers can scan lists of keywords and bid to have ads placed next to a search when a user enters the most common keywords.

Where did this market come from? Let’s review the highlights.

Google was founded by a couple PhD drop outs from Stanford. Larry Page and Sergey Brin met in grad school, experimented with a search engine using page-rank and started a business, eventually hiring Erik Schmidt as CEO.

Here is a relevant fact that is often forgotten. Were Page and Brin trying to compete with newspapers at first? No, at first they were selling a search service to others.

Look, in the late 1990s most of the mass market firms thought search was not very valuable.  Most of the computer scientists thought search was valuable (and still do).

As of the late 1990s the computer scientists were losing this argument because users did not use search engines very much. Users went to sites like Yahoo! and walled-gardens like AOL, who organized the online experience for the mass market. Search was for nerds, not the mass market.

Frankly, the computer scientists were losing this argument for a while in the most recent decade too. Most people forget that AOL had its peak year in 2003 and did not switch to being an advertising supported business until late 2006, which is about the same time that Google began to surpass Yahoo in popularity.

<digression> Here is a digression on my main point. Google is really a recent phenomenon by mass market standards. Only technologists think of Google as an old company because technologists are early adopters of technical services, almost by definition. Moreover, most technologists never liked AOL, and did not give it much credit for bringing less technically skilled readers to the Internet. <end of digression>

Back to the main point, Larry and Sergey were just trying to sell search services in 1998. That was it. And they had a hard time finding any buyers. Eventually Yahoo became a customer, but let’s not exaggerate. Yahoo signed a contract to have Google do search for them because Yahoo’s executives believed — just like everyone else — that search was not of strategic importance for their business. It was so unimportant that Yahoo could  outsource the service.

Why did Yahoo go to Google over others? The common answer is “page rank,” because it worked better at finding relevant answers. Another importance facet was Google’s speed. User’s do not like to wait. Most research shows that online users are super-impatient. Google got those two things right.

<digression> Yes, there were other design features that were appealing too. But this is the short version of the story, so let’s move on. <end of digression>

Let’s consider for a moment why Larry and Sergey went to Stanford to get graduate degrees.  Both were interested in information theory and practice. Their advisers were too. Their advisers got their money came from the National Science Foundation (NSF), a federal agency supporting research. Indeed, NSF had helped grow the Internet backbone and many other facets of the computer science that grew into the Internet, so NSF views itself (rightly) as one of the parents of the Internet.

Did the NSF want to start a new firm like Google? Of course not. NSF was funding initiatives to improve information retrieval and storage, a long standing topic in computer science. But for years such efforts had made only incremental progress because everybody focused on information retrieval in libraries. When the web started to explode, these initiatives were taken up again, and reconsidered, and re-imagined for the web. NSF actually did its job, recognizing a new opportunity, moving funding to it, and seeding research.

Did NSF know what would come from it? No, that is not the way federal funding works at the NSF, thankfully. NSF gives its scientists considerable discretion to apply the research as the scientists sees fit, so long as it stays within the domain of the topic outlined when the research was funded.

Let me say it this way. The page rank algorithm that Larry Page and Sergey Brin employed for their first search engines was standard mathematics by the late 1990s, invented many years earlier. They recognized its application to the web. That was their initial novelty. NSF could not have foreseen that, but NSF should get credit for funding general research in this area, which had enormous economic potential. The payoff to society in this case far exceeded the money spent.

Selling ads

Notice that this story has said nothing about selling ads. That omission reflects historical facts. Larry and Sergey were not selling ads at the beginning. That came later.

How did it come about? Like many marketplace innovations, the initial inventors of ad-based search got it wrong. Imitators profited.

Specifically, Google did not invent keyword bidding. Goto.com (later called Overture, and much later bought by Yahoo!) pioneered the idea of a word auction, and led the category for years. Google imitated Goto.com at first, them made it better.

Goto was a venture backed business, a product of Silicon Valley’s system for funding good ideas. Today not many people remember Goto.com except those who lost money on the investment, and the VCs who were very frustrated with the management. In Valley-Speak the local lore today refers to Goto as just another firm who had the right idea and at the right time but did not execute on it.

We know in retrospect that Overture got imitated by Google and eventually lost out in head to head competition. Why? Overture faced three problems: Bidders gamed the pricing of the first-price auction; porn sites found ways to outbid others when users did not want to see their ads; and advertisers did not like paying for impressions because they could not measure the success of their ads.

Despite those issues, Overture actually made a lot of money at first. The porn sites spent a lot of money on ads, after all.

Google came up with three key innovations, one was the use of second price auctions. This stopped a lot of gaming of pricing by bidders. This was such a great innovation that Overture soon imitated it. Notice that there is nothing technical about this invention. Rather, it is an adaptation of processes.

The second innovation had a similar spirit. Google paid advertisers for user clicks, not impressions. That gave advertisers the ability to measure what business their ads generated. There is some technical work in that, but not much. This came about because Google was trying to serve advertisers better. Overture imitated this one too.

The third innovation was the most clever, a ranking of the “quality” of the web site. Such ranking effectively devalued the ads coming from the porn sites <digression> which made Google MUCH better for most users — just ask anybody with kids <end of digression>.

That quality ranking was both a technical invention and good business.  As business, it did not let the money from advertisers get in the way of the the quality of the search for users. Google put the users first and let the advertisers follow users. Eventually it also gave web sites incentives to improve what they did, and get higher quality ratings.

<digression> To my economist friends…. an economist might be tempted to say: Any good two-sided platform model would tell you that… Well, no, that is historically wrong. Some of the great work on the economics of two sided markets came after Google’s success, and did not drive it. Theoretical economists have been the followers of events here, mostly. The true originality has been in the market place. <end of digression.>

Overture never matched the quality ranking. That is because Google actually had to do some very difficult inventing. The computer science of an automated quality ranking algorithm for the entire web is just plain hard. Real hard. Eventually Overture lost out.

All this took place in the context of competition between search engines.  I am merely telling the short version of this story and leaving out all the inventions that turned out not to work.

Notice, as well, that the quality ranking is a part of Google’s secret sauce. It is unsurprising that they hesitate to  be entirely forthcoming about how it works. Google claims web sites would game a transparent process, so a little murkiness is appropriate. Even if all that is true, it is also not surprising that some web sites are suspicious of Google. They have to live with this quality ranking, take it or leave it.

As a digression on that digression, if you really want to know more then watch Hal’s Varian’s explanation of how the auction works. As part of that, he explains a bit about how the quality ranking works. (Be forewarned: It is ten minutes long).

The entire ecosystem

A little later, Google invented one more thing, placing ads in designated places inside blogs, usually in little rectangle windows. A site can arrange this with Google, and Google splits the money from the ads with the site.

That invention is clever. Once again, it combines a bit of technical cleverness with a bit of imaginative energy. The computer science was not easy, but this should be understood as more than merely a technical invention. It involved a lot of entrepreneurial imagination and it involved building an organization behind it.

It also was important. It was responsible for fueling a big part of the boom in sites like Huffington Post, and others who have a little space to rent next to their blog posts.

That started to happen in 2003 and by the middle of the decade it had spread madly. “Ads by Google” became a common statement all over the web. Many little and big sites began using Google’s ad services, paying only for clicks. It was convenient, especially for smaller sites, generating enough revenue to support an inexpensive site.

It changed the ecosystem of the web. The long tail of ad-supported web began to depend on it. Many niches sites expanded accordingly. Once again, this is a long story, and I am merely giving the short version of it. Just suffice to say, however, this was another source of substitutes for news, and it drew readers away from newspapers.

What led to the decline of news?

Look, this post has a point. By now it should be obvious: One invention did not lead to the decline of newspapers, and one firm did not do it either. The loss of readers and the loss of ads came from the accumulation of a number of events.

Who is responsible? Let’s count. We have blamed Craigslist, other online classified sites, Overture, NSF funding, Silicon Valley’s ecosystem, the efforts of many clever computer scientists, and the efforts of many bloggers.

And that is just the short version of the story. The problems with newspapers did not arise at the hand of a single invention or a single firm. It was a gang.

12 Comments »

  1. This history doesn’t go back far enough in time. ;-) You know, to talk about radio and TV!

    I’ve been writing about the economics of newspapering for a while, given the high profile conversion of the Seattle PI from a print to an online daily. My most recent missive — http://wiredpen.com/2011/02/28/online-revenue-cannot-rescue-newspaper-biz-model/ — reminds the reader that newspapers reaped excess profits due in no small measure to the fact that the marginal cost of printing another copy of a newspaper is pennies (ink and paper). The big cost is all wrapped up in that first issue off the press. This economic reality led to market concentration and businesses happy to pay a small fraction of that cost (advertising) to reach all those eyeballs.

    Those days are gone, gone to digital competition.

    Unlike you, I believe that one invention — the Internet — combined with the predictable reaction that (most) incumbents have to innovation made the economic downfall of the current newspaper system inevitable. Local TV is feeling the same pressures. The “record” industry went first. Digital technologies make it easy and economical to atomize content — no need for an LP for distribution, no need for the entire paper for distribution, no need to watch the entire news cast.

    Comment by Kathy E. Gill — July 12, 2011 @ 3:13 am | Reply

    • Thats a good point and thanks for the link!

      Comment by fresno dan — July 12, 2011 @ 4:22 am | Reply

    • Thanks for the comment and link. I see your point about incumbents, and that is interesting. I will read your posts. You are right that I did not go back far in time, but I believe the result would stress similar themes. The Internet and the Web also are not merely one invention, but the accumulation of many. Cheers! Shane

      ________________________________________

      Comment by Shane Greenstein — July 12, 2011 @ 9:30 am | Reply

  2. Shane, what does this have to do with the decline of the newspapers? You thesis seems to be (i) Google grabbed attention; (ii) Google worked out how to sell ads … (iii) newspapers destroyed. What is the link between (ii) and (iii)? I think you have some idea that the advertisers who were on Google are the same advertisers who were in the traditional newspapers. This is surely not the case as study after study has shown.

    The problem is that the price of ads online for newspapers are a fraction of what they are for ads in print. The print ad prices haven’t really declined although price ad revenue has as circulation has dropped. The problem is people migrated from print to online where ad revenues were lower. And this is not because the advertisers have moved to Google or use it as an outside option. The ads are different. Display ads are different from keyword ads.

    The problem is more likely to be that Google has facilitated switching and that this has disrupted the market for advertising — making it less efficient.

    Comment by joshgans — July 12, 2011 @ 6:24 am | Reply

    • Thanks for the comment. I see your questions. I am sorry I was not clear. I wanted to point out that (i) the growth of online readership and (ii) the growth of ads online were distinct, and the former took readers away from newspapers and magazines, which lowers subscriptions. I did not mean to argue that (ii) alone took away all the ads. Classifieds moved for other reasons, after all. But I did mean to imply Adsense did help many blogs and other forms of online sites, which contributed to taking away more readers. And, yes, I did mean to imply that (ii) took away ads from print. Automobile ads. Vacation ads. Baby toy ads. Movie theatres. Home improvement. Some adwords, some banner. Yes, it is more than Google, you are right, and I am sorry I did not say that. But a fraction of the ad budgets for major advertisers is moving online. Surely that took away from the ad revenue going to newspapers. Google, to its credit, has profited from that trend. Keep up the great comments! It is important to have these discussion in public. Cheers! Shane

      Comment by Shane Greenstein — July 12, 2011 @ 9:42 am | Reply

  3. All this work and hypothesizing and you completely missed the single biggest factor that hastened newspapers’ decline…the do-not-call list.

    Comment by Blahblahblah — July 12, 2011 @ 8:14 am | Reply

    • Thanks for the comment. I am sorry, but I missed the connection. Yes, half the US is on the do-not-call list. How did that hasten the decline of the newspaper business?

      Comment by Shane Greenstein — July 12, 2011 @ 9:46 am | Reply

  4. Nice explanation of the process. In very basic terms it’s all about scalability. This digital evolution made information more scalable. Next the mechanical side of the equation will make it even more scalable as miniaturization and even nanotechnology affect the process.

    And the downside? Concentration probably. Scalability increases concentration. More and more concentration, not unlike that which created the financial collapse of three years ago.

    Comment by beezer — July 12, 2011 @ 8:29 am | Reply

  5. I think you’ve missed the point entirely. The news business hasn’t died; the newspaper business has died. (It’s rather the same with the music industry, by the way.) The news business has been dramatically altered by a technological shift — an essentially costless method of distribution. That monumental upheaval has dramatically lowered barriers to entry, resulting in an explosion of providers. Old economies of scale no longer apply. Now the news industry, with its nascent structure, is struggling with two interrelated problems: (1) fractionalization and (2) monetization. With a greater number of providers, audiences for any given outlet are naturally smaller. Advertisers pay for eyeballs; fewer eyeballs, fewer dollars. Meanwhile, advertisers are figuring out anew how to reach their audiences and what they’re willing to pay.
    As with the music industry, we are now watching as providers and customers wrestle with the question of how to value content rather than the medium. Such dramatically lower revenues (by firm) are presumptively insufficient to support the kind of on-the-ground first-person reporting we were accustomed to for many decades. It will be fascinating to watch in the coming years whether and to what extent everyday readers will be willing to pay for first-person reporting. Will they even recognize the difference? (The quality of current reporting suggests not.) Will news outlets find a way to distribute the costs of those kinds of resources, compensating for the fractionalization of readership, or will the solo practitioner with a blog be able to monetize his or her expertise with no structure whatsoever? We shall see.

    Comment by petery — July 12, 2011 @ 8:47 am | Reply

    • Thanks for the comment. Many interesting points. ________________________________________

      Comment by Shane Greenstein — July 12, 2011 @ 10:11 am | Reply

  6. Very intersting post. After reading it my understanding is that both that there were several events that led to the decline of the news AND, perhaps most important, many of these events were incremental innovations and/or unintended consequences of other inventions, which leave me wondering about the role of entrepreneurship and Schumpeterian innovation in this case.

    Comment by Roberto Fontana — July 12, 2011 @ 10:09 am | Reply

    • Thanks for your comment. Yes, accumulation of many intended and unintended consequences. Entrepreneurs does play an interesting role — partly as source of new ideas, source of new competitors, and more. It is interesting to ask about these roles in the context of the accumulation. Great question. Shane ________________________________________

      Comment by Shane Greenstein — July 12, 2011 @ 10:14 am | Reply


RSS feed for comments on this post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 60 other followers

%d bloggers like this: