Today, Google is a go-to source for news and current events, whether searching for information about an international crisis or the score of your local football game. Finding information on current events is so simple that we often take it for granted – but that wasn’t always the case.
When the terrorist attacks struck September 11th, 2001, it became readily apparent that Google was ill-equipped to serve their users with news information in their urgent time of need. Prior to 9/11, Google had no infrastructure or plan in place for delivering news and current events.
This tragedy would spark the innovation of Google News, a platform that has completely changed the way we discover news online.
“Our Search is Failing You…”
As the events and aftermath of September 11th unfolded, Google was bombarded with queries by people desperately hunting for information. As users searched time-sensitive queries like “World Trade Center” and “Terrorist Attack New York 2001”, they found nothing relevant.
Google’s Amit Singhal acknowledges the problem in a video titled “The Evolution of Search”:
“When September 11th happened, we as Google were failing our users. Our users were searching for ‘New York Twin Towers,’ and our results had nothing relevant, nothing related to the sad events of the day. Because our index was crawled a month earlier, and of course there was no news in that index. So we placed links to all the news organizations like CNN right on our front page saying please visit those sites to get the news, because our search is failing you.”
Minutes after the attacks, there were 6,000 queries per minute from users searching for CNN and a 60-fold increase in searches for news-related content. More than a missed opportunity, this was an information crisis that needed to be solved.
A Broken System… and an Idea
Recognizing their failure and noting that several major news websites were experiencing heavy loads or outages, Google went so far as to cache copies of popular news websites and recommend users check the radio and television for the most recent news:
The day after the attacks, Google continued to provide links to major news resources from their home page, as well as their condolences. It wasn’t elegant, and the news websites continued to struggle under the load.
Meanwhile, at a conference, Amit Singhal and Krishna Bharat began brainstorming as to how they could better serve current, relevant news to users in light of Google’s failure.
“My friend Krishna and I were attending a conference at the time, and Krishna started thinking about the problem, saying, ‘If we could crawl news quickly, and we can provide multiple points of view about the same story to our users, wouldn’t it be amazing?’ That was the birth of Google News.”
Krishna would pioneer the platform, originally creating a system for personal use that scraped just 20 major news websites and slowly grew it to over 200. It has since evolved significantly, pulling in news from thousands of news sources worldwide.
It was a big hit in the Google offices, and plans were made to develop it into a comprehensive platform.
The Challenges of News Aggregation
In developing Google News, Krishna faced several key challenges: How could an algorithm tell what counted as news? How could it serve up related news stories, even when those stories had different opinions, language and phrasing? How could Google crawl news websites frequently enough to display results in real-time?
On top of those considerations were others; Krishna and Google were unswervingly committed to presenting a broad array of opinions and perspectives on the news to avoid presenting a media bias or being accused of trying to sway the public’s opinion.
Their solution included a platform that doesn’t rank news sources by opinion or political leaning, but factors in location, freshness, relevance, diversity and more.
Since its launch in the spring of 2002, Google News has taken flack over what it counts as news and how it aggregates those news feeds, but one thing is for sure: when you search for a current event today – whether weeks later or moments after it has first been reported, you’ll find what you’re looking for.