As Bing, Facebook, Twitter, and less well-known upstarts nip at its heels, Google has hundreds of wizards racing to come up with smarter answers
Google may outperform other search engines, it still spits out plenty of clunkers—irrelevant sites or even occasionally no sites at all for a particular query. They also know that every disappointing result means someone is less likely to click on ads—the source of nearly all of Google’s $22 billion in revenues last year—and more likely to try another search engine. “If it turns out that somebody offers a better service than we do,” Manber, a former academic and executive at Yahoo! (YHOO) and Amazon.com (AMZN), says with characteristic understatement, “that’s a concern.”
Today more than ever, Manber and the brainiacs in the search quality group can’t afford to falter. Google’s competition has recently gone from pitiful to plentiful: Microsoft’s (MSFT) new Bing search engine picked up 1.5 percentage points of market share in August to hit 9.5%, according to market researcher Hitwise, while Google’s share fell from 71.4% to 70.2%. Bing’s gain is partly thanks to a $100 million marketing blitz complete with television ads knocking Google every which way but in name. Microsoft’s pending deal for Bing to become Yahoo’s underlying search engine, creating a combined entity with 27% market share, could produce Google’s first sizable competitor in years.
As the Web evolves, there’s no guarantee that search, or Google, will remain at its center. Not so long ago, portals such as AOL and Yahoo dominated the Internet as most people’s first stop online. They lost ground as search improved and helped guide people quickly to sites they were looking for. Now Twitter and others are becoming significant drivers of people’s attention to Web sites—Google’s raison d’être. The New York Times (NYT) recently said that some 10% of NYTimes.com’s traffic was sent by Twitter. Some marketers sense in the rapid rise of Twitter and Facebook a new way other than search for people to find information, connect with friends, and get recommendations and ads for products to buy. Twitter’s ability to raise a round of venture money that gives the startup, which has no meaningful revenue yet, a valuation of $1 billion reflects the potential some people see.
Google clearly understands this emerging new Web, given its investments in such services as the video sharing site YouTube, online applications, and even an operating system. But the company hasn’t been able to generate significant revenue from anything besides search-related ads, raising doubts about how long it can remain the key leader of the Internet economy.
Meanwhile, Google’s very success and size are starting to work against it. In the past year the company has been the target of three U.S. antitrust inquiries and one in Italy. Most recently the Justice Dept. on Sept. 18 said Google’s controversial settlement with authors and publishers, which would have allowed it to scan and sell certain books, must be changed to avoid breaking antitrust laws. Even Google’s own paying customers—advertisers and ad agencies—say they’re eager for alternatives to blunt Google’s power. Says Roger Barnette, president of search marketing firm SearchIgnite: “People want a No. 2 that has heft and scale.”
The group’s home is in Building 43 in the center of the sprawling Googleplex in Mountain View. Deliberately casual, the first floor features such Googley touches as a refrigerator case with free Odwalla juices and huge panels of whiteboards scrawled with product ideas and jokes (“Why are they called apartments if they’re stuck together?”). The company’s trademark primary colors are splashed everywhere, down to the bulletin-board pushpins. A huge pile of Legos is scattered at the top of the stairs for engineers who need a break from coding software. In a hallway, someone snoozes in a MetroNaps EnergyPod, a semiprivate chair module for catnaps. Much of the second floor, which houses many members of the search quality team, resembles a well-used playroom for a large family of overgrown kids, which is more or less what it is.
The team is composed of several hundred engineers, many from outside the U.S. and some with 20 years of experience in search and information retrieval technologies that predate the Web. In Google’s version of Silicon Valley egalitarianism, most people, including executives, share cramped glass offices. They’re crowded with multiple large computer monitors, along with whiteboards and yellow sticky notes covered with formulas and notes on arcane aspects of search technology. The office of Matt Cutts, head of the anti-spam unit, and four others bears an inscription above the door: “What could possibly go wrong?”
It’s a question Manber and his team contemplate every day. Google can claim a number of search breakthroughs in its 11 years, starting with the co-founders’ breakthrough PageRank formula, which gives higher ratings to pages many other sites link to, because they’re likely more useful than pages with few links. But it’s clear these guys—and a few women—view doing search well as something as difficult as curing cancer. “There’s just an endless supply of very difficult, challenging, but incredibly interesting problems to work on,” says Scott Huffman, who runs the search evaluation unit and mobile search.
If Google’s corporate mission is to “organize the world’s information and make it universally accessible and useful,” Manber articulates a similarly expansive vision of search. He notes that while the 20th century was all about conquering nature, the 21st century will be about understanding people—not just what they say they’re looking for, but inferring what they mean from the most minute behavioral clues. “Search is a big part of it, possibly leading the way,” says Manber, an intense but soft-spoken man who plays 10 different musical instruments strictly for his own enjoyment. “Our job is to do rocket science that will be taken for granted.”
The group’s work often begins with a complaint. Google users can flag a bum result by clicking a link at the bottom of results. In some cases, data collected from a standard set of sample queries that Google’s computers are constantly running may indicate people aren’t clicking on some results as much or in the same way as they did historically. Most ideas, though, come from Google engineers with an inspiration. At the Tuesday morning meeting of search leaders, briefings from colleagues all around Google may spark a new idea to improve a set of searches. Not least, engineers just keep their eyes open. While he browses local art shows, Cutts scribbles obscure artists’ Web sites in a small notebook so he can check later on how well they show up in a search—or don’t.
Whatever the source, “broken” queries and other proposals for improvements land in the e-mail box of Amit Singhal. The genial, 41-year-old Google Fellow (the company title for its most accomplished engineers) heads the core ranking team. Its job is to provide the most relevant links to pages, videos, and other information for every query and adjust the algorithms when results don’t seem to match the queries. Making improvements follows a regular regime. Engineers on the team can essentially try out proposed changes on one of Google’s copies of the entire Web, stashed on the company’s massive network of computers. Once they think they have a fix, they send the proposal to the evaluation unit, “eval” for short. Those changes can affect many other queries in unexpected ways, so it’s up to eval to run more rigorous tests to make sure a change is positive overall and doesn’t hurt other results too much.
To an outsider, many of the changes may look impossibly trivial. Several years ago, for instance, engineers noticed that while Google was returning useful pages when someone typed an acronym such as “CIA”—providing links to the government agency and to the Culinary Institute of America—people were taking a slightly longer time than expected to click on one of them. So on the results pages, Google began highlighting in bold the full names. Immediately, Google saw more clicks through to pages—and faster, too. How much faster? Perhaps 30 or 40 thousandths of a second, on average, Singhal says. That’s one tenth the speed of an eyeblink. “This was a small idea,” concedes Singhal. “But we have a real responsibility as a company to respect people’s time.”
More recently, Google has been focusing on providing results more relevant to specific regions around the world. Engineers realized that people in India searching for “bank” didn’t much care about Bank of America (BAC), even if it was in the news. So Google has been tweaking algorithms to emphasize the searcher’s apparent location. Now, a search on “bank” on the U.S. site, google.com, will bring up links to Bank of America and Wells Fargo (WFC), while the same search on google.co.in, Google’s Indian site, will bring up homegrown HDFC Bank (HDB) and ICICI Bank (IBN). It’s one small reason Google has even higher market share in many other countries than in the U.S.—such as 88% in India, according to comScore (SCOR).
The reason Google has become such a powerhouse isn’t just because its search technology is exceptional, of course. It’s because the company perfected a way to match advertisements to its search results. Hundreds of thousands of advertisers in its AdWords program bid in an online auction to buy “keywords,” or terms likely to be in search queries, that they hope will trigger their ads to run on the right side or the top of the search results page. It’s a good bet that people searching on, say, “sony cybershot” are in the market to buy a digital camera, and retailers that buy that term are more likely to attract clicks and sales. Spending on search advertising has soared in recent years because advertisers tend to get strong returns for their money.
How well are all the search improvements underlying those lucrative ads working? While no one knows for sure whose search results are best, a number of independent experts continue to give Google the nod over its rivals. And they think the current crop of would-be disrupters isn’t going to beat Google at its own game anytime soon. “None of these are big challenges for Google,” says search expert Sullivan. “I think Google is still better in quality.” [BusinessWeek]
Post information: