THE FOLKS AT YAHOO!, Excite and Lycos don’t want you to notice the mechanics of their Web-search software. They want to be user-friendly media companies. ““Technology’’ is a word to be avoided.
But it’s a mantra at the Inktomi Corp., the brains behind the searching software of sites like Hotbot and Microsoft’s coming gateway, Start. Instead of surrounding its search engine with flashy features like free e-mail and chat rooms, the San Mateo, Calif.-based company licenses it to sites that want to provide a searching capability. Net watchers call it the fastest, most comprehensive way to scour the Web for information. And next week Inktomi will add another feather to its cap when it announces it has displaced competitor AltaVista as Yahoo’s search-engine partner, answering user queries when they’re beyond the scope of Yahoo’s human-generated directory. ““Our job,’’ says 25-year-old Inktomi cofounder Paul Gauthier, ““is to make sure search technology keeps getting better.''
Like Godzilla, Inktomi seems out to prove that size does matter. Its engine searches an index of 110 million Web pages, about half of the Web’s estimated total (Excite searches 55 million, Infoseek 30 million). Specialized Web-crawler software is constantly scouring the Internet, updating the index at about 10 million pages a day. That’s essentially how all search engines work, but what’s distinctive about Inktomi’s system is that it’s spread among 166 low-cost Sun workstations that sit in an air-conditioned cage in a Santa Clara data center. The computers are ““clustered,’’ working together to provide the semblance of a supercomputer’s power, while also allowing for a gradual, inexpensive expansion as the Internet grows. Inktomi has also used this clustering technology to create a line of traffic servers that alleviate Internet congestion. ““Instead of trying to build a really big hammer’’ to keep up with Internet growth, says Gauthier, ““we’ve come up with a more sophisticated solution.''
A final challenge will simply be making a profit. According to its filing, Inktomi lost $7.9 million between October and March. But the Yahoo scoop will clearly be a boost–and it validates Inktomi’s business strategy. Yahoo dropped AltaVista because it was adding features like free e-mail and travel services to become its competitor in the portal arena. Inktomi seemed comparatively benign. Ironically, ““Inktomi’’ is the name of a mythical Sioux Indian trickster spider that defeats larger opponents in battle. As AltaVista discovered, it bites.
