ALSO: Search engine upgrades its relevance

By Robin Lloyd
CNN Interactive Senior Writer

August 4, 1999
Web posted at: 5:55 p.m. EDT (2155 GMT)

(CNN) -- A recently announced upgrade to the Excite search engine which promises soon to scour the entire Web and return more relevant information is not a guaranteed formula for success, critics say.

The upgrade, to be released within the next two weeks, is designed to bring users closer to all 800 million or so pages on the Web, with five times more indexing than before and a more thorough method of sifting for matches.

Brad Hill, author of 10 books about the Internet and personal technology, said Excite's efforts at increased relevancy are a good idea but massive indexing may not be the answer.

"Time was, all the search engines competed for the bragging rights to the most complete index. As the World Wide Web developed, it quickly became apparent that massive indexing of Web pages is irrelevant, wasteful of resources, unhelpful to the user, and detrimental to clean search results," Hill said.

Greg Memo, vice president of AltaVista, considered the search engine leader by Web aficionados, also was skeptical of Excite's potential for improvement.

"Obviously, Excite is way behind in terms of indexing the Web and around issues of relevance. They have realized this is an important capability they need to bring to their site," Memo said.

According to Memo, AltaVista has indexed 90 percent of the Web's some 800 million pages and will complete that project in the next year.

Excite says it will take a different approach from AltaVista, visiting all the Web's pages in coming years but indexing only 250 million, said spokeswoman Kris Carpenter. Comprehensive indexing can be a disservice to users, she said.

However, AltaVista's claim and Excite's run directly counter to a study published three weeks ago by the journal Nature. According to the study, search engine coverage of the Web has decreased since December 1997, with no search engine indexing more than 16 percent of the Web's indexable sites. The research, by NEC Research Institute, offered fresh bragging rights to those Web destinations with search engines that ranked highest.

"We don't think that is necessarily valuable to the consumer," Carpenter said. They don't want an index of 800 million web pages. That is not useful for a user. They do want a way to have access to high quality content."

Excite@Home, the new venture that produced Excite's upgrade, says it will provide consumers more quality information than they have been able to receive to date. Excite@Home was the result of a merger between Excite and the @Home broadband network.

Danny Sullivan, editor of the Search Engine Watch newsletter, was optimistic about the upgrade.

"For anybody using Excite right now it should be a huge improvement because one of the problems with Excite was it's a relatively small database and that has meant that there have been good sites that just get dumped," Sullivan said.

The number of pages on the World Wide Web is growing and is expected to exceed 1 billion early next century. The new Excite@Home service is scalable and designed to keep up with this growth, the company said.

Competition is intense among search engines

Excite's popularity is second to that of Yahoo!, excluding default search engines installed on Netscape and Microsoft browsers, Sullivan said.

However, AltaVista and Northern Light have strong reputations among more professional Web searchers.

Lycos also recently announced an expansion to the scope of its search engine.

While broader searches may clutter some Internet users with needless information, the services insist they are working harder to organize search results so that top-listed sites match what people are looking for.

Also, broader searches are more likely to benefit those who conduct in-depth research, such as scientists and journalists, rather than casual Web users just looking to buy books or CDs.

Better search power is a good way to draw more visitors. Twice as many people have been using Northern Light, the top-ranked search engine in the Nature study, since the report was published, said Susan Stearns, director of marketing for Northern Light Technology LLC, based in Cambridge, Massachusetts.

"People recognize that comprehensiveness is important," Stearns said.

But while offering more comprehensive searches is one way to lure more visitors to a site, it's no guarantee. The most visited network of Web sites, Yahoo!, ranked only No. 8 in the study.

Both Yahoo! and Lycos failed to return calls asking for comment.

The Associated Press contributed to this report.

Note: Pages will open in a new browser window

External sites are not endorsed by CNN Interactive.