advertising information
   personal technology

 custom news
 Headline News brief
 daily almanac
 CNN networks
 CNN programs
 on-air transcripts
 news quiz

CNN Websites
 video on demand
 video archive
 audio on demand
 news email services
 free email accounts
 desktop headlines

 message boards




Shopping bots are back

May 14, 1999
Web posted at: 1:03 p.m. EDT (1703 GMT)

by Scott Kirsner


(IDG) -- Bots are back. After a slow start, a new generation of Web intelligent agents—tools that help buyers evaluate products from a diverse set of merchants—is emerging. That's great news for merchant sites that understand how to work with bots, bad news for those that blindly decide to block them.

  CIO home page
  Make your PC work harder with these tips
 Reviews & in-depth info at's personal news page's products pages
  Year 2000 World
  Questions about computers? Let's editors help you
  Subscribe to's free daily newsletters
  Search in 12 languages
 News Radio
 * CIO radio
 * Fusion audio primers
 * Computerworld Minute

"Shopping bots bring you eyeballs, and attracting eyeballs is the ultimate goal for any site," says Marcus Zillman, founder of Inc., an online guide to the world of intelligent agents. "My conservative estimate is that sometime in the next five years we'll see a shift, and the majority of shopping that happens on the Net will be facilitated by bots."

The first notable shopping bot, BargainFinder, was launched in 1995 by Andersen Consulting. BargainFinder let Web users compare the price of a compact disc across a handful of online merchants. But immediately it started getting doors slammed in its face. Several merchants refused to let BargainFinder query their databases for a specific CD's price.

That early experiment exposed the two key reasons merchants fear shopping bots. First, they can strip away the merchant's carefully constructed brand image; for example, users of BargainFinder didn't see the pages Music Boulevard had built; they began their shopping expedition at a site designed by Andersen Consulting. Second, the earliest bots enabled only basic price comparisons; there was no way to evaluate the service level, the merchant's reputation, the quality of the product or even whether the product was in stock and ready for shipping.

But the new-generation bots are more sophisticated, and the entrepreneurs who are developing them are more sensitive to merchants' needs. Second-generation bots such as those devised by Cambridge, Mass.-based Frictionless Commerce Inc. enable more nuanced comparison shopping, informing a user about variations in product warranties for example. Josh Goldman, president and CEO of Santa Clara, Calif.-based MySimon Inc., acknowledges that merchants had to be taken seriously. "We recognized a while ago that our key to success was responding to merchants' concerns. You can't be antagonistic. It's hardly a recipe for success if your bot focuses only on price, and people have the perception that you're out to destroy their brand."

Some merchants, like music retailer CDnow, still block the bots. But if Zillman is right—and I think he is—that strategy will prove to be unwise. As more and more highly qualified potential customers start their shopping trips at sites like MySimon, and Excite's Product Finder, their buying power will be hard to ignore.

"People have been comparison shopping for 10,000 years," says Charles Collins, director of corporate development at Microsurf Inc. in Norwood, Mass. Microsurf runs two highly successful comparison shopping sites, and "When you have merchants side by side in a bazaar in Tangiers, and a customer can walk from one stall to another and evaluate the merchants, that's not too different from what we have on the Net. Why run away from it? If you're running the kind of company where quality is job one, then the Net is the place for you. You just shine, and the customer picks the best of the best." So if you're smart enough not to shut them out, how can you exploit the new bots most effectively?

Be selective

There are dozens of bots out there. Some serve millions of potential buyers each week, some receive just a trickle of aimless visitors. Some have sterling reputations for providing objective information to the customer and being fair to merchants; others do not.

Cyberian Outpost Inc., one of the Net's pioneering merchants of computer hardware and software, follows a sensible path in dealing with bots. "If we choose to work with a bot, we send them our pricing information," says Daryl Peck, the founder, president and CEO of Cyberian Outpost. "We don't allow [unapproved bots] to get at our pricing." Peck's company cooperates with the bots,, Junglee (owned by, Excite's Product Finder, and MySimon.

Still, Peck is far from being an ardent fan of the first-generation bots. "We think they're all inherently flawed, but you have to work with them if you want them to get better," he says. His biggest criticism? The prices supplied to the bots can be deceptive. "It's too easy for someone to have the best price and then charge $40 for shipping. I'd like to see [bots] compare the delivered cost. What will it cost to get this thing at my door?"

Second-generation bots strive to answer those kinds of questions. "Going forward, most comparison engines will add more information about the products and the merchants," says Andy Halliday, vice president and general manager of Excite Inc.'s commerce division in Redwood City, Calif. "It would be interesting to score merchants based on when orders were placed and when they shipped. I'd be happy to pay more to the merchant who can ship it the next day rather than in 72 hours."

Some bot sites strive to offer a level playing field. At, for example, staffers ensure that none of the 1,300 lenders who participate in the site are engaging in bait-and-switch, offering a lowball rate that they won't be able to give in reality. "We have removed 44 lenders for that," says Collins. "We find out about it from consumers, other lenders or our own software that checks whether their rates are within a reasonable guideline of profitability."

That kind of strict policing keeps lenders happy. "It lets people compare apples with apples," says Robert Kang, president of Mortgage Discounters Inc. in Annandale, Va. Kang says that about 15 percent of customer inquiries to his company now come from "[Comparison shopping] is great for the consumer, but it means that businesses like ours have less room to be inefficient. You have to be fast, and you have to be creative, and you have to be service-oriented," says Kang.

Build your brand

The gnawing worry about the Net in general and shopping bots in particular is that they will undermine brands and focus all competition on price. So far that concern isn't proving true.

"Most people are wary of the absolute lowest rate," says Collins. "Our stats show that a competitive rate, not the lowest rate, is the one that gets the click-throughs. People wind up evaluating lenders based on lots of other criteria: location, style, rapport, ease of use." Mortgage seekers want a lender they can trust.

Savvy online merchants understand that building brand is less about manipulating mass perception of your product or service than about creating an enjoyable customer experience—at the purchase stage and beyond. Sites that offer that kind of experience will convert onetime buyers referred by bots into loyal customers; they'll build their brand equity, and suddenly consumers won't mind paying a small premium for a trouble-free shopping experience with a trusted merchant.

Compete smarter

Rob Guttman, chief technology officer at Frictionless Commerce, predicts that within a year or two shopping bots representing customers will negotiate with bots that represent your business. A customer seeking to buy 100 desk chairs directly from a furniture manufacturer could give the bot specifications for features, delivery dates and price range (even telling the bot either to be flexible or miserly in negotiations). The bot would offer a price based on those criteria—higher if the manufacturer could meet all of them, lower if perhaps the chairs couldn't be delivered within the customer's specified time frame. Guttman calls it simply, "the end of fixed pricing."

Boston-based writer Scott Kirsner also contributes to The Industry Standard, Wired and Fast Company. He can be reached at

Web shopping booming in Japan
May 4, 1999
Battle for the fridge
April 9, 1999

Branding tall: Is your business positioned to be top-of-mind in the age of the Web?
QuickStudy: Agent technology
There is intelligent life on the Web
Lego goes robotic
Want a robotic cat?
(PC World)

Note: Pages will open in a new browser window
External sites are not endorsed by CNN Interactive.

Frictionless Commerce Inc.

Note: Pages will open in a new browser window
External sites are not endorsed by CNN Interactive.

Enter keyword(s)   go    help

Back to the top   © 2001 Cable News Network. All Rights Reserved.
Terms under which this service is provided to you.
Read our privacy guidelines.