More Web sites turn to test tools
October 25, 1999
by Carol Sliwa
(IDG) -- On the Web, there are times when even the best-laid testing plans can't bring the desired result.
Encyclopaedia Britannica Inc. did load and performance testing before this week's launch of a free, advertising-supported Web site. KBKids.com LLC is furiously trying to check key applications before the Christmas season hits. Charles Schwab & Co.'s electronic brokerage division devotes 80 of its 300 information technologystaffers to quality assurance of its biweekly application releases. Each of these companies has turned to commercial testing tools to help them stress-test their sites in the face of intense time-to-market pressures.
Even so, Schwab and KBKids.com often can't find the time to take advantage of their tools. And a year's worth of testing probably couldn't have helped Chicago-based Encyclopaedia Britannica, which suffered a very public crash Wednesday.
A study by Newport Group Inc. in Barnstable, Mass., indicates that many companies do performance tests on their e-commerce applications before a launch -- about two-thirds of 117 companies surveyed earlier this year. Many didn't do tests and later found scalability problems.
Newport Group analyst Billie Shea said she sees an uptick in the adoption of commercial tools. "I've followed the tools market since '94, and back then, it was like pulling teeth to get people to recognize the benefits. Now people are starting to seek out tools more," Shea said.
Attendees at this week's user conference of one of the leading tool vendors, Mercury Interactive Corp., said the best they can hope for is lowering the odds that their sites will suffer performance problems.
"There are no panaceas," said Mickey Lutz, vice president of information management and technology architecture at PHH Vehicle Management Services. Yet now that the Hunt Valley, Md., company offers Web-based applications to help customers manage their vehicle fleets, it wouldn't consider cutting testing tools from its budget.
"This was not a debatable issue," Lutz said, noting that his company earmarks 7% to 15% of development costs to test engineering. Six years ago, the company didn't even have a quality-assurance department.
KBKids.com recently learned the hard way what cutting corners can do. The Denver-based online retailing arm of KB Toys was doing a database conversion from Microsoft Corp.'s SQL Server to Oracle Corp.'s Oracle8i at the same time it ran a promotion to give customers a $10 discount if they spent $30 or more in the Web store. But the code ported from SQL Server to Oracle8i contained a logic flaw, and customers got the $10 price cut with a 30-cent purchase.
"It was a result of pushing the system test cycle," said Shawn Davison, vice president of technology and operations at KBKids.com. Fortunately, the problem got fixed before more than 100 customers took advantage of the extra-special offer. But it highlighted the tightrope Davison must walk. "There is no ideal scenario in this business where you have the time to test everything that you'd like," said Davison.
Although some users complain that they don't have the time or money and that e-commerce testing tools are still immature and buggy, analysts warn that the cost of failure is so high that they can't afford to not make the investment. Leading offerings include products from Compuware Corp., Mercury Interactive, RadView Software Inc., Rational Software Corp., RSW Software Inc. and Segue Software Inc.
But even companies that make the tools investment are finding it difficult to carve out the time necessary to orchestrate the switch from their manual processes to automated testing. KBKids.com, for instance, estimated that 75% of its testing is done manually. Schwab, which experienced site problems this week with customers accessing their accounts via America Online or the Microsoft Network, wants to do more automated testing of its electronic brokerage applications. But the San Francisco brokerage's biweekly application release schedule makes it difficult for IT staffers to carve out time for setting up automation -- even though in the long run, that could speed some testing, said Hugh Westermeyer, a vice president at Schwab's electronic brokerage division.
Still, the nearly infinite possibilities of user behavior make testing an inexact science. "The problem with the nature of this technology is I don't know if you won't find the break points," Lutz said.
New technology makes searching easier
RELATED IDG.net STORIES:
Schwab's Web site troubled again
|Back to the top||
© 2001 Cable News Network. All Rights Reserved.|
Terms under which this service is provided to you.
Read our privacy guidelines.