November 30, 1999 Web posted at: 9:02 a.m. EST (1402 GMT) by Tim Greene

From...

MADISON, WIS. (IDG) -- The state of Wisconsin last week installed its 1,385th T-1 access line to BadgerNet, closing in on the end of a three-year effort to connect state agencies, schools and municipalities to the new, cost-effective statewide ATM-over-SONET backbone.

The sprawling private network reaches even the smallest Wisconsin towns -- potentially 5,500 sites in all -- and in many cases costs agencies and schools a third of what they would pay carriers for comparable connections.

Surprisingly, though, voice has been left out of this massive network undertaking. The reason: AT&T is shouldering that load on a virtual voice network and charging a mere 1.8 cents per minute.

"That low a rate means any form of convergence just isn't worth considering," says Loren Lamphear, BadgerNet project manager in the Bureau of Tele-communications Management.

Virtually all other state traffic, however, now rides BadgerNet in one form or another.

BadgerNet grew out of a 10-year-old AT&T network used to connect remote agency offices to mainframes in Madison, the state capital. As these networks became routed and band-width needs increased, the state started looking for a more flexible alternative.

In 1994, the state put out a bid to provide frame relay access to a backbone of 30 T-1s: Ameritech eventually won the bid. The frame relay traffic had to be handed off to AT&T for long-haul runs because regulations prohibit Ameritech from providing inter-LATA service.

But it wasn't long before the state became convinced it could install a private backbone that would handle long-distance traffic for less than AT&T was charging. The state also wanted to build in some redundancy, so it issued a request for proposal for a redundant 155M bit/sec OC-3 backbone, preferably SONET-based.

AT&T, Ameritech and newcomer Norlight bid on the project. Norlight, an amalgam of electric companies that was constructing an OC-48 SONET ring on power pole rights-of-way throughout the state, ultimately won the project by promising to deliver the OC-3 backbone for $29,000 per month. Although the state only contracted for the OC-3, Norlight put aside an entire 622M bit/sec OC-12 for state use.

In July 1996, while the bid was still out, the state realized it needed a full-time coordinator for the project and hired Lamphear, who in turn brought in Marcy Egges, an ATM expert; Christopher Alberts, a network designer for Ameritech; and other staff as the project evolved.

Today BadgerNet links all parts of the state to Madison via the Nortel-based SONET ring fed by Cisco ATM switches.

MORE COMPUTING INTELLIGENCE

IDG.net home page

Network World Fusion home page

Free Network World Fusion newsletters

Inexpensive T-1 service on tap from start-up, 09/06/99

MCI WorldCom, Sprint tout T-1 deals, 11/29/99

Reviews & in-depth info at IDG.net

E-BusinessWorld

Year 2000 World

Questions about computers? Let IDG.net's editors help you

Subscribe to IDG.net's free daily newsletter for network experts

Search IDG.net in 12 languages

News Radio

Fusion audio primers

Computerworld Minute

Access RFP

Although network designers wanted ATM as the switching technology for the backbone, they rejected the option of running ATM out to remote sites because agencies and schools were already familiar and comfortable with frame technology. Furthermore, the designers didn't want the 15% to 20% ATM overhead to cut into throughput on thinner access links. That overhead could have forced some sites into needing more than a single T-1, Lamphear said.

So the state put out an access RFP telling carriers it wanted T-1 frame access links delivered to its ATM backbone via channelized DS-3 pipes, and wanted to pay for the T-1 links on a postalized rate basis. Postalized means the state pays a flat fee, including for the DS-3s, per T-1 link regardless of circuit length, making it easier on bookkeeping.

Mixing individual T-1s into channelized DS-3 trunks meant the state could avoid the need for banks upon banks of DSU/CSUs at access nodes for each T-1.

In the end, the Wisconsin BadgerNet Access Alliance won the access bid. WBAA is a cooperative of Ameritech, GTE and a confederation of 54 independent local phone companies. It proposed aggregating all traffic at 30 nodes and offered a postalized rate of $355 per month per T-1.

The $355 represents just the cost for the access facilities, not what the state charges agencies for use of BadgerNet. After rolling up the charges for the use of a router, router management, backbone transport and access, the total bill for T-1 access to BadgerNet is $900 per month.

According to Alberts, a similar carrier connection from, say, Superior to Madison, would cost about $3,000 per month. "We don't care where you're located, it's $900 per month," Lamphear says. "You can't do it for less if you've got an area the size of Wisconsin."

Once the access links started going in, things moved fast, Alberts says. Three state employees and an outside vendor were installing the routers at a frenetic pace, the peak being 40 routers in a week. At the height of the project, BadgerNet filed 91 requests for T-1s in one day. "We stretched the limits of the telcos," Lamphear says.

In the year from October 1998 to October 1999, some 894 T-1s were installed, and the network now boasts a total of 1,385 T-1 access trunks.

These access lines feed into state access nodes. At the nodes, Cisco MGX frame relay concentrators pass the frame traffic into Cisco BPX ATM switches. The BPXs in turn convert the frame to ATM. Nortel S/DMS Transport Node SONET add-drop multiplexers pick up the traffic from the BPXs via OC-12 fiber links and drop it on the SONET ring. At these same nodes, Cisco Lightstream 1010 ATM switches drop traffic from university campuses onto the backbone through the BPXs.

All of the BadgerNet traffic terminates in Madison and is distributed throughout the city to state agency buildings via the Madison metropolitan area network, or MADMAN. MADMAN is anchored by four Cisco Lightstream 1010 ATM switches and many of the intracity links are on state-owned fiber.

With all of that bandwidth in the backbone, one would think voice would have fit rather neatly into the BadgerNet picture, and at one point the idea was entertained. The state issued an RFP for long- distance voice, and said bidders could propose solutions that included BadgerNet.

AT&T won with a Software Defined Network that costs the state 1.8 cents per minute for on-net calls. For off-net in-state calls, the rate is 5 cents per minute, and 6 cents per minute for interstate calls. Calls to 800 numbers cost 5.5 cents per minute.

Lamphear expects that long-distance carriers will continue reducing rates in order to keep large customers from migrating voice traffic to data networks. The incremental cost of adding traffic to an established long-haul voice network is low, he says. "They'll keep ratcheting down the prices."

Lessons learned

A billing system is in the works to track what each agency owes to the state, and what the state owes to individual vendors that provide pieces of the net, she says. And the administration team needs a system for verifying that each vendor lives up to its contracts.

"Nobody likes administration and billing, but it's a huge component of being cost-effective," Egges says.

Management tools are also somewhat lacking, Egges says. "It's not as proactive as we would like it to be. Now, we find out about trouble when an agency calls and says a router isn't working."

Ideally, planners should build their business processes first and have them in place as the network is turned on, Egges says. "We were unaware that administration of the network would be as people-intensive and time- consuming as it turned out to be."

Looking toward the future, the BadgerNet team expects to embrace digital subscriber line (DSL) technology as access bandwidth needs continue to grow (DSL offers higher-thanT-1 speeds for less than the cost of multiple T-1s or fractional T-3s), and ultimately may look at some optical switching technology when it becomes available.

Asked if some day it wouldn't be easier to swap out all the BadgerNet components and simply tie remote locations to Madison over a big Internet-based virtual cloud, Egges and Alberts were emphatic about how far away that future lies.

"Quality of service is still not there," says Egges, an ex-Bell Labs researcher who still does a lot of work on QoS. "There are so many pieces needed that are just not there yet. I'm not saying it can't be done, but at this point it takes a lot of engineering resources to make it work."

"Show me that it can work and that it will be cost-effective, and then I'll consider it," Alberts says.

For now, the state is content to sit back and reap the savings from this gargantuan net project.