From...

October 6, 1999
Web posted at: 2:58 p.m. EDT (1858 GMT)

by Rich Morin

(IDG) -- The Perl community is served by a wide range of excellent books. It also has two magazines (and columns in others), thousands of pages of online documentation, a fancy bug-reporting system (perlbug), a distributed worldwide archive (the comprehensive Perl archive network, or CPAN), and a plethora of mailing lists, newsgroups, and Web sites.

Most open source projects, however, are far less fortunate. The typical open source project is lucky to have a Web site and a mailing list to its name. Its offerings may be relegated to temporary storage on a university FTP server, or, worse, be scattered over several disconnected servers.

Indexing

Despite the indexing efforts of Debian, FileWatcher, FreeBSD, Freshmeat, and others, particular packages can be very difficult for users to find.

The different indexes use different names for the same packages and create different hierarchies to contain them. FileWatcher, which currently tracks some 10,000 packages, is clearly the largest of these efforts. Its hierarchy is also the best developed, in my estimation.

Nonetheless, it can be a trial for users to find, download, unpack, and examine a multimegabyte piece of software, only to find that it isn't relevant to their needs, hardware, or operating system environment. There should be an easier way for users to browse the source code and documentation for arbitrary open source packages.

MORE COMPUTING INTELLIGENCE
	IDG.net home page
SunWorld home page
Previous Silicon Carny columns in SunWorld
Comprehensive listing of all SunWorld articles by subject
Reviews & in-depth info at IDG.net
		IDG.net's server hardware page
		IDG.net's workstation page
IDG.net's personal news page
Year 2000 World
Questions about computers? Let IDG.net's editors help you
Subscribe to IDG.net free daily newsletter for system admins
Search IDG.net in 12 languages
News Radio
	Fusion audio primers
	Computerworld Minute audio news for managers

Porting and installation

Even when a user has located the original tarball for a package, there may be difficulties in installing the package. Has the current version of the package been ported to the user's system? Where might this port reside? Is the installation painless and/or well documented? What other packages need to be installed first?

The Debian and FreeBSD folks have automated solutions that lead the field in handling these matters. Unfortunately, the solutions only work for their own users. If you're on a Solaris system, the FreeBSD work is (mostly) irrelevant. For that matter, the Debian work is of no direct benefit to a FreeBSD user.

A generalized porting and installation solution is clearly needed, but it is far from clear who will develop it. The Debian and FreeBSD folks have their hands full, keeping track of more than 3,000 packages each; the Red Hat folks seem to be content with RPMs (Red Hat package managers); and nobody else is even looking at the problem.

The open source matrix

The sheer size of the open source matrix is a critical part of the porting and installation problem.

Assume, for purposes of discussion, that we wish to support five hardware architectures (e.g., Alpha, I386, PA-RISC, PowerPC, and SPARC) and 10 Unixish operating system families (e.g., AIX, *BSD, Digital Unix, HP-UX, IRIX, Linux, Mac OS X, Solaris, SunOS, and UnixWare).

This gives us a matrix of 50 architecture/OS combinations, containing perhaps 25 worthwhile ports. Actually, the porting matrix is far larger than this; for instance, neither *BSD nor Linux are completely standardized. Nonetheless, this is a reasonable starting point for our analysis.

25 ports is a challenging task for any open source developer. How can Dave Developer gain access to that range of machines, let alone find the time to learn about their vagaries and port his software to them? The answer is simple: he can't.

Nor, being an isolated, overworked hacker, will he set up an Internet-accessible Concurrent Versions System (CVS) repository to accept patches and ports from volunteers around the world. There might be volunteers out there who have the needed access, time, and skills, but Dave simply doesn't have the resources to perform the needed administration.

Similarly, Dave may not be able to set up a bug-reporting system, e-mail lists, a well-developed Web site, FAQs, or any number of other helpful facilities. He'd like to, of course, but he doesn't have the time.

Now, let's say that Paul Programmer wants to try porting Dave's new Wombat package to an Intel/Solaris machine. Paul can start from the base Wombat distribution (whatever Dave Developer has at his site) or try to locate a few similar ports (e.g., SPARC/Solaris and Intel/FreeBSD). Either method will take skill, luck, and persistence.

Now step back and consider the big picture: 25 ports each of 10,000+ packages. Even using these (rather conservative) numbers, we have more than 250,000 ports to accomplish. And, of course, new packages (and new versions) are coming out of the woodwork every day!

Without proper infrastructure, we can't even track this many items, let alone hope to make a dent in them.

Automated assistance

So, let's give the developers (and ourselves) some help. Let's create a set of CVS (or whatever) repositories that can hold every interesting version of every port of every significant open source package. At current mass storage prices, this would cost only a few thousand dollars.

Developers could use the repositories to find relevant versions of packages. With luck, they might even end up folding some of the variant strains back together, reducing the space problem a little.

By making these repositories accessible via Web browsers, we could let Paul Programmer (and Alice Administrator) find and peruse the code and documentation of specific packages. If Paul or Alice can browse a package in 15 seconds, rather than 15 minutes or an hour, they'll be able to find what they need far more quickly.

While we're at it, let's provide a "dating service" for system vendors and developers. If Dave needs access to a particular kind of machine, for porting or regression testing of his software package, shouldn't the machine's vendor be willing to provide it? If the vendor knows that any incoming developers have been vouched for by a responsible party, there really isn't all that much of a risk.

Some developers (or interested users) might also be willing to maintain current information for specific packages, as long as they don't have to find an ISP, edit Web pages, and the like. Thus, a semiautomated information-gathering system, based on one or more of the existing open source indexes, might be a real possibility.

By providing free, centralized support facilities, we might be able to greatly increase the ease with which all of us interact with open source software. In any case, I think it's worth a try.

Rich Morin operates Prime Time Freeware (www.ptf.com), a publisher of books about open source software. He lives in San Bruno, CA, on the San Francisco Peninsula.