Open Source Documentation, and specifically Firefox

[jducoeur]

So we're now deeply immersed in the project of getting our fancy and powerful Browser Plugin working in Firefox. (Suffice it to say, we feed information from ASAP into the browser, so that the targeted webpages can have some features ranging from difficult to impossible otherwise.) The process is confirming all of my biases about Open Source.

On the one hand, the architecture of Mozilla is really rather nice. There are some silly decisions (like coming up with a new version of COM that is just different enough from Microsoft's as to be incompatible), but the overall notion of having the XUL+Javascript layer in control, and the C++ layer largely subordinate, seems both wise and rather innovative. A bit of a pain in the ass to work with for C++-centric apps like ours, but probably lovely in most cases.

That said: ohmigod, the documentation is such a mess. It's improving, and they're getting to the point where it's decent for conventional applets -- a tad scattershot and cryptic, but most of the information is there. But when you try to do something sophisticated, you're left with little choice but to dig into the source code (not exactly well-commented itself).

For example, right now I've got an unusual but straightforward problem. I want to have the Javascript on my page call into the C++ layer, passing in a Javascript callback to be invoked when we have an answer. Not that weird, right? After quite a lot of random digging through the source, I finally figured out that the answer is to use the "function" keyword in the IDL. Where is this documented? Nowhere, as far as I can tell. All of the online documentation for XPIDL points to the same document, with parenthetical comments that it is "a little out of date". Well, yeah -- it was last updated seven years ago! It's not even the "this is what we wrote" document, it's the "this is the spec for what we intend to write" document. Oy.

None of this is surprising, of course: open source projects are largely at the mercy of what the programmers want to do, and most programmers don't like writing documentation. But man -- it would all be so much more useful if there was a clear documentation plan here...

Comments

[metahacker.livejournal.com]

While it's true that OSS projects definitely suffer from this sort of problem, I'd have to point out that *commercial* software often sucks on this front. See earlier rants about symptoms like the "getObjects()" function and its documentation ("returns an array of objects"), as well as a complete lack of a roadmap for the docs of the last three highly expensive commercial packages I've had to use. ("Hmm. Maybe I'll see what's in the com.XXX.scv.crs.model package. Something useful might be in there.")

[jducoeur]

True. Oddly, this is one way in which I am spoiled by Microsoft. While their API documentation isn't as well organized as I might wish, it is really magnificently complete. (And even, in large part, fairly clear.)

Overall, I would say that Firefox is a better-designed browser than IE, but IE is surprisingly a bit easier to program to, solely because of the relative quality of the docs...

[edolecki.livejournal.com]

See: http://jducoeur.livejournal.com/227261.html

You could add your findings to the documentation and help others who will be doing the same things you are right now.

[jducoeur]

True in theory, but not so great in practice. I'm still a relative novice in this codebase, so most of my observations are inevitably a bit naive and oversimplified -- that means that I'm unlikely to write very *useful* documentation. And what it most needs isn't more scattershot observations: it needs folks who really know, eg, XPIDL well to write a decently comprehensive document describing it.

Good documentation has to be written by the people who know the code at least adequately well. I do that where I can, but I'm a *long* ways from being able to do that well in this case. Really, at this point I'm in a very weird place: very knowledgeable about a lot of strange nooks and crannies of Firefox, but fairly naive about the browser as a whole. So while my natural instinct is to jump in and add to the pile, it isn't at all clear to me that that's the right thing to do here.

As I said, they're improving, which I take as a good sign. In particular, the documentation they *are* coming out with shows some signs of planning, which is essential to really good documentation. (It's a curiously good sign that there isn't an obvious place for me to *make* my random observations.) I have some hope that they will prove me wrong in the long run, and end up with really solid docs. But for now, it's a tad painful...

[(Anonymous)]

"Curiously good" - why?

[jducoeur]

It's a sign that the documentation is much more *organized* in the current push than in the previous incarnation of the Mozilla site. In the past, they've tended to have very ad hoc documentation, as many such projects do, and the results were pretty chaotic -- rife with contradictions, hard to find the answers you need, and generally very out of date. The new site, by contrast, shows signs of people really thinking about the documentation problem: while it isn't *nearly* complete enough, it's at least decently organized. It's not full of random disorganized notes. (At least mostly not.)

Hence the hope: they seem to be moving in a direction that might produce genuinely good documentation, if they can get enough knowledgeable people to work on it...

[jducoeur]

Actually, I should amend my earlier comment. It *is* probably worthwhile for me to write up my experience on mozillazine, which is the community side of the project -- thanks for getting me thinking in that direction. That's not nearly as good as Real Documentation, but it's something I can do that isn't counterproductive...

[talvinamarich.livejournal.com]

Do you ever use Lisp?

I ask, because I am the Tech Writer for a company that is putting out an implementation of Lisp as part of its flagship product. While I am very new to the Tech Writing field, my rule-of-thumb throughout has been "Sooner or later, somebody working late at night, living on caffeine, and facing a deadline is going to have to USE what I wrote!"

If you are interested, let me know and I will point you to our website. (It's a work in progress, we're still ironing out the wrinkles.)

--Colin

[jducoeur]

I've actually done major near-commercial-grade code in Lisp. (The prototype for what was at the time a fairly fancy CASE system for Ada.) But that was many years ago, I'm afraid. I'm fond of the language, but haven't used it for anything significant in a while.

But it sounds like you've got the right viewpoint here. Heaven knows, good docs are worth their weight in gold...

[goldsquare.livejournal.com]

Off topic, but you may want to see this.

(I created the feed this morning, it's Galleron's blog on Blogspot.)

[jducoeur]

Intriguing. Fairly typical for period (pure luck-oriented gambling games were popular), but with a little more psychology than some. Thanks for the pointer!

[baavgai.livejournal.com]

I completely sympathize. I wrote a XUL app for a project a few years ago (for phoenix?). I recall XULPlanet was up, a much smaller site then. Few other resources were available. Figuring out functions sometimes involved CVS tree climbing.

At the time, I couldn't shake the feeling that I was one of the few programmers in the world who'd ever tried to write a XUL application who wasn't directly involved with the mozilla project. Seriously, it felt lonely.

Best of luck. Those kind of projects tend to be rewarding when you get the bloody thing to work, but much cursing tends to precede that point.

[jducoeur]

Yep, that's pretty much the sensation. I'm still not very impressed with XULPlanet, honestly -- I've come to realize that much of the content there is simply auto-generated from the source code, and is generally *less* complete than the comments in the code. It's nice enough when you're getting started and have *no* idea where to look, but it's really not very useful for complex problems.

Best of luck.

Thanks. Having survived this once already, I'm sure we'll get through it. (The IE documentation is quite rich, but the threading model is insanely persnickety, and the bugs remarkably hard to track down.) But it'll be nice to get done with the research phase and move into simply getting the code working...

XPCOM

[metageek.livejournal.com]

There are some silly decisions (like coming up with a new version of COM that is just different enough from Microsoft's as to be incompatible),

FWIW, that was a Netscape decision, not a Mozilla decision. And it still seems like a reasonable one to me: Netscape needed to bring down the cost of cross-platform development (we had a terrible time finding people to develop on Mac), and some people at Netscape liked COM, so building a cross-platform COM made sense.

Of course, in the current Mozilla codebase, I expect XPCOM is less important than expected, since most people are probably writing components in JavaScript instead of C++. Of course, that just means that nobody's learning XPCOM well enough to document it...

Re: XPCOM

[jducoeur]

so building a cross-platform COM made sense.

Oh, absolutely -- IMO, it isn't possible to write good, decoupled code in C++ without COM. (COM fixes the two worst flaws in C++: interface polymorphism and reference-counted memory management.) The silly decision was to make it 98% compatible with MSCOM, but change all the names and very *slightly* change the semantics, just enough to confuse the heck out of experienced COM programmers and render the objects incompatible.

Mind, COM is at its core fundamentally cross-platform. Microsoft layered a lot of Windows-specific junk onto it, but the essential ideas work perfectly well anywhere, and platform-specific COM components work perfectly well with platform-independent ones. (Indeed, I've reimplemented those core ideas on a Linux platform myself.) That's why the decision to make XPCOM incompatible surprised me: it would have been no harder to make it compatible, at least for basic functionality. XPCOM winds up doing a few things better than MSCOM, but at the cost of making other things harder, so it seems like an even trade to me.

As for Mozilla: actually, it's still pretty critical. Simple little things are easier and better to leave to the Javascript/XUL layer, but for really fancy problems, you still need to resort to C++. And indeed, even on the Javascript side, it's sometimes best to structure it as an XPCOM component. (Which can now be written in a bunch of different languages, not just C++.)