Hmm. How does one build an open source community?

[jducoeur]

One side-effect I'm observing, as I work on the Girl Genius game for Intercon, is that it's getting me fired up again for the Querki project. The GG LARP is being built using ProWiki, the pseudo-next-gen wiki system I hacked together from the UseMod codebase. That functions adequately, but it mostly is reminding me that (a) UseMod was a pretty crappy and ancient codebase to begin with, and (b) I could do *so* much more if I did this from scratch. The more I learn about wiki-tech, the more appalled I am at the way most of them work. Most of the code I've examined is pretty dreadful (yes, I've dug through many of the "big name" wikis, and even most of those are pretty mediocre structurally), and the common religious orientation towards being world-editable prevents many from being all that useful for my kinds of practical work, by winding up with miserable access control. And none are flexible enough to easily change the back end as dramatically as I need to.

So it's pretty clear that I'm talking myself into moving forward on Querki, the true next-gen wiki that I've been gradually designing, and I've largely talked myself into starting from scratch. At the core, it's very different from ordinary wikis -- a sort of hybrid between an OO database and a wiki -- and I have a clear idea of what that wants to look like. But it's also clear that, if this is going to be a serious tool for general use (which is the objective), it will also eventually need the various bells and whistles that folks expect of a real, enterprise-grade wiki these days. And that means that this is more than a one-man project.

Which leaves me with the slightly daunting prospect of creating an open source community. I have only the vaguest idea of where to start. I figure that I'll probably build the initial skeleton of the system myself (starting after the game runs in March, and I have some time again), but once that's put together I really need to puzzle out how to attract a good group to work with me on this. I'm also going to have to learn how to steer the thing appropriately -- how to let the other contributors have their heads without letting the thing dissolve into a puddle of ill-designed goo.

It's an exciting prospect, but *quite* scary. I've worked with plenty of software projects over the years, but my experience with open source is pretty tangential. If anyone has any pointers on how one kickstarts a project like this, I'd be interested to hear them. (And if anyone might want in on the project, do speak up! It'll be the middle of next year before it all begins coming together, but I'm starting to get my ducks in a row now, and it would be encouraging to know that there are programmers interested in helping out...)

Comments

[talvinamarich.livejournal.com]

I am not an expert. Nor can I help you get in touch with the author of this.

But, check out:
http://cloudwiki.sourceforge.net/doc/Cloud%20Wiki.html

It does have an option for Access Control. We used it in my documentation gig.

[jducoeur]

Oh, access control isn't *rare* -- most enterprise-grade wikis have at least some form of it, and even some of the public-editing ones have a bit. (For instance, it's clear that MediaWiki has some access controls stuffed quietly away in the back.) But it's still surprisingly common to find wikitech that is completely religious about the idea that everyone can edit everything. (Or which just lazily ducks the question...)

[bkdelong.livejournal.com]

SourceForge is sort of dying as an open source community. I've built many OS communities and it has as much to do with infrastructure as PR and Marketing to bring the bodies in to help and contribute. I've been building communities in the OS space for about 6 years. Next time we're at a guild practice or maybe even an event and we have OOC time to talk, let's chat. Or catch me on IM in the evenings.

[jducoeur]

Very interesting. Yes, at some point I should pick your brain on the subject...

[goldsquare.livejournal.com]

My observations are that you need 1: luck, 2: PR, 3: a small and useful project as a first release, which really demonstrates your gravitas and ability, and which is also useful.

Consider that many of the open source projects I worked on had lots of 2, not enough 3 and poor enough 1 that it never took off....

[jducoeur]

I'm not too worried about (3) -- this project is happening precisely because I am finding this a really useful approach myself, and want to do it right. After using the prototype for years now, I've got a pretty clear idea of how to push this to the point where it becomes useful in a wide variety of fields.

But (2) and (1) are a much more interesting problem...

[metahacker.livejournal.com]

Brief experience on failed projects has indicated that you need a champion, visibility, and coolness. The champion soldiers on while times are tough, but always in an obvious and positive way; a blog may help here. People like to know the project is alive. Visibility is hard, but the usual methods (word of mouth) may be enough to reach a small critical mass; after that bland methods like recruiting on public websites (SF, wherever) may work. Coolness is hard to quantify: snappy screenshots and a clear statement of why you're better than the 1e06 other wikis out there may be important ingredients. You need to interest potential developers in about eight seconds, which means advertising.

Also, having something that is small but useful, with a firm foundation, and that can be built upon in an understandable fashion, seems to help avoid the death of over-reaching that so many projects (OSS and otherwise) fall to. The best FOSS projects -- say, GAIM -- started with a small, working kernel that did what people needed. Do enough of that, and couple it with your intrinsic abilities to attract, interest, and organize people, and I think you'll be on the right track.

I would ask why you feel it necessary to build a wiki from scratch; there really are a million of them out there now. If you haven't hit http://www.wikimatrix.org/ then I recommend spending some time looking at it.

[jducoeur]

Very good observations. I'm clearly going to be the initial champion myself: I think it's a very cool project technically, and I am *passionately* opinionated about the architecture. The blog idea is an excellent one, which I'll need to keep in mind.

The coolness thing is where life gets tricky for me. I can probably come up with the High Concept statement of why this is a cool idea, but I can't do the look-and-feel. I Am Not A Designer, and I'm fairly sure that this project will need a serious designer to make the initial incarnation look pretty enough for folks to pay attention, and to provide feedback on what a designer *wants* out of such a system.

The small-but-useful point is well-taken. I find myself leaning towards a classically Agile approach to this project. I'll probably come up with several *hundred* Stories of what I want it to do, but I should be able to prioritize them into a useful list. And I have several targets apps that will make this thing really shine.

As for why: from what I've seen so far, this really is *very* different from everything I've seen out there. (I haven't looked at everything in the mammoth WikiMatrix, but I've delved into quite a bit of it.) The difference is deep in its guts. Most wikis *think* of themselves as a bundle of random text data (pages), which might have a bit of metadata tacked on. As a result, all the instances I've found are pretty half-assed about working with that metadata -- inefficient at best, and generally incapable.

By contrast, ProWiki has data built pretty deeply into it, and Querki isn't actually a wiki. It's actually a slightly specialized object-oriented database that *looks* like a wiki. Whereas all wikis that I know of optimize for the completely unstructured data (text pages) and treat structured data as a minor add-on, Querki considers the structured data to be the heart and soul of the system, with the text pages as the simple degenerate case.

The result is that the Querki architecture is in many respects vastly more powerful than any wiki I've encountered. I already routinely do things in ProWiki (which, remember, is just a highly-simplified prototype) that are difficult and inefficient in all other wikis. For instance, my LARP development is all about creating a very complex but highly structured data model for the world, with a schema that involves things like Characters, Factions, Items, Plots and so on. By using a genuine data model under the hood, I am able to trivially build queries, do complex linkages and summaries, and so on -- all sorts of things that are hugely useful, but hard to do in conventional systems.

ProWiki is already unusually powerful, but it is (a) not at all pretty; (b) hideously unscalable; (c) fairly hard to use. Hence the Querki project: to build something even more powerful, that solves the above problems. Which are easy to solve, if I build the thing correctly from the ground up, instead of trying to weld it onto an existing wiki system.

The possibilities are pretty dramatic. I have my use case already to hand, but lots of others come to mind quite easily. For example, while thinking through my process, it's clear that I want an Agile Story management system. I believe I'll be able to build such a thing in Querki in half an hour. It's all about what I think of as managing "semi-structured data" -- multi-user applications that thrive on having some loose data structures organizing text. Many, many applications are really little more than that. This is the platform to whip such things up, quickly and easily.

Querki isn't about trying to beat wikis at their own game -- if what you have is simply unstructured text, I don't expect it to be any better than they are. Rather, it's the next-generation evolution of wikis, providing a highly flexible framework for imposing semantics and structure on your text. While I'm reluctant to use the already-overused "Web 3.0" buzzword in public, that's *exactly* what Querki is about -- it is to wikis more or less what the Semantic Web is to the WWW. It's the sweet spot between a database and a wiki, something that (IMO) the world desperately needs. Certainly it's something that I've found *I* desperately need, which is a good start...

[jducoeur]

Another way of looking at it, again along the lines of "this isn't a wiki". Querki is intended to be to text-focused prototype-based object-oriented databases (IMO, the most useful type of database for general work) what a wiki is to the web: a way for people to quickly and easily slam data in, without having to spend so damned much time worrying about the nuts and bolts.

It just happens to be the case that a conventional wiki is the simplest example of this sort of thing -- a flat database with no inheritance, where each object has exactly one plaintext property. It's time to break out of that limitation, and really explore the potential power here...

[metahacker.livejournal.com]

Note that things like MediaWiki are just starting to move toward this model, with tagging moving onward onto more complex things like dynamic templates, sidebars, etc. However, talking about it as a OO-db is too high-level to appeal to anyone but the abstract hackers, and they're not going to make something folks will use -- and where's the fun in that?

Can you, or have you, come up with a few specific examples of exactly how this is used? Concrete is good, especially if you have two or three defining examples. prog had this problem with Volity -- "a shared, open, modular game environment" was too abstract, but "a place where you can play Fluxx online, and you can write your own games too" grounded it much better...

[jducoeur]

Oh, that's the plan. Initially I'm probably going to "market" it to the LARP community, since that's the app I originally built it for, but I've got several more use cases already in mind. (Such as the story-management system, the threaded-discussion forum, and so on.) I'm figuring that no one's really going to get the idea unless I provide examples.

Indeed, I'm increasingly thinking that I need the ability to export/import schemas so that people can easily share "apps". The thing really is basically a platform, and like all platforms, it's only going to be as appealing as its applications...

[mindways]

I've not contributed to any open-source projects, so I can't tell what you do need.

But I can tell you that the prime reason I've never actually gotten to the "productively chipping in" stage on any open-source project has been activation energy: the initial effort to get even a copy of the program up and running to do bug-testing was often just too damn high, let alone actually getting the codebase locally installed, configured, compiled (or whatever), and running. (Partly this was lack of documentation, partly this was that - in some cases - the documented process simply failed.)

This may be just me - I'm well aware that one of my personal programming pecadillos is a particular loathing of time blown on "wrestle with intractable machine/program config for hours just to get to the point where I can begin programming" - but I suspect not.

[jducoeur]

Excellent point, and well worth my keeping in mind. I've got "easy installation" of the final product on my mental requirements list (although I really should add that to the explicit Requirements page), but it hadn't occurred to me that this might be necessary very early on in order to draw people in.

Hmm. I'll have to think about how far to carry this -- since the idea is to come up with a fairly platform-neutral technology, the installer problem is going to be pretty serious. But maybe I can at least make Really Really Clear Instructions of how to take a zip file and get it quickly up and running for a basic local install to play with. (If I decide to use Rails as the technical basis, which I'm considering, I'll have the advantage that they've already simplified *that* install a lot.)

Double-hmm. This has also been a starting issue for *me* getting involved in projects. It's a typical problem of SourceForge. I need to think about how to overcome it. Thanks -- this is really serious food for thought...

[doubleplus.livejournal.com]

Much belated, but I'd be interested in helping out. As with the first time you were tossing something like this around, I don't necessarily have any skills in the specific likely languages, but what the heck, it's been a while since I've learned a new language. My pet peeve with open source projects is that the parts that get the most work are the ones that are the most fun, which means that documentation is half-assed or completely out of date all too often, so I'm willing to pitch in there as well.

[jducoeur]

Neat! When the time comes (probably middle of next year or so), I'd definitely love the help. One of the key issues that I suspect I will struggle with is that I have high standards for such a project -- I want both good documentation and a professional-grade (indeed, enterprise-grade) architecture and design. So having some collaborators with a professional attitude would be more than welcome...

[jducoeur]

For my own future reference: it turns out that O'Reilly has a book on the subject. Looks fairly detailed, and seems to get into the politics...