This is a very long-term question -- there's no chance anything will be done about it in the next six months, and it's likely to be a year. But it occurred to me, and it's an interesting question, so I toss it out to the Lazyweb for pointers and/or thoughts.

Priorization

In Agile development, Estimation is almost always a group project -- we figure out how much effort a given User Story is likely to take by playing Planning Poker or something similar. But Prioritization -- deciding what order we're going to tackle things in -- typically isn't: most often, it is a single Product Manager sweating bullets, rearranging the Story Stack every hour or two and getting tagged to describe the current state of the world at the beginning of each sprint.

(Note: I am assuming a proper prioritized Story Stack -- the User Stories are arranged in strict order of priority, defined however you like, and the team pulls the top stories off the stack for each sprint unless there is a good reason to do otherwise. I have never seen another system that actually works *well*, at least for the near-term stories. For more distant stuff you can be more coarse-grained, and simply mark things High/Medium/Low, or something like that.)

There are good reasons for this approach, and I think it's always the way to make the final decisions. But what about when you have a project like Querki, with literally hundreds of user stories (indeed, I suspect I'll go over a thousand before the end of the year), not nearly enough resources, and you want deep community involvement in the prioritization process?

I've seen sites that do something simplistic for this, with a simple thumbs-up, or maybe up/down voting. I don't think it actually works all that well, though, because it's prone to all sorts of systematic problems -- it's very easy to game, it's prone to the highly-voted items getting more up-votes, the newest items tend to become trendy, and so on. The Law of Large Numbers helps, but my sense is that it's a pretty weak system for making *good* decisions. (It's perfectly fine for making the community *feel* involved, but not as much for working with them to make good prioritization decisions.)

So it occurred to me that it might make more sense, instead of treating it as "voting", to literally have the community participate in prioritization -- just at a small enough grain to be practical.

A Possible Design

Here's a strawman scenario for your consideration:

As an active member of the community, I go to the "Prioritize" page. The page offers me five random User Stories in a drag-and-drop list. I can click on each to see more information about it, and I rearrange the list to match my *personal* opinion of what their priority order is. I press "Done" to record my vote, or maybe "Done and Give Me Another" to keep playing the game with a new list. My opinion then gets fed into an overall Querki-wide list, which combines all of these votes to come up with a current master state of things.

There are many possible tweaks -- tracking my personal "priority order" or not, being able to see and/or work with a larger list, letting me simply declare "Pass" on the entire list because I don't care about any of these -- but that's the core idea I'm pondering: taking lots of little easy-to-digest lists and combining them into a big one.

Personally, I think this would not only be a good way to get the community genuinely involved, I believe it would actually be kinda *fun*. I could actually see myself getting a lot more involved doing this than I would in the usual upvote systems.

The Problem

Plain and simply, the issue is: how does it actually *work*? What is the algorithm to take these individual snippets of prioritization, and combine them into a single gigantic list that represents the gestalt feelings of the community?

I'm sure that I could come up with *some* sort of adequate approach if I sleep on it for a day or two. But I'm wondering whether there are any well-established algorithms for this sort of problem. In airy theory, the question is how to take a collection of inconsistent ordered sublists, and combine them into a single best-fit complete list. Intuitively, it feels kind of like a statistical problem, but despite three years at Memento, my statistics are quite weak. I'd love to find that there are already some well-established best practices, and maybe even a proof of a "best" algorithm within certain constraints.

Suggestions? This is a case where, while I probably could reinvent the wheel, I suspect it's smarter not to...

One reason I've mostly been fairly quiet lately is that my brain is being positively eaten by work. Being Product Manager is a whole new set of responsibilities to get used to. In particular, while I can traditionally leave my programming at work, and get away from it in the evenings, that's harder now -- a lot of my job is simply thinking about the product in the early mornings and late evenings, continuing to feel around the edges and try to figure out where it needs to go. On the one hand, it's a lot of fun, especially to the world-building side of my brain. OTOH, it's very distracting.

All that said, the shift to being a purer startup is proving excellent for my blood pressure. Yeah, we still have arguments, and they can still get pretty heated, but they're just plain more *manageable* at this size -- they're typically between two or three people, not a whole roomful who are factionalizing. And since the group is now made up more or less entirely of professional programmers, all of whom are decent at getting their egos out of the way, we're tending to resolve the arguments pretty quickly.

And as I'd hoped, we are finally doing something that's credibly Agile. We've dumped most of the tools we were using (including the execrable VersionOne), in favor of keeping track of things mostly on 4x6 index cards taped to the white board. The cost of this is that we lose some precision in keeping track of what's going on and where we're going -- OTOH, I'm not sure we're losing any *accuracy* in that process. Our schedule so far is mainly based on Onur's and my gut-feel about how long things ought to take to implement; so far, that gut-feel looks to be about as accurate as our old heavyweight processes were.

Perhaps most importantly, we don't know exactly what we're building yet, but that's really a feature as far as I'm concerned. Historically, we've tended to spend months agonizing over every detail, nailing down specs in one form or another, implementing to those specs -- and then finding that we were wrong in the first place. Instead, we're attacking the system on a story-by-story basis, making progress far more quickly, and I don't see any evidence that we're heading in any less correct a direction. Rather, I've been drilling home the mantra "Nothing Is Set In Stone". The odd advantage of knowing that we're out on the bleeding edge here, and are going to have to learn as we go, is that we're spending a lot less time making decisions: since every decision may change down the line, we're a lot more willing to try stuff out and see how it goes.

All of which boils down to: yeah, this is why I like working in startups. It's crazier and riskier, but man, it's a lot less soul-deadening.

As for what the product itself is -- I'll probably start talking about that soon. We're not really in stealth mode this time, having decided that that's really not very helpful, but we want to get a little further along before we start shouting it to the rooftops. Basically, we're combining a number of fairly conventional ideas about communication and conversation, stirring in a bunch of little elements that really *ought* to be common but for some reason aren't, and baking it all together into a communications tool that's just plain more *useful* than any other single product out there. If I'm right about this, we'll wind up with a feature set that everyone else will then trip all over themselves trying to copy, and we'll be off to the races...