[GEEK] ProWiki 0.1a Released

[jducoeur]

Okay, good enough. The first draft of ProWiki seems to be stable enough to actually use; I've written the documentation and the beginnings of an example game, so I think folks can see what's going on.

The full details can be found at the ProWiki site -- this includes a tutorial on the major concepts, some discussion of the internals, and a few initial characters from The Incredibly Dippy Subway Game, a micro-LARP set in Harvard Square Station, which is mainly an example of how to use ProWiki.

The basics, for those who are curious, is that this is a heavily enhanced version of UseModWiki, specifically designed for use with semi-structured data such as LARPs. (The explicit motivation for this project was to make Tabula Rasa III a little easier to write and track.) The primary innovation is the concept of properties -- named characteristics that a page can have. Properties can serve as data, holding arbitrary strings; they can also serve as links between pages, defining relationships between them.

Most of the interesting parts of ProWiki come in the form of queries: structured statements that allow you to generate the contents of Wiki pages based on the properties of those pages or others. There are queries that allow you enumerate pages based on their properties, queries that enumerate the properties on a specific page, and properties that act through inheritance. Yes, as part of all this I've added a very simple concept of inheritance between pages.

It's all quite rough-and-ready, as much a technology experiment as anything. But I think it should make Wikis far more useful for the sorts of purposes I have in mind, and I suspect that I've scarcely scratched the surface of the idea. If you want to really grok how it works, dig around in the sample game and look at what the page sources look like -- if you're used to normal Wiki technology, you'll probably be a bit surprised. Feel free to download the source and play with it, but be warned that this is all still very experimental code, and probably far from stable. Please do *not* edit the ProWiki site itself without asking me first. Thanks, and feel free to ask questions if you have them...

Comments

[siderea]

Huh. You've reinvented the CMS, only.... sideways. And in a wiki.

And, unfortunately, in perl which writes to disk. This would be so much cooler in php in a real db. Maybe I'll do that, someday.

[jducoeur]

The wiki-ness is desireable for my purposes. The nice thing about wiki as a concept is that it's pretty nicely designed for building complex nests of pages quickly and easily. It just wasn't sufficient for my needs, so this turned into an experiment on what would need to be added to *make* it sufficient. It's proving a useful illustration of the requirements, even if I expect the eventual implementations (and indeed, the eventual syntax) to be quite different.

I don't know CMS. Pointer?

As for perl and flat files -- yeah, I'm painfully aware of the shortcomings there. Perl continues to annoy the hell out of me, as I try to use it for serious programming: it is agonizingly idiosyncratic, and far too willing to conduct implicit casts that I don't want. Its attitude towards passing complex structures as parameters and returns is especially irritating, and the source of many subtle bugs.

One of these days I may reimplement the whole thing in a language and data format that makes me happier. PHP wouldn't have occurred to me, but I suppose it's probably powerful enough for the job if it has adequate regex support. Personally, I'd prefer to write the guts in C#, but I'm not willing to run IIS to get that...

[siderea]

I don't know CMS. Pointer?

"Content Management System". Google. Check out cmsmatrix.org, sourceforge.com, or opensourcecms.com. You are now well into that space, yet uniquely in a space of your own...

Basically, you've invented a LARPing/co-authoring CMS.

As for perl and flat files -- yeah, I'm painfully aware of the shortcomings there. Perl continues to annoy the hell out of me, as I try to use it for serious programming: it is agonizingly idiosyncratic, and far too willing to conduct implicit casts that I don't want. Its attitude towards passing complex structures as parameters and returns is especially irritating, and the source of many subtle bugs.

Uh, all these things, true and irksome as they are (and you have your fingers on the pulse of my irritations with perl) my concern had more to do with security. Things like PHP, which is meant for web usage, have built-in aspect to help with certain security issues. Like auto-escaping quoting to prevent SQL insertion attacks.... not that, writing to disk, you'll have any problem with SQL insertion attacks. :}

Have you considered security at all? Even just users and ACLs and stuff?

PHP wouldn't have occurred to me, but I suppose it's probably powerful enough for the job if it has adequate regex support.

It's a perl derivative, so it might.

[jducoeur]

Hmm. It's really interesting that people keep calling wiki a CM system. It doesn't *feel* like one to me, but that may be history speaking. I spent several years working on a Serious Industrial-Grade CM System back in my early days -- basically, the in-house CM system for Delco Electronics and a bunch of the rest of GM, which was extremely advanced for its day. The wiki environment is so dramatically different from both that and the sorts of CM systems I actually use in my day-to-day work life that the comparison seems strange to me. But it may be more like a typical web-oriented CM system.

To be more specific: when I think of a CM system, I think of sophisticated versioning as the front-and-center feature, and everything else is secondary. With the wiki, I think of it more as a creativity-enrichment tool than anything else. I don't see much other point to wiki: there isn't much there that you *couldn't* simply do in HTML, but the wiki environment makes it much easier to just blast stuff in if you accept its limitations. ProWiki is very much in that same spirit -- informal and limited, but very easy to use.

Truth to tell, I'm largely ignoring the more CM-ish aspects of it for now: I'd bet that they don't consistently work in ProWiki at this point, since I've paid absolutely no attention to the interaction of versioning and properties. (Properties introduce cross-file dependencies, and I suspect that the CM code in UseMod doesn't really cope with those.) That's a limitation, but one that I don't care about much for this particular application.

Have you considered security at all? Even just users and ACLs and stuff?

There's a *bit* of security built into UseMod, IIRC, which of course I inherited. Broadly speaking, though, that's not the point with Wikis: they're an intentionally chaotic environment.

For my specific purposes, it's largely irrelevant, because the LARP-writing orientation means that it's mainly intended for private environments. For the moment, I'm simply doing security-through-obscurity: the real, important wiki is not linked into the Web, and isn't going to be. That's poor security conceptually, but often the best bang for the buck.

The question of Perl being potentially dangerous to the machine is a real one, and a modest concern. I'm explicitly assuming that UseMod has addressed the most serious issues (mainly eval hacks), and I've largely picked up its design style for that reason. Provided perl is safe from eval attacks within regexp substitutions (which I do not know to be true, but it seems necessary in order for it to be at all useful), ProWiki is probably decently safe.

In general, though, I don't regard this as industrial-grade code: it's a very ad-hoc solution. I might someday want to reimplement the whole wiki system from scratch and do it right, but it's not worth that very large effort until I understand my needs better. At this point, I don't even really have my hands around what I want to do with properties -- I expect to learn a lot while writing TR3. At that point, once the requirements are more mature, I might consider a full rethink about how it works.

BTW, it seems like you're under a misapprehension that this is a designed system, which is far from the truth. The scope of this project was, "What can I accomplish in about a week?". Of the 172k of code here, only about 12 of it is mine -- the rest is all straight-line evolution from the original wiki system, which has gone through about six major releases since 1997. If I was actually *designing* something, it would look a lot different.

It's a perl derivative, so it might.

Really? I hadn't realized that, but I haven't done anything serious with PHP. (Closest I've come, really, is ASP.NET -- same concepts, but wildly different implementation.) Then yes, it probably has reasonably solid regex support. The whole *point* of Perl is regex, far as I can tell...

PHP vs. Perl

[metageek.livejournal.com]

It's a perl derivative, so it might.

Really?

No, I don't think so; bits of it are inspired by Perl, but it's fundamentally different. Worse, mostly--it's got even more evolutionary scar tissue than Perl. For example, the thing about automatically quoting strings for SQL is terrible when you realize that only a minority of your string-handling code is passing strings to SQL. I've sworn off PHP.

(Mind you, I haven't seen it since version 3, and it's now up to 5. I don't consider that reason enough to look at it again, though. There are plenty of real languages I can use instead.)

[jducoeur]

Some more thoughts, on a bit more reflection.

It's rather interesting: I think part of why I have trouble seeing wiki as a CM system may be that I think of CM systems differently than you. When I look at the CMS things, they sound like relatively integrated development environments -- indeed, the fact that they are called *content* management systems tends to indicate that.

When I say "CM", those initials stand for "configuration management", and the difference is significant. The sorts of CM systems I'm used are generally rather decoupled from the development environment -- they're designed for a world where the IDE and the CM systems work together, but aren't the same package. The CM system is generally fairly content-neutral, focusing instead on sophisticated versioning capabilities. I mean, I've spent my past three jobs all working in Visual Studio as my primary development environment, but each has used a different CM solution -- Perforce, SourceSafe and CVS respectively. And it's noticeable that cmsmatrix doesn't list any of those, despite the fact that they are what *I* think of as the three main "CM" systems.

So I guess it's partly that, since I think of wiki as a simple IDE, I *don't* think of it as a CM system, since they are so unrelated in my mind. Besides, it's just too simple for me to respect it as a CM system: having worked on them professionally, and worked with most of the best commercial ones, I have high standards for CM.

The other thing has to do with mindset. When I think about what's going on with ProWiki, I'm really not thinking about a collection of web pages -- I'm thinking of a specifically object-oriented data-representation system. My mental model for what I'm trying to accomplish has more to do with MOO and similar interactive world-building systems than with the web, even most DBMS based web systems. So that makes the CMS comparison feel like a non-sequiteur to me: it's actually a fairly valid comparison to normal wiki, I suspect, but likely to get less so as I go along, because the data model is rather different...

[mindways]

Very neat!! If I had spare time, I'd probably want to get involved in helping out with it. Ah, well.

The only problem I ran into while browsing the small sample LARP was the number of dead-ends, which interfered with my usual "just keep clicking" Wiki navigation style. This is partly a result of content-free pages such as page templates, but more because query-generated text doesn't show up in the backlink searches - so if you're at GuitarGuy and you click on the page title, the only link that shows up to it is itself, instead of Characters as one would expect.

(I can understand *why* it doesn't show up - annoying to implement, that would be.)

[jducoeur]

Hmm. Actually, it wouldn't be terribly hard to make the Loiterer link at the top of GuitarGuy go to a listing *of* the Loiterers, instead of to the Loiterer base class -- I'm already doing stuff with generated listing pages, so most of the infrastructure is there.

Indeed, it might not be a bad idea to replace the base-class link with a list of ancestors instead, each of which goes to a listing of that class, and it probably wouldn't take even an hour to implement. I'll give that some thought -- thanks...

[jducoeur]

And it occurs to me: is it normal Wiki style to skim around using backlink searches a lot? Man, that's expensive. I can probably do something a lot more efficient than that with property-based browsing...

[mindways]

It entirely depends on the structure of the Wiki in question. Under many circumstances, backlink searches can compensate for poor navigation/interlinking.

For instance, in the example we're talking about, ideally one would implement the scheme you mentioned in the prior comment. Failing that (or perhaps in addition), it would be nice to have a link to "Characters" at the bottom of each Character page (easy enough to do by editing the Character Template - which would also give one a way out of the otherwise dead-end Character Template page, which is nice). Failing either of those, backlinking can often get one to someplace else relevant (except in this case, due to the parameters & suchlike - but in theory).

Basically, when one is looking at a page, there ought to be some non-pain-in-the-ass mechanism for navigating to any other page that could be considered "related" to it. (Many Wiki proponents would use "ought" in an aesthetic sense; I use it mostly in a practical one - browsing a well-linked Wiki is a marvellously easy experience.)

For a Character, the "related" pages I can think of off the top of my head would be:

1. A list of all Characters
2. A list of all Characters in the same group as this one
3. Any item the Character has (handled by in-page links)
4. Any special ability the Character has (handled by in-page links)
5. Any other pages linking to this Character (handled to an extent by backlinks)
6. The display template being used for this page
7. The homepage for the LARP

#1 and #2 would be handled by the tweak you propose; #6 and #7 could be handled by appropriate page templates - though #6 might be something to put in the main navigation in some way; it's a universal enough concept in ProWiki.

[jducoeur]

Ah, of course -- hadn't thought about putting it into the template, but it's clearly correct. Indeed, that's even more worth my while to think about for the TR3 wiki, which has a much more complex semantic web. (With all sorts of interlinking characters, archetypes, geopolitical entities, factions, plots and so on.)

I suspect I'm not going to bother making it as easy to get to the display templates, because they're not something one goes to very often once things settle down. And the homepage for the LARP will usually be the Wiki homepage, in an actual running system. (That's the way the private TR3 wiki is set up.)

But the rest make a lot of sense, and argue for making the editing-oriented display templates quite rich in links. This strengthens my case for enhancing the system to permit multiple display templates for a page, but I was planning on that anyway. (It's another of the reasons I implemented display templates in the first place, so that I can have separate "editing" and "production" templates for each page type.)