Delimiter matching in Perl?

[jducoeur]

Here's a question for the Perl and/or regexp experts in the audience; all help is solicited.

ProWiki has a query language built in. Simplifying greatly, the syntax looks like this:

{? [query terms] : [display results] ?}

This translates roughly as "for each page that matches the given query terms, show the display results, interpolating the properties of the page". That all works nicely, and is at the heart of what ProWiki does.

The problem is, I'd really like to be able to do this recursively. That is, I'd like to be able to construct a query like (to take today's example, one of many):

{?~Faction : %%Name%% -- {?~Character && Faction==%%PAGENAME%% : %%Name%% ?} ?}

That would translate as something like, "For each Faction, display the Faction's Name, and then for each Character in that Faction, display the Character's Name". Basically, nested foreach loops.

That's conceptually straightforward, but I'm stuck on how to parse it. ProWiki, being based on UseMod, uses Perl regex for its parsing. That mostly works fine, but I can't figure out how to get it to work recursively. I need to find the *matching* {? ?} pairs, extracting as plaintext any pairs that might be contained inside them. (The Perl code itself will then deal with the recursion into the plaintext subexpression.)

Can this be done straightforwardly in regex? It seems like a fairly common problem -- it's basically a fancy variant of parenthesis matching -- but I'm not hip enough to regex to see the answer. It's not simply a matter of matching first and last delimiters in the string, since a given page might contain several unrelated top-level expressions; therefore, I need to find the genuinely *matching* delimiters.

I know there are a bunch of Perl gurus out there, so if you can outline the solution to me (even the solution to the basic parenthesis-matching problem would probably show me how to do it), I'd be grateful. Thanks...

Comments

[jducoeur]

Innnnteresting. Good observations. I hadn't thought about that because I haven't used that construction before -- but of course, the only reason I haven't used it is that I can't parse it yet. I'm not going to change things for the older ProWiki system (which is nearing the end of its life), but this is worth thinking about for the next-gen Quirki system.

On the theoretical level, I twitch slightly, because there is no *formal* relationship between the Faction objects and the Faction property of the Character objects that happen to point to them -- the two type names match only by convention. That said, it's likely to be a common construction, and one worth encouraging for purposes of making clear and obvious databases, so providing neat defaults that DWIM if you follow that model makes a certain amount of sense.

Very interesting syntax suggestion; I'll give that some serious consideration. I've been tending to think in terms of a more explicit and detailed syntax (along the lines of typical programming-language expressions) because it's easy to see how to fully generalize it, but your suggested model is nicely clear. I need to think about whether there's a good middle ground, that provides both the clarity of yours and the power of mine. (This also plays into the question of how much people are going to be writing the expressions themselves, and how much using a WYSIWYG editor, which I'm still mulling.)

As for pluralization, it's an interesting point in context, since Rails (my likely platform for the next generation) *does* have a built-in pluralization mechanism. But I doubt I want to rely on that for anything -- I'd be willing to bet that it's quirky and often inaccurate...

[siderea]

Glad you enjoyed that. Are you unfamiliar with CF? If so, at some point I should explain the CF CFOUTPUT syntax and usage to you. Not that I think you necessarily want to adopt them, but because having that paradigm floating around in your brain colliding with other stuff will probably be usefully inspiring.

there is no *formal* relationship between the Faction objects and the Faction property of the Character objects that happen to point to them

Why not? I would think that asserting all object/property names within a namespace/wikispace must be unique (except some special built-in keywords like "name" and "number" and "length" and "type", perhaps), so that a reference to an class as a property means a the property is a foreign key to the object. I'm trying to imagine a case where that is bad, and not coming up with something, though that may be an insufficiency of caffeine on my part.

[jducoeur]

I'm only slightly familiar with CF -- what it is and does, but no details. (It's fairly unusual for me to be working with an application that relates closely enough to the DB for CF to be appropriate.)

I would think that asserting all object/property names within a namespace/wikispace must be unique

I'll think about it. The language geek in me somewhat rebels at the notion of having type relationships be implicit in the name like that, but I can't deny that there's a strong ease-of-use argument for it, and I'm not sure there are any compelling use cases against it...