*God*, I love working in Scala

[jducoeur]

Okay, time for a little burble. [This one is solely for the truly geeky programmers; the context is, of course, Querki. I don't have anyone to do code reviews with, so I'm afraid you get the burbles.]

So two days ago, I implemented _modTime, which produces the time that the received Thing was last changed. Yesterday, I realized that this was really only useful if you can sort on it, so I enhanced _sort to take a parameter, like this:

[[Page._instances -> _sort(_modTime)]]

But of course, that produces the page in time order, which is almost never what you want -- 9 times out of 10, you want to print things in *reverse* time order, from newest to oldest.

So I clearly wanted some sort of "_desc" modifier on _sort, so that you can tell it to sort in descending order. My first instinct was to just add a _reverse function, an easy and just plain useful tool to just reverse the list order, but that's not quite right: I want the results to be in descending order by time, but ascending order by name if the times match. I specifically want to reverse _modTime, *not* necessarily the entire results.

Thinking about it a little, I decided that the obvious syntax had to be:

[[Page._instances -> _sort(_desc(_modTime))]]

That is, "sort by _modTime, descending".

But wait a second -- what does that *mean*? I mean, _modTime is being applied to each element in the list; saying that an *element* is "descending" seemed almost meaningless, until I thought about it a bit more, and realized that what I'm doing is transforming the *type* of the result -- "_desc(_modTime)" clearly means "return each item's modTime, with the sort order of the returned type reversed".

That's kind of insane. And I was completely floored to discover that this is, pretty much, all the code it required (going into Scala now, and omitting documentation):

class DescendingType[VT](baseType: PType[VT]) extends DelegatingType[VT](baseType) {
  override def doComp(context:ContextBase)(left:VT, right:VT):Boolean = !realType.doComp(context)(left, right)
}

object DescMethod extends InternalMethod(DescMethodOID, toProps(setName("_desc")))
{
  override def qlApply(context:ContextBase, paramsOpt:Option[Seq[QLPhrase]] = None):QValue = {
    paramsOpt match {
      case Some(params) => {
        val innerRes = context.parser.get.processPhrase(params(0).ops, context).value;
        innerRes.cType.makePropValue(innerRes.cv, new DescendingType(innerRes.pType))
      }
      case None => WarningValue("_desc is meaningless without a parameter")
    }
  }
}

That is, when you call "_desc", it processes its own parameter (params(0).ops), passing in the received context; rewrites the result using the same Collection (innerRes.cType), and wrapping the Type (innerRes.pType) in a pseudo-Type (DescendingType) that reverses the result of comparisons (doComp()); and passes that down the pipeline.

It simply works, and took less than half an hour to come up with. Wow. Yes, I understand that the above looks kinda cryptic, but seriously: I can't think of any other language I've ever played with that could do this, preserve type safety, and not have me tied in knots for a day or two.

(And the implications here are staggering. I'd added DelegatingType for a fairly minor boot-time requirement last month, and hadn't given it much thought. But if this works, it means that Querki can do almost arbitrary type transformations at runtime internally. Which means that sooner or later, we're going to be getting the same sorts of high-level Type operations in QL itself. Neat...)

Comments

[dsrtao]

Querki can do almost arbitrary type transformations at runtime internally.

forgive me if I'm misunderstanding this, but this translates in my head to "Querki will be nearly impossible to debug". Am I wrong?

[jducoeur]

We'll see -- it's a good question. That doesn't necessarily follow, though: at best, Querki's never likely to have as much power as Scala itself, and that certainly is debuggable.

I suspect that this will wind up as yet another argument why Querki will gradually wind up more and more strongly typed. That hadn't been the original intention -- in certain ways, Querki is intentionally *un*typed -- but I'm gradually coming to realize that the data pipeline itself can and probably should be very strongly typed. That will make it easier to write in (specifically, it'll make it easier for the system to do hand-holding, and suggest sensible options); it's also likely to make debugging much easier.

But like I said, we'll see. I'm deliberately allowing myself to be radical here, and I suspect it *will* be a long and serious project to get to the point where folks find it easy to debug Querki code. There's going to be a lot of experimentation, and some hard lessons.

OTOH, if you're concerned about Querki *itself* being difficult to debug -- that's not quite the least of my concerns, but it's not high up there. That's the beauty of Scala's extremely strict type system: most bugs don't even *compile*. Which doesn't mean there are no bugs, but the number is actually surprisingly low given the size and complexity of the code. (And the *vast* majority of the bugs are in the client-side Javascript, not the server.) There's a lot of test automation that still needs to happen, but I believe we'll be able to achieve reasonable robustness without *crazy* effort...