Is there a standard Scheduling/Control API? There really should be...

[jducoeur]

Related to the new changes in CommYou -- I've just added commands that let you turn CommYou off and on, on an ad hoc basis. This should hopefully be good enough for the time being. But I know what the next request is going to be: full-scale scheduling of that function. Certainly, if I really *needed* it to be consistently gone during work hours, that's what I would want.

That's on the story list, but I'm not going to deal with it for the time being, for the simple reason that it is *hard*. Really Hard. Those of you who haven't written scheduling apps don't appreciate what a pain in the ass they quickly become.

Yes, it looks straightforward enough: what's so hard about saying "turn off at 9am and on at 5pm"? The problem is, that's never really good enough. I mean, first you discover that you only want that schedule on weekdays, not weekends. (Or you are using this for work, so you only want it *on* during work hours.) Then there the consultants, whose definition of the workday is different. Then there are holidays. Then you find that you really need multiple schedules per day, because you want it *on* during the lunch hour so you can check in with folks. And of course, it's deathly important to turn it off next Sunday afternoon, because your grandmother will be visiting and god forbid that conversation about the bachelor party should come across while she's in the room. Etc -- before long, you're writing a complete scheduling app, and those really are a bloody pain.

(We spent a lot of time fighting these particular fires at Convoq, and never did manage to really get it right in any of the cases.)

What I find that I really want is a standard way for you to be able to say, "Here's my calendar, and this category of appointments are for CommYou". CommYou contacts it via some sort of API occasionally, or maybe even gets control signals *from* the calendaring app. You do all your scheduling through an app that's designed for the purpose; you therefore can manage all your schedules in one place; and I don't have to get into the hassle of building calendaring into an application that isn't *about* calendaring.

I'm sure that this could be done with Google Calendar, which is pretty open, but I dislike coding to proprietary APIs unless I must. What I want is a *standard* for this, so that you can be using Google, Microsoft, Zoho or Whatever, and just point CommYou at a URL, or point the calendaring app at CommYou, or something like that. CommYou and the calendar app talk, and you don't need to hassle further about it.

It's a standard the world really needs. And AFAIK, it doesn't really exist yet. There are standards for exchanging calendar data, but not really this kind of command-and-control API for *non*-calendar apps. There should be. I may yet build the CommYou side of this, and see about building one or two generalized plugins for the calendar apps to get the ball rolling...

Comments

[metahacker.livejournal.com]

Yes, indeedy. I started designing this as a part of the scheduling app I didn't write, and got into a LOT of ratholes. On the flip side, it resulted in a new way of thinking about the 'todo' problem (as one of resource prioritization/utilization) which may be fruitful.

We might be having problems in different areas, though, so perhaps collaboration would yield results. Specifying (programatically) a schedule wasn't too hard -- it was essentially a priority set of sets of intervals, with higher-priority sets overriding lower-priority. (So, "Sunday 3-4 at grandma's" overrides "be available every Saturday and Sunday".)

The time-interval logic is all built, along with a base-line visualization...what really got thorny was letting the user *express* complex intervals. Falling back on NLP, even structured NLP, was driving me insane; but other methods seemed way too geeky, hardly better than cron's obfuscated syntax.

What kind of features do you need for CommYou? Is this "tell IM to alert me at these times, and not at these"?

[laurion]

Layered priorities strikes me as a good way to handle the edge cases, and seems to follow life scheduling fairly well. (I can give you a one paragraph summary of my standard week, and then whole volumes on the deviations.)

[jducoeur]

What kind of features do you need for CommYou? Is this "tell IM to alert me at these times, and not at these"?

Correct, although I've dealt with enough versions of this problem that I'm looking at the general case as well as the specific one.

what really got thorny was letting the user *express* complex intervals.

And that's the rub. I can write the bloody schemas and control systems if I need to. But writing a good calendaring UI is *huge* pain in the ass, and making it sufficiently powerful is directly in tension with making it usable.

So I really, really, really don't want to be in this business. There are plenty of companies that *are* in this business, and I would much rather delegate the scheduling to them. Not only does that get it off my plate, it makes it better for the user, who isn't trying to juggle multiple applications with calendars in them.

Hence, what I want is a *protocol*. And it seems to be lodging itself in my brain -- I've actually wound up with an initial strawman architecture to think about. CommYou would expose an API that describes the categories of events it can handle, provision credentials, and accept authenticated control signals. On the calendaring side, we'd write plugins for the calendaring apps that would take those categories, allow users to associate events with a category, and send appropriate control signals at edge-triggers on the schedule.

There are a bunch of problems with that design, but it's a starting point for discussion, and helps focus on the essence of the idea, which is that scheduling should be done in a scheduling app, not in a conversation app. The more I think about it, the more I conclude that this really ought to be a co-operative ecosystem, and it's not there yet...

iCalendar

[metageek.livejournal.com]

Look at the iCalendar spec; it's a lot more capable than...well, than it should be. The recurrence rule syntax is particularly convoluted, and can probably express most priority rules, since it has EXRULE (basically, set subtraction). See, some of the people who participated in the working group wanted to be able to use it for process control.

I'm not saying it's an ideal solution, but it is at least a standard; you can use it to talk to other people, you can (probably) find a library to implement it, and you can save yourself the effort of designing your own protocol. Spend the time on something more directly related to user value.

Re: iCalendar

[jducoeur]

If I were to simply do schedule exchange, I'd certainly use iCalendar -- I may dislike that standard deeply, but at least it *is* a standard. (And learning a bit about SNMP has given me a better perspective on just how bad a standard can be.)

But that's not really what I want. I'd prefer not to do data exchange -- I'd rather have *control* APIs set up for this. That is, there really ought to be an ecosystem where services can, through a general and standard protocol, make themselves available for scheduled control by outside systems. I'd rather have CommYou be entirely dumb about the schedule, and simply get told by an outside system, "It's time to turn this off".

A lot of the pieces exist within the WS-* standards suite, but there are some categorization and authentication elements that I don't think are there yet. (And the plugins to actually manage the schedule within the calendaring apps almost certainly don't exist yet.) It's a fairly big and messy problem, but a properly-designed protocol would, I suspect, be fabulously useful...

Re: iCalendar

[metageek.livejournal.com]

Oh, I see—you're pushing back the scheduling problem another level.

Re: iCalendar

[jducoeur]

Yes, but I think of it more as decoupling unrelated functionality. (It sounds better that way.)

[metahacker.livejournal.com]

Mmm. I see. So you want a scheduling daemon that has permission to tell other bots what to do, and that takes input from...wherever. Sounds like a useful module indeed. It's effectively what cron does, but with multiple users and permissions, and more complex specifications of times and more complex messages than simply "run this". (Though of course "run this" is a Turing-complete message...)

Then the specification routines/UI can be a separable piece entirely, written by someone with UI clue. Or have multiple UIs, depending on your needs. Hurray for SOA.

Multiple UIs

[metageek.livejournal.com]

Then the specification routines/UI can be a separable piece entirely, written by someone with UI clue. Or have multiple UIs, depending on your needs. Hurray for SOA.

Unfortunately, the experience of iCalendar says that, if your protocol is powerful enough to do everything everyone wants, people won't be able to write reasonable UIs that handle the whole thing, so they'll implement subsets. This wouldn't be so bad, except for the fact that everybody implements a different subset. That's why someone using, say, Notes, can send a meeting invitation to someone using, say, Evolution, or vice versa, and discover that they've used recurrence features that the invitee can't understand.

Maybe iCalendar isn't such a good choice for this problem after all. :-(

Re: Multiple UIs

[jducoeur]

So, so painfully true. Having spent a lot of time writing a scheduler that did data exchange via iCalendar, it taught me a lot about how variable this standard really is. (And how variably *strict* the implementations are. On more than one occasion, Lotus taught me the hard way that my system, which worked just fine with other apps, didn't conform to the spec precisely enough...)

[jducoeur]

It's effectively what cron does, but with multiple users and permissions, and more complex specifications of times and more complex messages than simply "run this".

Even more importantly, it's a *distributed* system. The point is to allow the user to use their own calendaring system -- wherever that is on the Net -- to be their point of schedule management. Hence my focus on protocol rather than technology.

This introduces all sorts of edge-cases of its own (in particular, the protocol has to have clear designs for the edge cases where either the scheduler or target are down during the edge-triggers), but it seems to me that from an ecosystem POV it's the conceptually right way to go.

Hurray for SOA.

Just so. Even three years ago, this architecture wouldn't have (and didn't) occur to me. But in our increasingly service-oriented world, it starts looking very sensible...