There are very few categories of people who I would line up against a wall and have shot...

[jducoeur]

... but Russian spammers have managed to make their way into that list.

I just discovered that the Rolls Ethereal have been vandalized -- utterly, meticulously and very intentionally destroyed -- by a spammer. Special. Not going to be simple to recover, either, since the bad data is interleaved with the good in the logs. In theory, I can fix this; in practice, it hasn't been done before, and some of the tools I need don't exist yet (I added the logging facility a long time ago, but not the tool to recover from the logs), so a bunch of time that was going to be spent on wiki-tech instead has to be spent on the Rolls. What fun.

Anybody know a good captcha package for Perl that I can drop in here?

Comments

[herooftheage.livejournal.com]

I take it restoring a backup from a date before but as close to the vandalism as possible is not a viable option?

[jducoeur]

Unfortunately, it wasn't a single incident -- it appears that they originally noticed it a while ago, and then gradually vandalized it over some time, interleaves with genuine changes. So there's no single clean snapshot. (The danger of the Rolls being as well-automated as they are is that I don't need to pay a lot of day-to-day attention to them -- this has probably been brewing over the course of several months, although I suspect that the real wipeout was within the past few days.)

Fortunately, I long ago implemented a detailed logging tool, which tracks all changes. So the theory is that I should be able to edit all of the undesired modifications out of that log, apply it to the last known clean backup, and get a clean state. The only snags there are that (a) the last known clean backup was a *long* time ago, so there's a lot of data to hand-screen, and (b) the tool for applying the log entries hasn't been written yet.

So there's a fair pile of scutwork that needs to be done before I have a clean DB again. Worse, I need to add a good deal of infrastructure to make sure this doesn't happen again, which is going to be a pain in the ass. (Sadly, I think I'm going to have to make the Rolls at least lightly moderated...)