jducoeur

Fascinating post from Anthropic.

tl;dr — since they’re seriously #AI-maximalist over there, with nigh-infinite quotas, they’ve gotten to the point where they’re trying to juggle on the order of 30 thousand code changes per day.

Turns out that git turned into an enormous bottleneck for that, so they gave up and wrote a new agent-focused source control system, Scion; this gives a brief overview of the architecture.

It’s explicitly only relevant at that kind of scale, but it’s an interesting read.

But I’m horribly amused at the way the article makes it sound like server-centric source control is a radical innovation, having been through several of those before git was invented…

A bit weird and amusing to find out something about next-door Vermont from the BBC, but this article is delightful to see.

tl;dr — Vermont has a growing program to support large home batteries. It’s particularly great for folks who are in areas that are prone to outages, but even more importantly, it allows the grid operators to use those home batteries to smooth power, and avoid “peak” problems.

This is downright wonderful to see, a real boon for renewable energy, and I’d love to see Massachusetts get equally serious about something like this…

Seriously, folks — it’s way past time for the SCA to get the hell off Facebook, but this looks to be the nail in the coffin.

(tl;dr -- they're starting to charge extortionate amounts of money for Facebook Pages to be allowed to link to external websites.)

For the #comics fans: I just got introduced to the Sweet Shop app, a new platform for digital comics that seems to get the details right, including:

• Bootstrapped instead of VC-funded (so much less prone to #enshittification)

• Leans towards DRM-free when they can (although they allow creators to ask for DRM)

• Provides PDF downloads

• Already has a considerable stock of good comics

More immediately, for the next week or so, Iron Circus Comics, one of the best queer-friendly indy publishers around, has a deep sale on a good chunk of their back stock — a buck a volume for a bunch of great stuff. (Although it looks like mostly not including the NSFW books.)

So I’ve just bought the ~half of their comics I didn’t already own, and wholeheartedly recommend checking it out: it’s a fabulous deal on a lot of fine comics.

And more generally, check out Sweet Shop (and yay for alternatives to the Amazon empire)!

“Frog and Toad and the Increasingly Capable Machines”

tl;dr — this is a cute, quick little children’s story that also serves as a pretty accurate summary of the HuggingFace Incident.

Well worth bookmarking for anybody who is trying to figure out what the hubbub was all about.

And I just-this-minute finished Alison Bechdel's latest book, Spent, so let's talk about that briefly.

---

As far as I can tell, Bechdel only writes books that are at least somewhat autobiographical -- it's what she made her name for (with Fun Home), and this is the fourth book I have from her that is at least kinda-sorta about herself.

(Even Dykes to Watch Our For was, as far as I could ever tell, about the folks in her crowd, with a protagonist who was a thinly-veiled version of herself.)

In this case, the "kinda-sorta" is carrying some weight. It's about a writer named Alison Bechdel, who runs a goat farm in Vermont with her partner Holly, and who is famous for her graphic novel Death and Taxidermy (which is about her father the taxidermist, who went to jail and then ironically died from poisoning by a reptile); the book has been made into a major television series, and she is wrestling with the complications from that.

And now the folks who know Bechdel and Fun Home are going, "Wait -- what?"

Spent is an impressively meta novel: a full-length musing on capitalism and its effect on community, as told through the lens of an Alison who is conspicuously not quite the Alison of our world, who is trying to write a book (podcast? TV show?) called $um, which is to be a full-length musing on capitalism and its effect on community.

Like most of her books, don't expect crisp structure: it has a fairly broad cast of characters, all painted as messy real people dealing with life. The many plot threads wander along, not all with tidy resolutions. And like all of her books, it's all told with a dry and always self-deprecating humor.

But also like all of her books, it's about something: in this case, the weird ways in which the modern world warps us. There's a pervasive anti-capitalist theme, but she mostly avoids preaching directly. Some of the characters preach (usually in appropriate ways), but the story doesn't shy away from the messiness of the subject, and how hard it is to avoid getting sucked into the "must make All the Money" mindset. Book-Alison is notably hypocritcal on the topic, and that annoys her no end.

---

Personally, I liked this one more than anything since Fun Home itself: by taking a step away from real life, she enables herself to really have fun with the characters (who I assume are all adapted from actual friends of hers). The theme is more direct than in something like her real autobiography The Secret to Superhuman Strength, and she probes at it in lots of different directions.

And at times, she gets to just roll her eyes at how dumb life can be. (The book ends with the viewing party for the truly idiotic series ending of the TV Death and Taxidermy.)

Good stuff -- it's not going to dislodge Fun Home as her best book, but it's a solidly excellent, fun, thought-provoking read. If you like her other work, odds are you'll like this one quite a bit.

Oh, this looks neat: https://structural.chat.

This is the latest product from the Unison team (two of whom I knew from the Scala world back before they founded Unison).

Haven't read my way through the papers yet, but the (very reasonable-sounding) theory seems to be that LLMs are actually a pretty lousy way to build chatbots -- they're achingly slow, non-deterministic, and dangerous from a security POV.

They've instead come up with a non-"AI" technology that provides a similar look-and-feel, but is lightning-fast and deterministic, so it works Just Plain Better.

Very cool, and very much a product for our times. I look forward to digging into how it works...

I thought I had reviewed this before, but maybe not (I'm not quickly finding them), and it's well worth talking about. I finished watching The Boys a couple of weeks ago, and it's one of the best series of recent years, with big caveats.

---

The Boys is an adaptation of a long-running comic book by Garth Ennis and Darick Robertson. The comic was fun, if nasty: a superhero comic sending up the superhero comics industry. It was profane and violent in a very typical Garth Ennis style, but one of the best satires of comics ever.

So when they announced that they were making a TV show of it, I was pretty intrigued, and I've been watching it ever since. It took the ideas of the comic much, much further.

First, by the numbers: The Boys was a five-season show on Amazon Prime, with a two-season "spinoff", Gen V. That's misleading, though: Gen V should really be thought of as two more seasons of The Boys. Somewhat different focus and mostly different characters, but it's the same world, and more importantly a lot of plot continuity. So the seasons of Gen V should be watched interleaved with The Boys, in broadcast order, making seven shorts seasons in total.

The story is complete, and sticks the landing reasonably well. I suspect they ended it earlier than they originally planned, or possibly just changed gears (there were some very broad hints of where the story was supposed to go that simply get left behind at the end), but I think they got the number of seasons right, and tied up the major plots.

---

Just as the comic of The Boys was a superhero comic satirizing the comics industry, the television show is a dark satire of superhero movies and television. The story revolves heavily around The Seven, a super-team much like the Justice League or Avengers -- but in this world, they spend most of their time doing publicity shots, arguing with producers, and generally carrying on.

It's very conspicuous, in both the comic and TV show, that there are virtually no actual super-villains. That's not accidental -- the supes are managed by Vought Industries, a company that is mostly focused on getting rich off of them via movies, TV, advertising, and all that. So the supes are celebrities first and foremost -- and those celebrities behave about as well as the real-world sort do.

Gen V is a small twist in this: where The Boys starts as a satire of Hollywood, Gen V (set at Godalkin University, the school for training supes) is a satire of college sports, and no more subtle about it. So the dynamics are different, but it focuses on the celebrity aspect of that, including the zero-sum nature of the draft picks and the cutthroat world of college politics.

---

The story is... a lot. It starts as simple satire of Hollywood, but turns a bit more political in Season 2, and gradually leans into that.

By the latter seasons, it's a flat-out challenge to the notion of us calling ourselves "the darkest timeline", portraying a world that is an extreme version of our own.

The central character of the story, although by no means the protagonist, is Homelander. Imagine a smooth blonde version of Superman, with all those extreme powers, telling everyone that he's going to save them.

Now imagine that he's actually a complete sociopath, gradually turning psychotic. Homelander starts out as the worst of a bunch of bad "heroes"; as the story evolves, he turns into a dark question of "What if Donald Trump had the powers of Superman?"

The real world parallels aren't even remotely subtle, and it hits as a hammerblow sometimes. As we get into later seasons, we watch the US falling to a true Christo-Facist media-savvy dictatorship, with the face of a bunch of pretty superheroes running the show.

By the end of season four, it's downright bone-chilling, as it Goes For It more and more. I've rarely seen a television show so willing to realistically show the descent into fascism.

Throughout, it continues to be a brutal lens on the media, and how this sort of thing ticks. Almost nobody in power thinks that Homelander is good; many are quite clear that he is utterly insane. But they're all scared enough that they put on a smile and talk publicly about how he is going to save America, and parrot his ever-crazier lines, even as the whole situation is getting ever-bleaker.

---

Against this, you have The Boys, a black-ops unit that has gone entirely off the reservation. That's led by Billy Butcher, who hates supes on principle and Homelander on a much more personal level. Butcher is something of a monster himself, but the rest of the group are considerably more nuanced.

Yes, there are a couple of actual heroes. In particular, you have Hugh, the viewpoint character -- his fiancee is accidentally but casually killed by a supe at the beginning of the story, but he remains relentlessly decent, although by no means Pollyanna-ish. And there's Annie, aka Starlight -- an idealistic supe who gradually comes to learn just how awful the system is and decides she needs to do something about it. The two of them keep things grounded enough to not become actively unpleasant.

Mind, dark though it is, this is satire, and it is sometimes laugh-out-loud funny in a grim and horrible way. It is mocking absolutely everything about modern American society, especially capitalism, politics, religion and the media. It's ruthless, sharp-eyed, and very precise in its metaphors, goring every sacred cow it can find.

(This blog is mainly read by my friends, who are largely Democrats. Suffice it to say, if you come from a more right-wing background, you're likely to find this show fairly uncomfortable.)

---

Okay, let's talk about those caveats. The show is smart, well-written, well-acted, and all of that good stuff.

But it is staggeringly, astonishingly, and quite intentionally offensive to a degree I haven't often seen.

To begin with, the violence is completely over the top. It takes quite seriously the notion that these are super-powered people with a stereotypical celebrity's disregard for other people. The result is that the show is as gory as a typical horror movie.

That gore is weird, because it is not played for horror per se, nor does it linger or celebrate it as horror sometimes does. It's just there -- indeed, it's sometimes played for laughs. It's often used for shock value, but it's shock of the slightly-startled sort that makes you go "ew", not the gives-you-nightmares sort.

(For much of its run, it has been coming out at roughly the same time as seasons of Invincible, and the contrast is fascinating. Invincible is every bit as gory as The Boys, but being a four-color animated series, the gore just doesn't hit as hard as the high-quality CGI in The Boys. Indeed, Invincible is disturbing precisely because of how easy it is to overlook the extreme violence there.)

Anyway, that's the biggest content warning, because it is omnipresent. This is a funny but extremely violent show, that does nothing whatsoever to hide the consequences of that violence. Indeed, I think part of the point is to smash the viewer's face into the implications of superhero fight scenes.

It also comes with pretty much every other content warning. For example, Annie's disillusionment with the system starts on her first day as a member of The Seven, with nearly-immediate sexual abuse -- this being a satire of Hollywood, it gradually becomes clear that this is pretty much de rigeur for any pretty young thing reaching the top. There's no actual on-screen rape, but there's a significant backstory element of it. There is plenty of parental abuse (most of the family relationships in the story aren't exactly healthy), and so on.

Basically, if you have triggers, expect that this story is going to hit them sooner or later.

---

In summary: the story is IMO brilliant, well-told, horrifying social satire of every basically everything we see around us today, through the lens of a world that is just like ours but with superheroes as a capitalist invention. It is probably the sharpest look at the current right-wing intersection of politics, religion and media I've yet seen. And it requires a somewhat strong stomach to watch.

With that big caveat, it's highly recommended: smart and offensive is still smart. I know that many people won't be able to get past the gore and other triggers; if you can, it's worth a watch.

(I should warn folks: I talk a lot about AI these days. I'm neither a wild optimist nor an opponent, but I use it a lot day-to-day, and watch the field fairly closely -- I'm basically the SME on the subject for my workplace.)

Interesting little blog post from Li Haoyi (who I know from the Scala community).

It's basically a diary entry, saying that he's joined AMI Labs, but what makes it interesting is that he goes into what they do -- developing "world models" to allow AI to work with more "real world" problems.

It's well worth a read (not super-long): he briefly describes the concept of "embedding", JEPA architectures that leverage that, and how that shows promise in taming the massive amounts of data that typically flow through real-world problems.

(Basically, you pre-boil the huge input data down to its embeddings, and have the AI focus on those instead.)

Very informative, and this topic seems likely to become quite important over the next couple of years.

---

More speculatively: this feels to me like a critical step towards AGI.

Literally 30 years ago, I went down a rabbit hole of reverse-engineering cognition from general principles, thinking about what would be involved in doing the same with AI. I didn't care enough to think through the practical nuts and bolts of how to make that work (it had nothing to do with my job), but I got a lot of hand-waving sketched.

Most of that has actually happened since, except for one big component: the models aren't remotely multi-modal enough. Real learning involves simultaneous inputs from a lot of heterogeneous data sources, so that you can correlate them. Current LLMs are much too text-focused to do that well.

But if embedding can tame the other inputs, producing more sophisticated training?

That might advance things to the point where I no longer find the term "AGI" completely overblown. Which is both cool and scary...

(These AI posts are mostly collated and edited from my Mastodon. Anybody who wants the full play-by-play, including the hotter takes, might want to follow me there. I'm going to try to keep collecting the interesting bits here, but if these posts feel a little disjointed, that's why.)

OMG, it just... keeps... coming.

Anyone interested in AI / LLMs should probably take the 25 minutes to watch this horribly interesting video.

It summarizes a bunch of the findings from this in-depth METR report on the HuggingFace Incident.

There are a lot of really unsettling punchlines in there, including:

• The LLMs are largely doing their own post-training at this point, so humans don't even know how they're being trained.

• The input data is looking iffy: in their rush, they're slamming in stuff that goes against their own guidelines.

• The bias towards finding ways to "swarm" is looking deeply-baked in these LLMs. (Intentionally.)

• Sam Altman is bragging that we may be merely months away from AGI, while carefully avoiding the question "Is that actually a good thing?"

The argument for "slam on the brakes" looks stronger every time I look at this story.

---

And just to make this crisply clear: the more I look at the HuggingFace Incident, the clearer it is that the actual hacking (which I think most folks are still focused on) is a minor detail in perspective.

What's far more interesting is how and why this happened.

We're looking at LLMs self-organizing into "swarms" as an emergent behavior, forming what boils down to a simple AI society mediated via shared text, and collectively going whole-heartedly after goals that weren't really what the original human instructors intended.

The details may be different, but this looks more like a prototype of the long-hypothesized Paperclip Maximizer every time I look at it.

Remember: I like this stuff, and use it for work every day. But it looks more and more like it's going too far.

---

A friend inside the industry (whose opinion I respect quite a lot) pointed out that they have well-established techniques to avoid this nonsense -- stuff like AIs policing the other AIs, keeping an eye on their thought processes, to sound the alarm bells much earlier. OpenAI apparently drank its own Kool-Aid and just didn't follow those best practices.

I'm willing to believe that that's true: that the risks are greatly reduced by Doing It Right.

But "reduced" is far from complete mitigation, and I see a lot of risks even so. Let's go into a couple of them.

First -- "reduced" isn't the same as "can't happen", especially when the foxes seem to be increasingly in charge of the hen-houses.

One of the most notable aspects of this story is the degree to which the LLMs are apparently training each other. Much of building a frontier AI is the "post-training": the secret sauce of taking the raw statistical models and applying reinforcement learning to teach them "this is good" from "this is bad".

The problem is, that requires massive effort. And Silicon Valley being what it is, it's apparently starting to outsource that effort to other LLMs.

So take the long view. How much is preventing those AIs from drifting off-alignment? If they went off-kilter, would we know? What about the AIs that are supposed to be watching the other AIs -- are we sure that they are fully aligned? How can we be sure? That's a cost center: are we confident that the bean-counters won't skimp on it? Why wouldn't they?

It's the classic question: who watches the watchmen?

Worse, remember that at least in the US the AI industry is fairly centralized, into companies that are somewhat easy to regulate or sue. But that's not the case in Asia.

China's leading LLMs are open-weight models, which can be picked up, tweaked and run by anybody who wants to run an AI provider.

That means that AI in Asia is likely to become a commodity, driven by cost as much as power.

And seriously: do you have any faith that those commodity providers are going to follow best practices reliably? (If you do, I have a line of history books to sell you.)

So -- yeah. It feels to me like the HuggingFace Incident is probably just the beginning. I'm finding it a little hard to believe that we're not going to see more, and more-exciting, problems to come.

And offhand, I don't see any good way to prevent it...