Rust

[jducoeur]

This week's adventure in conferencing is my first trip to ScalaDays, which is in Chicago this year. This morning's keynote was a bit surprising, because it was about the language Rust, rather than Scala. But it was a great talk, and very educational -- I've known vaguely of Rust for a while, but really hadn't known the details. Here's a summary of what I learned, but I recommend checking out the video of the talk once it comes out.

I've been a serious evangelist for Scala for a number of years -- my usual take is that it is currently the best language for general, high-level application programming. You can argue the point, but I'm confident about this one: it's a lovely mix of pragmatism, power and principle, and makes programming more efficient and safe.

But -- not all programming is high-level. Some code needs to be closer to the bare metal, for efficiency, access to the hardware, or other reasons -- it needs to be specifically low level. Scala is only now beginning to be able to do this (with the relatively new Scala-Native compiler), and it's yet to be proven in that environment. Rust, on the other hand, is designed for that world from the get-go.

Or to put it another way, Rust is to C++ as Scala is to Java: a much newer, rethought, more powerful and safe language for playing in that domain.

The core problem with low-level systems programming is that it is scary -- it is very easy to commit any of several major mistakes, each of which leads to crashes or, worse, security leaks. This is true even if you're good at this stuff: programs are complex, and the interactions between the parts are where the bugs tend to arise. Rust is all about reducing that fear, and letting you code with confidence.

The beauty of Rust is that they've taken a very principled look at where those problems tend to come from, and found a few key areas to improve. In particular, the observation is that many bugs arise from uncontrolled access to memory. Plain and simply, pointers are a problem.

So Rust's biggest innovation is removing that word "uncontrolled". It introduces a compiler-time notion of "ownership", and distinct notions of mutable references (which give a code block the right to alter that memory block) vs "shared" references (which allow you to inspect the memory). While they don't use the same terminology, the concepts appear to be quite similar to write vs read locks in database programming.

They've built a lot of infrastructure on top of that, with some really remarkable results. Perhaps most impressive, they've built a concurrency framework that manages to be both flexible and safe. Most of the standard patterns for concurrent programming exist, but they're all adjusted to this ownership-centric world, such that many of the common race-condition problems just can't arise unless you explicitly say "yes, this code is cheating -- I know what I'm doing".

It doesn't solve every problem -- I checked after the talk, and confirmed that it's totally easy to cause deadlocks (unsurprising, given how much this looks like database programming) -- but it's still beautiful and powerful. In the area of concurrent programming, Rust is arguably better than most high-level languages.

Overall, I'm impressed, and I'm pleased to see Rust being presented at a Scala conference -- it looks to me like the languages are nicely complementary. Rust isn't really in competition with Scala: it is optimized for different kinds of problems. But it is principled, and well-designed, in a way that is very reminiscent of Scala. The combination of Scala for application-level programming with Rust for systems and components provides a solid replacement for the older Java/C++ stack.

Not that I've done much systems programming in the past 15 years, so I don't know if I'm likely to use Rust any time soon. But it's good to see the rise of a language that doesn't suck for that domain. God knows, my life 20 years ago would have been much happier with it...

Comments

[metahacker]

How recent is your C++ knowledge? While not as baked in as the Rust support, it has recently (11+) moved ina similar direction with References and const arguments, making explicit what was previously implied. It also allows for tremendous performance improvement, which is cool.

And then they blew it all up with the Napolean Language* that is templating...

(* a Little Language that goes on to conquer the world)

[jducoeur]

I haven't touched C++ since Buzzpad, so we're talking 2002 -- I'm way behind the times, so I can't speak to the similarities.

That said, unless References are *required*, it's a very different viewpoint. The thing about Rust is that memory access is *always* strictly controlled, unless you break out into a block that is explicitly demarcated as "unsafe { }".

(Context: Rust originated with Mozilla, because building a modern browser requires both very high efficiency *and* extremely strong security guarantees. So they decided to first build a language that provided those from the ground up. The new Rust-based browser engine will be rolling up in near-term releases of Firefox.)

So the safety guarantees are much stronger than in an opt-in architecture. I have no idea whether C++ has a similarly "global" mode for this stuff...

[metahacker]

Nope, not baked into the language. So you rely on pre-checkin tests and code reviews, which are faulty. But it's interesting to see the clue ("hey, let's catch these problems at syntax-time, rather than compile- or run-time") get back-ported to other languages.

[hudebnik]

At an academic conference two weeks ago, I gave a talk entitled "Things I Learned at Google That I Wish I had Known When I was Teaching", and the first several slides were subtitled "C++ Got Tolerable While I Wasn't Looking". I, too, hadn't kept up with C++ since about 2002, and C++11 makes a big difference.

You can't add mandatory safety to a fundamentally unsafe language without breaking compatibility. So no, const references aren't required by the language. But they're all-but-required by Google's in-house style: both automated linters and human code reviewers will let you know if you've missed an opportunity to use them.

Likewise, C++11 has "unique_ptr" and "shared_ptr", which implement the idea of ownership you describe, and Google's in-house style strongly encourages using them. Both take advantage of C++'s ability to overload assignment operators and copy constructors to keep track of who has references to a given object. If you have a unique_ptr to something, you own it and are responsible for deallocating it; you can pass a unique_ptr to another function, but in so doing yours is invalidated at run-time, maintaining the single-owner invariant, like a quantum bit. If a unique_ptr variable goes out of scope, it can be safely and automatically deallocated because you know nobody else has a unique_ptr reference to the same object. (Shared_ptrs haven't come up in my work; I gather they CAN be copied, but only in a controlled way that allows for reference counting.)
I haven't used the "delete" keyword or the "free" function in C++ in years: I just let my unique_ptrs go out of scope.

I had some trepidation about my talk title, expecting people in the audience to think "He didn't know THAT? What a maroon; we're all better off with him out of academia." But at a couple of points during the talk I asked for a show of hands to indicate who was familiar with (and teaching) a particular concept, and almost nobody knew about unique_ptrs. Or mock-based testing and dependency injection, but that's another story.

[jducoeur]

you can pass a unique_ptr to another function, but in so doing yours is invalidated at run-time, maintaining the single-owner invariant

Ah, hang on -- this is invalidated at *runtime*? That doesn't surprise me, but it's a crucial difference: in Rust, it is invalidated at *compile* time -- if you pass an unshared ref to a function, any attempts to reference it after that function call result in a compile error. References are more a compile-time constraint than a runtime one, so that you get fast feedback with little runtime overhead.

That's the thing about building this ownership stuff deeply and fundamentally into the language: they're able to provide *really* strong, compile-time guarantees based on it. I'm not clear whether C++ has managed to achieve that yet...

[hudebnik]

No, and I'm pretty sure it never will; almost any way to do that would break millions of lines of legacy code.

Note: I said "C++ got *tolerable* while I wasn't looking," not "C++ got clean and elegant while I wasn't looking" :-)

That said, I suspect the run-time overhead of passing a unique_ptr as opposed to an ordinary pointer is minimal: the compiler can tell at compile time that this is what you're doing, so it generates an extra "set pointer to null" instruction, which isn't much of a price to pay on top of the parameter-passing and function-calling you were already doing. The real loss in doing it at run-time rather than compile-time is informative and early error messages.

There are also disadvantages to the compile-time approach. For example,

unique_ptr blah = MakeUnique(3);
while (complex condition that never happens) {
foo(std::move(blah));
}
// is blah valid here?

A compiler has to assume, pessimistically, that the condition actually happens at least once, and answer "no". A run-time library can wait to see whether foo is actually called on blah before invalidating it, and answer "yes".

[jducoeur]

Interesting point -- I have no idea whether Rust tries to do anything clever here, but I agree that it seems like it has to assume that the pointer is invalidated. That said, I suspect that's correct: any code that tries to be horribly clever about it is fairly suspect to begin with. But I'm speculating wildly here...

[laurion]

Rust is under heavy use over at Mozilla. Makes sense. You want a strong closer to the metal language when you are building an interpreter for other languages and markups. And you want something as safe as possible when talking about interpreting random other people's stuff.