Shared mutable state is evil. So functional programming languages thought: let's have no mutable state. So Rust thought: let's have no sharing of state.
Rust is not a functional programming language. It is a different approach to the same problem.
I appreciate the distinction, but I believe I disagree with allowing mutability on variables without explicit statements.
In particular, I want to be able to ensure that the only variation is the variation I explicitly state - for single threaded or for multithreaded code, I don't care.
Having blobs of mutable state being passed around in a single threaded program is also a huge source of bugs. It is NOT a good idea. I don't have time today to tease out the implications of this proposal, but if it allows structs to be passed around and modified en-route in the same thread by default, just like C, then Rust loses a great deal of appeal. Sorry.
Global mutable state in Emacs Lisp has been a nightmare whenever you're trying to make multiple libraries that are still being updated play nice together. Still single-threaded, still ick.
I'd go a step further and say I don't want side effects that aren't explicit either, but oh well. That's why I use Haskell :)
"that the problems with data races and memory safety arise
when you have both aliasing and mutability. The functional
approach to solving this problem is to remove mutability.
Rust’s approach would be to remove aliasing."
This is the crux of the argument and it makes sense to me.
I can't help but wonder if mutability and aliasing are secretly the same thing. For example, we can't optimize memcpy by reordering reads if the source and destination arrays are aliased. But if we had a hypothetical language where arrays can be mutable or immutable, we could require the source array to be immutable and the destination array to be mutable. Then we'd be able to reorder reads just fine, because writing to the mutable destination array can't affect the immutable source array!
fn mutter( x : ~int ) -> ~int {
let mut y = x;
*y = 24;
return y;
}
fn main() {
let y = box 42;
let x = mutter( y );
println!("x:{}", x);
}
That's currently legal, but we mutate the box with 42 in it
in a function that doesn't declare itself mut, because it has the only reference to the box, it can freely start mutating it, and then give it back at the end.
Interesting. However, in main(), the variable `y` can no longer be referenced, since it would fail the borrow checker. Doesn't that effectively mean that it is immutable, as it won't be referenced any longer?
Perhaps things are different in rust, but in C assignment to something which ought to not change (either by the = vs == typo or some higher level cognitive error) is not too uncommon source of bug.
Being explicit about what data should be immutable or not as a way of expressing intent helps eliminate these bugs. In rust, the mutability is analyzed strongly by the compiler and is important for the ownership rules, so I'd expected the use of the mut flag to be more successful than const is in C / C++.
> For a language that's supposed to have its 1.0 release later this year, these sure sound like pretty significant breaking changes.
And that is why I think Rust is one of the most exciting new languages out there. They have not been stubbornly sticking to their original plan when they found out that it wasn't really a good idea after all.
For example the change from N:M thread scheduling to native threading.
The language and the compiler have been serving as pretty nice research vehicles for advancing practical and useful programming language and compiler design.
Don't get me wrong, change to make things truly better is probably worth it. But I'm not convinced that's what we're really seeing with Rust any longer.
Many of the recent changes seem to be what I'd call "oscillation".
So a problem is noted with some specific functionality or feature of an existing programming language. A different approach is initially taken when implementing something similar for Rust. Yet problems are found with this new approach, so a different approach is tried. And problems are found with this, so something else is tried.
The matter discussed in the article appears very much to be yet another example of this.
Continually searching for some vague "optimal" solution seriously impacts the usability of the language, especially when there might not even be a solution with satisfactory trade-offs. It's disappointing when code written a week ago suddenly needs significant updates due to language and library changes, especially when the language is supposedly "stabilizing".
Rust doesn't exist in a bubble. It's facing some stiff competition from languages like C++11, C++14, Go, Scala, and even Java. Maybe they aren't as "perfect" as Rust aims to be, but they offer many of its core benefits, and they let people get work done today, while still undergoing improvement. Rust, on the other hand, appears to be stumbling around, searching in vain for some vague notion of "perfection", while being unusable to real-world developers who need at least some stability.
You are not convinced, but most Rust users are convinced. I guess time will tell.
I started using Rust from Rust 0.5 (December 2012). I saw zero oscillation so far. From what I can see, Rust is converging, rapidly.
Consider this: you can observe these at all because Rust is developed in the open. I think you would have similar reaction if you could observe Go development done behind the closed door before the release.
Actually, there are a few things in which Rust has recently gone back to old behaviour (though “oscillation” would be a bit strong as a description of it). Nested block comments is one that was removed a year or two back as unnecessary and reinstated a few months ago. I can’t think of any of the bigger ones off the top of my head; calling them oscillations could also be a bit strong too, as they tend to end up subtly different when they come back—different and superior.
I think to be "oscillation" they'd have to change their minds three times or more. chrismorgan in this subthread mentions "Nested block comments is one that was removed a year or two back as unnecessary and reinstated a few months ago.
That's trying out something both ways, and after experience deciding one is best. That they changed their minds twice just means their initial first cut turned out to be "right" after all.
Now, if they're doing X -> Y -> X -> Y that would be cause for concern.
I think your concern for "stiff competition" is perhaps overblown, I only see C and C++ as being a real competitor, I don't see Go or anything based on the JVM as being even in the same category, you're a lot further from the metal. Personally, after doing plenty of C++ in the '90s I've decided it's no fun at all (a supremely dangerous language), am defaulting to C at the moment and will at some point decide if Rust meets my requirements and is likely to be viable for a long time. I gather I'm not the only one.
I draw a different conclusion from the "stiff competition". Your conclusion is along the lines of "my god! they need to hurry up and finish it or they'll lose the future!", whereas mine is more "what's the rush? there are good alternatives right now, and there will always be more code that needs writing in the future".
By the way, it is actually very possible to write real software in rust already - none of the changes in the last 6 months or so have really required all that much surgery to get up to date. Even this change sounds like it would have a fairly straightforward update path. The only big frustration for me is the lack of a mostly-implemented and "blessed" package manager. I'm watching progress on that front with much more interest than these language changes.
Yeah, in some sense, these are all just endgames. To me, the last hard change was @-removal and interior mutability, which happened in October and November 2013. So your "last 6 months" comment is exactly correct.
Yeah, I guess a more recent biggish one was having to re-do a good deal of vector usage as those apis moved around and changed. But I would say my recent experience has been >90% crate moves and renames, which are really easy to deal with, especially with the right mindset of "I'm using pre-alpha software".
I also agree with whoever it was (you?) who pointed out elsewhere that the only reason people are even able to gripe about this stuff is that the development has been way more collaboratively open than other languages. That openness has its pros and cons!
"Oscillation" is definitely the wrong word---I haven't seen any real instance of reverting to previous behavior without a good reason and a general consensus to settle on the one behavior. Although this mut thing has me nervous.
On the other hand, at least as far as I can see, "convergence" is also not a particularly good description. Admittedly, the code I have played with was written to exercise Rust's unique features, but going from 0.5 to 0.9[1] I got hit with a fair number of major changes. Take the M:N -> 1:N threading thing (and no, I still have very serious doubts about the long-term stability of "let's do both!"); that's almost completely transparent most code, but it's still the only significant difference between Java and Erlang.
[1] Yeah, I really need to find some time to catch up again.
Well it can have all the breaking changes it wants, since it's not 1.0 yet. Which is kind of the point - show a unified, logical interface to the programmer that still covers memory safety without mandatory GC, concurrency without data races, speed, functional programming.
Most of the changes that have happened in the last year have been either to simplify the language, or to remove superfluous features. This is a good thing, especially for a language that is carving out a new domain. By the time 1.0 is out, the developers will be confident that the language is good, and they'll be less likely to change it down the line.
With this (though I know it's still just proposed) and the unique pointer change[1], it appears Rust's periodic table of types[2] (often cited as a sign of over-complexity in the language) is being heavily simplified!
I think this is a good idea and something that has been badly misunderstood by probably the majority of the community for a long time. Consider the following code in an unspecified language:
var int x
x = 1
x = 2
print(x)
x = 3
Clearly, "x" is a mutable variable. However, in isolation, this is not really true. We've long known about "Single static assignment" (SSA, [1]) and the fact that modulo memory issues, this ss equivalent to the above:
var int x1 = 1
var int x2 = 2
print(x2)
var int x3 = 3
(Obviously the next thing the compiler will observe is unused variables. Bear with me on that so I can keep the code samples simple. :) )
So in the context of a single simple executable context like this, the distinction between mutable and immutable is actually meaningless. Again modulo possible memory consumption issues, there's nothing you can write that will actually distinguish between "being in an immutable environment simulating mutability" (post-SSA transform) or "being in a mutable environment" in this single, small execution environment. If you can not distinguish between the two, then there is no difference.
Let me highlight this again. The existence of this transform is not merely an interesting trivia bit that happens to make optimizing compilers easier to write. It is a fundamental mathematical statement about the relationship between mutability and immutability at this particular scale. This is an important truth.
When does mutability matter? Well, maximally pathologically, with something like this:
var int x
x = 1
spawn_a_new_thread(fun() {
x = 2
})
print(x)
Now what happens? Obviously now we have a way of "witnessing" mutability vs. immutability (at least on a probabilistic basis).
Less obviously, in a mutable language with references mutability still allows "action at a distance", where a function holds a reference to some value X, calls another function with Y, and doesn't "realize" that function will also as a side effect update X. In theory, you could transform this to a purely immutable representation, but in practice, this is the same problem as in the threading case... lighter weight since now it's deterministic and that makes it much easier to deal with than the threading case, but it still explodes the complexity of the program.
The crime of "mutability" is not actually the mutability per se. (After all, under the hood it's all just RAM anyhow... if mutability was fundamentally bad, we'd have already lost!) The crime is performing mutations on the context of a function without the function being aware of it. Threading makes it much worse, but even in a single-threaded context this can produce disaster. So the goal should be to ensure that functions can be assured that all the context they know about won't change without their knowledge and/or consent (to continue my anthropomorphization). More mathematically, it's really hard to prove (formally or just to yourself) any particular invariant in an environment where really any time you release control (single threaded) or even just whenever (multithreaded) things you "own" may be changed out from underneath you... in the worst case, composite data structures can change while you use them! The function becomes practically nondeterministic, and that's hard to work with.
A way to do this is by ensuring that everything is immutable, but for this particular purpose that's overkill. Ensuring that only things you uniquely own can be mutated works just fine... you own it, you "know" you changed it, and you've proved that nobody else can be surprised by any changes, even within the same thread, then you're back where we started at the beginning... you're a simple transform away from "immutable", and despite multithreading, the function is again deterministic.
Erlang has always struck me as suffering from this particular badly... the coding style often ends up with Value1, Value2, Value3 as we perform transforms on some value, but Value1 is immutable so we can't "update" it, but even if we could, it wouldn't matter because there's already no references in the language, so there's already no way to send these things across processes. The immutability is really just an annoying side show [2], what matters is that the language has strictly value-only semantics for inter-process communication. It makes the language significantly more annoying to program in for what is basically no gain, because back in the 90s this confusion about what immutability is really for was quite prevalent.
This is the key to writing simple, yet robust code; for a function to know that once it starts, there are no surprises. Whatever state it witnesses at the beginning will not change out from underneath it, so we need not continuously be screwing around with locks or fearing what other code may have references to our values.
Incidentally, true pervasive immutability becomes important if one is also lazy. You can't have a thunk in such a language that may end up either one thing or another, "depending". The nature of a "execution context" in such a language is very different and the line between functions becomes much fuzzier, especially with multithreading permitting true nondeterminism of thunk evaluation order. In a strict language, though, I think immutability is a side-show, what matters is ownership and change visibility.
[2]: And yes, I also know that "=" is not "assign" but "pattern match with binding", but we could trivially add a := "pattern match and re-bind" operator to the compiler and the VM would never even have to know.
'You can't have a thunk in such a language that may end up either one thing or another, "depending".'
Well, you can, and on a very rare occasion that's what you want (see "amb"), but you should be explicit that that's what you're asking for (and probably why) because those occasions really are pretty rare and usually pretty complicated/subtle.
Simplifications, of course. :) See also "lazy IO" and the joys of "seq" and all the manifold subtleties it has.
But I'll stick to my original claim, modified by "in general"... in general, nondeterministic thunks (where "nondeterminism" means "influenced by things that are not immutably set at the function's start of execution" or something like that) is dangerous and almost (but not quite) always wrong.
Yes, it has no effect on "correctness". The two are equivalent. But your example is nonsensical. If you scale this up to non-trivial problems I have a hunch we'd find this pervasively affects the design losing the ability to compose functions. It implies an ordering on the "dependencies/ownership" of the program.
Once you write your program, if a program is fully SSA you are right it is equivalent to an immutable one. However this affects how one designs the software in the first place. You would not write the code the same with the different tool-sets.
I have struggled with this because working in Haskell taking Rusts approach seems very appealing as it allows mutability. But every time I just come to the conclusion that it is a bad idea as it will end up greatly affecting modularity and composability.
To give an example. If I have a pure function A, which now calls function B which mutates a value, function A is no longer pure. Now you lose the ability to compose that function in many ways. This would happen all over the program.
If I'm reading your post correctly, what you're calling "nonsensical" is deliberately part of my point; you may want to re-read my post in light of that fact. The difference between a local context and a fully-composed program in a conventional mutable language is a core part of my argument, not something I overlooked.
What I am saying is mutability is not something that tends to stay local. Either you isolate it at the top, ala Haskell or it tends to find it's way into everything.
SSA is great, except that as you say it is functionally equivalent to immutability. There is no advantage to it over using immutable data structures. In fact, it is inferior as it only covers a subset of problems that immutability does.
The examples in the blog post illustrate this. He wants to make local variables mutable because otherwise he can't pass a reference to a function which mutates a value. That mutation is completely unnecessarily and could of been performed with a pure function instead.
Note the blog author says: "Think: when was the last time you responded to a compiler error about illegal mutation by doing anything other than restructuring the code to make the mutation legal?"
I actually do this all the time. That is probably the largest challenge coming to a language like Haskell from an imperative background. Sometimes you can't just fix the little mutability problem locally, it has to be redesigned with immutability in mind.
I think what you're actually talking about is really
the desire for Referential Transparency[0] vs. the mutable/immutable distinction.
I certainly agree that, to some degree (as long as you have some way of guaranteeing referential transparency) then the default for local variables being mutable/immutable doesn't really matter. The important thing is the RT bit, and if Rust can guarantee that (via no-aliasing or linear types or whatever) then it's already ahead of the bunch in terms of safety.
Of course mutability can prevent algebraic reasoning and trivial reuse via simple algebraic extraction of code/functions, but so can strictness, so...
"I think what you're actually talking about is really the desire for Referential Transparency[0] vs. the mutable/immutable distinction."
I don't think so... in the arrangement I'm talking about nothing stops you from getting input from an arbitrary source. I suppose this implies there should only be one witness to the change at a time, but that's not all that hard of a restriction.
"Of course mutability can prevent algebraic reasoning and trivial reuse via simple algebraic extraction of code/functions, but so can strictness, so..."
Indeed... if we're going to write off a class of useful things, let us make sure we get as much benefit from it as possible. And I can't imagine Rust dropping default-strict anytime soon... in the domain it seeks to conquer it is arguably the correct choice even from a Haskell perspective. Haskell lacks that truly low-level systems orientation.
Input from an arbitrary source (i.e. without referential transparency) implies that there can be no equational reasoning, almost by definition. This is kind of the case for IO in Haskell[0], but for subprograms/procedures/functions I tend to find it an absolutely essential property, both in terms of reasoning and in terms of testability. For example, it's the whole reason that QuickCheck works.
[0] Modeling IO as a World->World function doesn't quite work.
The functional programming guidelines I've seen (including, for example, Paul Graham's "On Lisp") have in effect said that if you need to mutate local variables you own, that's no big deal, as long as you can encapsulate that within one function. As far as consumers of that function are concerned, it is "pure," regardless of the fact that it uses mutation within its definition.
That much seems fairly obvious to me, but I'm somewhat new to functional programming. Is that all that is at stake here?
I was part of the Reddit discussion linked in the article, and although I haven't had the time to think through the issues (nor do I have the same experience as the Rust dev team), I'd like to offer some thoughts.
First, I am a big fan of functional programming, coming from OCaml and Haskell mostly and not so much experience writing software that needs to be as fast as possible. This definitely colours how I judge a language: all other things being equal, I will prefer a language that is closer to ML than to C. In most functional languages, if you want to a name to refer to a mutable cell, you need to declare it as such (let x = ref expr for OCaml, some sort of state monad in Haskell). I have grown to prefer to have as many of my variables be immutable as is practically possible: it feels like there are less moving parts in the program that can go wrong. This is a reason I was happy with Rust's initial design choice. I also quite enjoy the fact that you need to be explicit about your intent to have a variable refer to a mutable cell and that immutable is the default, unlike in languages like C or Java.
pcwalton asked what bugs immutable local variables help prevent, and honestly, I can't think of anything non-trivial. I can imagine some scenarios where a developer would get a compiler error instead of weird runtime behavior, for instance:
let mut i = 0;
let j = 0;
while i < 10 {
some_processing();
j += 1; // oops! meant i += 1;
}
If `let mut` is removed from the language, this piece of code would compile, but I suspect that it should be relatively fast and easy to spot the bug and fix it. However, as a programmer, I like that a simple keyword can help me reason about code without having to inspect a larger portion of the program (i.e. scan the block containing the definition to see if the variable is modified).
Another thing that immutable-by-default local variables seem to encourage (at least, they encourage me) is to approach problems in a more functional programming oriented way. For instance, let's say you want to find the x value such that f(x) gives the greatest result. You can use regular imperative programming:
let mut max = 0;
let mut max_result = 0;
for x in range(0, N) {
let y = f(x);
if y > max_result {
max = x;
max_result = y;
}
}
println!("f({}) = {}", max, max_result);
But maybe you'd look at the two mutable variables and think "maybe there's a higher-level way of doing this..." and you end up with:
(Hopefully I didn't make too many mistakes in the code.) Perhaps this style of programming doesn't need any encouragement, but for me, when I see mutable variables, I pause and consider for a moment if I couldn't perform the task in a manner that would be more declarative and less error-prone.
When I think about it, I can sort of see that it makes sense that mutability be linked to uniqueness: most problems with mutable variables comes from the fact that they can be modified in surprising ways. Though I must ask: if a reference can be mutated if you have the only reference to it, could there be cases when you want to have the only reference to a cell and still be unable to modify it (i.e. &mut const)?
Finally, I appreciate the practical implications for the compiler writers; I am part of the Sable lab at McGill where we work on a MATLAB compiler, and the crazy edges cases of Matlab are really a headache when doing any work in the system, and I can definitely understand wanting simpler semantics. If we want Rust to be a safe language, having a semantics that is simpler to implement and verify is definitely something important.
With that said, I'll re-read Niko's post (some parts, especially about closures, were not clear on my first read), and I hope that although this issue seems to be raising some passions, in the end the community and the dev team will figure out what works best for everyone and Rust will be stronger for it.
I do like the comfort of being able to nail-down local things so that they don't move, as it were. The encouragement it provides is subtle, but i do appreciate it (especially on a large team, where colleagues span the full range of the talent spectrum).
I think it's worth noting that even a very shallow notion of immutability would provide enough comfort for me. Compiler writers could call it a "syntactic blanky".
Something along those lines shouldn't be mutually-exclusive with the current proposal, local reasoning being sufficient. Something like 'final' or 'const' (or, better still, replacing the 'let' keyword with 'val' and 'var' as in Scala.)
> This is the same problem as the Env example above: what’s really happening here is that the FnMut trait just wants a unique reference, but since that is not part of the type system, it requests a mutable reference.
Wait, what? The closure `|| * errors += 1` wants a unique, but NOT a mutable reference to it's environment, so that it can mutate the `errors` variable?
I guess that using `only`/`uniq` instead of `mut` would definitely be clearer and less confusing. Although I don't understand why they want to remove local immutable variables, which IMO have nothing to do with "immutability", but rather with "rebinding".
For a reference to be mutable, it has to be the only outstanding reference to an object. You can create as many immutable references as you want, because the object can't change. On the other hand, if you have the unique reference to an object, there's no particular reason you shouldn't change it.
Rust's mutability is more than rebinding, though. If you want to change a value of a field in a structure, which is something that doesn't involve rebinding the variable, you have to have a mutable variable holding the structure. The mutability is "inherited" from the variable.
Well, it doesn't mutate the `errors` variable. `errors = &mut someothervar;` would be illegal there. It dereferences a pointer (that itself is immutable, this doesn't matter as long as it's unique) to a mutable value and mutates that value, like it says on the tin.
I like the idea of &only as it greatly simplifies an understanding of the borrow checker. However, I think we still need to be able to mark things as immutable for things like rodata stuff, mapped in rom, persistent data structures, etc. And if we're going to be marking mutability, I would prefer immutable to be the default. It probably matters more for function arguments than locals, though.
if the compiler says something must be mutable, that
basically always means I forgot a mut keyword somewhere.
(Think: when was the last time you responded to a
compiler error about illegal mutation by doing anything
other than restructuring the code to make the mutation
legal?)
Shared mutable state is evil. So functional programming languages thought: let's have no mutable state. So Rust thought: let's have no sharing of state.
Rust is not a functional programming language. It is a different approach to the same problem.