I find async is so much fun in Python and meshes with the other things you can do with generators but that is because I have the reference collector cleaning up behind me.
Looking back with like 30 years of hindsight it seems to me that Java’s greatest contribution to software reuse was efficient garbage collection; memory allocation is a global property of an application that can’t efficiently be localized as you might want a library to use a buffer it got from the client or vice versa and fighting with the borrow checker all the time to do that is just saying “i choose to not be able to develop applications above a certain level of complexity.”
The problem with garbage collection is that it doesn't work for other kinds of resources than memory, so basically every garbage collected runtime ends up with an awkward and kinda-broken version of RAII anyway (Closeable, defer, using/try-with-resources, context managers, etc).
Static lifetimes are also a large part of the rest of Rust's safety features (like statically enforced thread-safety).
A usable Rust-without-lifetimes would end up looking a lot more like Haskell than Go.
> basically every garbage collected runtime ends up with an awkward and kinda-broken version of RAII anyway (Closeable, defer, using/try-with-resources, context managers, etc).
RAII works only for the simplest case: when your cleanup takes no parameters, when the cleanup doesn't perform async operations, etc. Rust has RAII but it's unusable in async because the drop method isn't itself async (and thus may block the whole thread if it does I/O)
In my experience async drop is a nice to have, not a must. Futures::block_on is good enough for the happy path in low stakes scenarios.
When dealing with async operations they tend to end up at a network boundary and thus the service enters distributed system land.
Now the async drop also has to handle the server crashing before the drop happens and at any time when it happens. Keeping that in mind trying to actually drop something becomes quite meaningless since you need to handle all other cases either way.
I agree but does RAII necessarily imply parameter-free destruction?
Personally I love Rust’s `fn foo(self, …)`, which is just like a regular method but consumes the value.
Deallocate by default is fine, but sometimes you need to run specific destructors (linear type style). I’ve long wished for an opt-out from implicit drop semantics for resource/handle types.
You can (kind of) emulate linear types by making `drop()` unlinkable[0]. Of course, I wouldn't recommend doing this since the error messages are awful and give no explanation at all of where the actual problem is...
People miss that there is a quantitative aspect to these things as well as a qualitative aspect. That is, many programs allocate millions of pieces of memory a second and for a wide range of different purposes whereas there might be a limited number of other resources of an even more limited set of types. Any change in the code probably has some affect on memory allocation, but many changes won't have any effect on allocation of higher-level resources.
Thus the complexity of handling memory is greater than that of other resources and the consequences of getting it not 100% right are frequently worse.
I quite like context managers and try-with-resources style constructs. They make lifetimes explicit in the code in a fairly intuitive way. You can get yourself turned around if you deeply nest them, etc but there are usually ways to avoid those traps.
You make an interesting point. Has any language introduced a generic-resource-collector? You're not supposed to use deconstructors to clean up resources because you're left to the whims of the GC which is only concerned about memory.
Has anyone build a collector that tracks multiple types of resources an object might consume? It seems possible.
Erlang is probably the closest. The word you want to search for is "port". If it doesn't seem like it at first, keep reading. It's a very idiosyncratic take on the topic of you view it from this perspective because it isn't exactly their focus. But it does have a mechanism for collecting files, sockets, open pipes to other programs, and a number of other things. Not fully generic, though.
Python handles all kinds of stuff with garbage collection.
The problem is that things like closing a socket are not just generic resources, a lot of the time nonmemory stuff has to be closed at a certain point in the program, for correctness, and you can't just let GC get to it whenever.
I don’t think this is true. Context managers call special magic “dunder” methods on the instance (I don’t remember the specific ones), and I’m pretty sure those don’t get called during regular garbage collection of those instances. It’s been a few years since I was regularly writing python, so I might be wrong, but I don’t believe that context manager friendly instances are the same as Rust’s Drop trait, and I don’t think their cleanup code gets called during GC.
Python is a fun case of "all of the above" (or rather, a layering of styles once it turns out a previous one isn't workable).
Originally, they used pure reference counting GC, with finalizers used to clean up when freed. This was "fine", since RC is deterministic. Everything is freed when the last reference is deleted, nice and simple.
But reference counting can't detect reference cycles, so eventually they added a secondary tracing garbage collector to handle them. But tracing GC isn't deterministic anymore, so this also meant a shift to manual resource management.
That turned out to be embarrassing enough that context managers were eventually introduced to paper over it. But all four mechanisms still exist and "work" in the language today.
Are you saying that a finalizer is guaranteed to run when the last reference is deleted? So you could actually rely on them to handle the resources, as long as you are careful not to use reference cycles?
In CPython 2.7, yes. In CPython in general, I believe it's currently still the case, but I don't think it's guaranteed for future versions.
For Python in general, no. For example, as far as I know Jython reuses the JVM's GC (and its unreliable finalizers with it).
It's also easy to introduce accidental cycles. For one, a traceback includes a reference to every frame on the call stack, so storing that somewhere on the stack would create an unintentional cycle!
The tracebacks were a lot of what made me cut back on using weakrefs and trying to make things manage their resources automatically.
Now I use close() methods for anything that needs to be closed. If I mess up and there's some obscure bug, hopefully GC will fix it, but it seems too brittle and easy to make mistakes with to rely on.
I agree that memory management can't be solved locally. The situation in C++, where every library or API you use has a different cleanup convention, that you need to carefully read about in the documentation to even properly review a pull request, is proof of that.
I disagree that this criticism applies to Rust. For 99% of the cases, the idiomatic combination of borrow checking, Box and Arc gets back to a unified, global, compiler-enforced convention. I agree that there's a non-trivial initial skill hurdle, one that I also struggled with, but you only have to climb that once. I don't see that there's a limit to program complexity with these mechanisms.
>The situation in C++, where every library or API you use has a different cleanup convention, that you need to carefully read about in the documentation to even properly review a pull request, is proof of that.
Lol wut. The C++ resource management paradigm is RAII. If you write a library that doesn't use RAII, it's a bad library. Not a fault of the language.
There’s a lot of C++ code out there and a lot that interfaces with C.
RAII is one method of cleanup but it doesn’t work in all situations. One that comes to mind is detecting errors in cleanup and passing them to the caller.
So it’s not right to call every library that doesn’t use RAII “bad.” There are other constraints, as well. Part of the strength of C++ is to give you a choice of paradigms.
Either you write code with good performance, which means that functions do take references and pointers sometimes, in which case you do have all of the usual lifetime issues. This is the proper way to use C++, and it's perfectly workable, but it's by no means automatic. That's the reality that my comment was referencing.
Or you live in a fantasy land where RAII solves everything, which leads to code where everything is copied all the time. I've lived in a codebase like this. It's the mindset that famously caused Chrome to allocate 25K individual strings for every key press: https://groups.google.com/a/chromium.org/g/chromium-dev/c/EU...
You're missing a bunch of very important stuff in that page you linked to. See what they listed as the culprits:
> strings being passed as char* (using c_str()) and then converted back to string
> Using a temporary set [...] only to call find on it to return true/false
> Not reserving space in a vector
c_str() isn't there for "good performance" to begin with; it's there for interfacing with C APIs. RAII or not, GC or not, you don't convert to/from C strings in C++ unless you have to.
The other stuff above have nothing to do with C++ or pointers, you'd get the same slowdowns in any language.
The language has come a long way since 2014. Notice what they said the solutions are:
I'm responding to a comment that claims all lifetime issues are solved by RAII.
My argument was that for efficient code, you need to pass references or pointers, which means you do need to care about lifetimes.
And your argument is that's not true because we now have std::string_view? You do realize that it's just a pointer and a length, right? And that this means you need to consider how long the string_view is valid etc., just as carefully as you would for any other pointer?
> I'm responding to a comment that claims all lifetime issues are solved by RAII.
I don't see anybody claiming this. The parent I see you initially replied to said "the C++ resource management paradigm is RAII", not "all lifetime issues are solved by RAII".
> My argument was that for efficient code, you need to pass references or pointers, which means you do need to care about lifetimes.
Of course you do. Nobody claimed you don't need to care about lifetimes. (Even in a GC'd language you still need to worry about not keeping objects alive for too long. See [1] for an example. It's just not a memory safety issue, is all.) The question was whether "every library or API you use" needs to have "a different cleanup convention" for performance reasons as you claimed, for which you cited the Chromium std::string incident as an example. What I was trying to point out was:
> that's not true because we now have std::string_view? You do realize that it's just a pointer and a length, right?
...because it's not merely a pointer and a length. It's both of those bundled into a single object (making it possible to drop them in place of a std::string much more easily), and a bunch of handy methods that obviate the ergonomic motivations for converting them back into std::string objects, hence preventing these issues. (Again, notice this isn't just me claiming this. The very link you yourself pointed to was pointing to StringPiece as the solution, not as the problem.)
So what you have left is just 0 conventions for cleanup, 1 convention for passing read-only views (string_view), 1 convention for passing read-write views (span), and 1 convention for passing ownership (the container). No need to deal with the myriads of old C-style conventions like "don't forget to call free()", "keep calling with a larger buffer", "free this with delete[]", or whatever was there over a decade ago.
> And that this means you need to consider how long the string_view is valid etc., just as carefully as you would for any other pointer?
Again, nobody claimed you don't have to worry about lifetimes.
I agree that a lot of that happens in the real world. I disagree that RAII is not used in the real world. I worked on a very large codespace for ATM client software and we used it pervasively, and the only memory leak we had in my time there was in a third-party library which ... required the careful reading of documentation you mentioned.
widely used though. not sure if that count for appreciation, but i think it's one of the highest forms.
it's not bad, not not great either. i miss proper sum types, and it really lament the fact that static things are nearly impossible to be mocked which prompts everyone to use DI for everything instead of static.
Java has sum types now with sealed interfaces and pattern matching. Records have detructoring out of the box, and I believe supporting it for general classes is in the works.
sealed interface Shape {}
record Square(int x) implements Shape {}
record Rectangle(int l, int w) implements Shape {}
record Circle(int r) implements Shape {}
double getArea(Shape s) {
// Exhaustively checks for all alternatives.
return switch (s) {
case Square(var x) -> x * x;
case Rectangle(var l, var w) -> l * w;
case Circle(var r) -> Math.PI * r * r;
}
}
Loved the article. Thanks for sharing. Maybe I (coming from Haskell and Elm) should not be bothered too much with the verbosity of Java/Kotlin "sum types" :)
The only Java thing I work with is ElasticSearch (and in the past other lucene based search tools like Solr). These can be resource hungry depending on what your indexing but they are also faster and more scalable than other tools I’d used before.
Looking back with like 30 years of hindsight it seems to me that Java’s greatest contribution to software reuse was efficient garbage collection; memory allocation is a global property of an application that can’t efficiently be localized as you might want a library to use a buffer it got from the client or vice versa and fighting with the borrow checker all the time to do that is just saying “i choose to not be able to develop applications above a certain level of complexity.”