RE: https://social.rust-lang.org/@rust/117605013073050966
I'm not sure whether I'm more excited about the stable never type, or DropGuard in std.
Which is a reminder that I'm kind of strange, since the word "allocator" didn't appear in that sentence. 🤪
@cliffle allocators are runtime type checking and we would be better off without them. (In system programming)
@xentrac I have not worked with it but have been following it. It looks interesting.
@Moore I haven't tried installing seL4 either, but I mean, what do you think about its approach to resource management, in the abstract?
@xentrac I don't know enough to have an option but it looks interesting. The time accounting is interesting.
@Moore An interesting thing about that is that they only enable the time accounting for "mixed-criticality systems", where you're relying on the kernel to prevent your Linux virtual machine or whatever from causing glitches in your motor control waveforms or 802.11 acks.
But the main interesting thing about the kernel in this context is that it doesn't allocate memory at runtime. There's no heap.
@Moore How do you envision statically checked memory management working? Are you satisfied with Rust?
@xentrac the easiest way is to statically allocate everything but that can lead to more allocations then strictly necessary if there are two allocations that never overlap in use.
Another idea is to have an allocator as an explicit argument and add capacity to the type of the allocator and an API like:
let (value, allocator) = Box::new(allocator);
This way it would be a type error to over commit.
@xentrac Another idea is that if you have something like an event loop you can require all functions to be total and then use static analysis to compute the worst case allocation which would be easy if you used a bump allocator in the event loop. (Long lived allocations would need to be moved to statically allocated structures before returning out of the event loop.)
Really the root idea is no computer has infinite memory, so why do we write programs like they do.
@Moore
I think the basic answer to "Why do we write programs like the computer has infinite memory?" is a little complicated.
In what I think you're calling "system programming", we write programs because we know what they will output, and we want them to output it reliably, without failing. For example, firmware for a motor controller. In those cases it is very bad to write programs like the computer has infinite memory, because it means those programs will fail sometimes, and we don't want them to fail.
Other times, though, we write programs because we *don't* know what they will output, and we are running the program to find out. Like, if you already have the output that a compiler will produce in a given case, there's no point in running the compiler on the same input again. In those cases, it's very helpful to be able to pretend that the computer has infinite memory, because usually we have a pretty good plan for what to do if the program fails.
This dichotomy is more glib than the reality, of course, and there are lots of complexities and fuzzinesses, but I think these two general attractors do sort of exist in the real world.
@Moore I don't think so, because some problems have small input but require a lot of working memory, and some are by nature incompletely computable.
@xentrac but you can still report that error clearly instead of having your process killed for using too much memory.
@xentrac my other strong opinion is that computers have finite resources, we should write software that accepts that instead of running to failure.
@Moore we should be friends
@xentrac To push back on this a little more. If you use bounded data structures when programming it forces you at each step to ask "what do I do if this operation exceeds capacity". You can then either return an error or rethink your algorithm to avoid the case.
If you take the current approach of "use more resources" you end up with crashes, missed SLAs, and resource exhaustion attacks.
@Moore Yes, I agree. But consider, for example, the problem of calculating one cell on the billionth row of the Rule 30 elementary cellular automaton with the canonical seed. The input is pretty small, something like (30, {0}, (1000000000, 0)), and the output is a single bit, but, as far as anyone knows, you need on the order of a billion bits of state to compute it. Something like Hashlife may be able to cut down the order of growth of that number, but I'm guessing that you're still going to be up in the hundreds of megabytes even with Hashlife, and Hashlife makes the resource usage less predictable, not more. (5emel intends to allow you to budget for that kind of resource usage, but there are currently a lot of holes in its implementation of that budget, and probably inevitably the result of picking a budget that is too small is that your runtime goes exponential.)
Generally speaking, though, for programs in my second category of "programs we write because we *don't* know what they will output", crashes and missed SLAs are no big deal, because we have a pretty good plan for what to do in that case.
The difficulty arises when we try to program the whole computer system in such a best-effort fashion.
@xentrac I would say for this case just returning with "no solution found" would be what I would do instead of crashing.
An example of this approach is SMT solvers like z3 which return sat, unsat, or 🤷♀️.
@Moore Yes, sometimes that's absolutely what you want! And sometimes it isn't.
@xentrac I guess we just disagree on whether there are times when it is not a good idea.
@xentrac my hypothesis is that with advances in programming languages we can get ergonomic programming environments that produce programs that cannot crash outside of hardware faults. You could give them input that is too large to process but that they would always return with an error that explains what would be required to succeed.