Your post suggests a bit of confusion about the meaning of "lifetime" in Rust - a confusion which is common and why we are somewhat unhappy with how that terminology has played out.
Variables don't have a "lifetime," references do. When we talk about lifetimes we talk about the lifetime of references to variables, during which time the variable cannot be moved, dropped, etc. The "lifetime of the variable must be greater than the lifetime of the reference" is a common mental model but this lifetime of the variable doesn't really come into play in reality.
Your 1) and 2) always coincide in the abstract model, though the compiler may optimize out memcpys that have no impact. When you move a pointer around, you don't move the object it points to.
This is sort of the problem: its very natural to refer to that as a lifetime! In fact, more natural than the lifetime of a reference, because we intuitively think of variables as being "alive" and references as being short-term "views" of the variables.
As in the sibling comment, "scope" could be used, but indeed maybe we should have just called lifetimes "scopes" or something (though they are not lexical, whereas scopes are usually thought of as lexical pyramids).
I probably would just say lifetime usually, but I would be being imprecise and potentially unclear!
This touches on some interesting history: lifetimes were originally (in other PL literature) called "regions" (MS Research's Verona also uses this terminology). Lifetime was chosen because an analogy to time seemed more intuitive for this concept than an analogy to space - space analogies being usually used for sections of program memory, rather than periods of program execution.
I don't want to be defensive but this is completely off base, and very disheartening to see you post. Allowing users to write safe self-referential code was never a design goal of Pin - only to make it safe to manipulate self-referential objects generated by the compiler, which it has succeeded at. The incredibly overblown soundness hole had far more to do with the quirks of #[fundamental] than Pin (and the soundness hole is being resolved without changing the Pin APIs at all or breaking any user code).
The guarantees of Pin are now even being used beyond self-referential types but also for intrusive data structures in tokio's concurrency primitives. Pin is not for everyday users, but it is one of the greatest success stories of Rust's unsafe system. That you would call it a mess to a forum of non-users is quite disspiriting.
Sorry, I should have phrased that in a kinder way.
The fact that Pin is expected to enable more than it does is part of the problem. It's not for usual self-referential structs. It's not for usual umovable types. It has 2000 words of documentation, and I'm still not sure what to do with it. I need to understand pin/unpin, structural/non-structural projections, and drop guarantees. It has a lot of "you can do this, but then you can't do that" rules that the compiler can't help with.
For me most Rust features are either easy to understand (like UnsafeCell or MaybeUninit), or I can rely on the compiler make me use them correctly (like borrowing or generics). But with Pin my impression is that it pushes the limits of what can be done without compiler's safety net.
This situation is the sad and absurd result of two processes going on in the Rust community.
The first is that the Rust community on Reddit is in a feedback loop of groupthink and outrage, making it into a powerful vector for harassment. The maintainer of the actix project had a particularly terrible experience of escalating harassment from the users of this subreddit, and its extremely sad. This has caused a lot of grief for the maintainer of actix and prevented real (but far overblown) code issues from being fixed in a productive way.
But I think Steve undersells the absurdity of the unsafe anxiety. Some Rust commmunity members are conflating two wildly different scenarios together. The first is a library exposing an API which, if used in an unlikely and contrived way, could result in a program using that library containing undefined behavior. Then, since that program has undefined behavior, it could contain a memory bug. If that were the case, someone could potentially exploit that bug to attack a user of that program. You'll notice this is a series of conditional statements - its a funnel of decreasing probability.
So yes, library APIs which can allow UB in safe code - even unlikely and contrived safe code - must be fixed. The goal of Rust is that safe APIs can never cause UB. But people should have a proportionate understanding of the risk profile of these bugs (again: a programmer using this API in an unlikely way could create a program with a bug that could potentially be exploitable). This is a miniscule increase in the risk of another heartbleed, it is not the same thing as heartbleed.
The spark that lit the kindling of the toxic Reddit community was a blog post by Shnatsel, a member of the RustSec team. This blog post didn't get attention here, but I want to take a moment to look at how ridiculously it frames things by examining its analysis of a different project: reqwest (sort of the most standard HTTP client library in Rust).
The fuzz test, which is what the library is about, found no security issues. It found some hangs in 6% of cases and Shnatsel traces them to a known deadlock issue. This is a great result.
But Shnatsel spends most of this talking about a custom hashmap implemented in the http library, which the RustSec group did a security audit of just a few months ago. That security audit found only two issues, both UB that would result from a contrived use of a minor library API (they are linked in the blog post, but not explained). These two issues were fixed, and the fix released, by the maintainers of the http crate in 10 days.
This is an incredible success! The security audit of a foundational library found two minor issues which were promptly fixed, and the fuzz confirms that the entire stack on top of it seems to contain no memory issues. Wonderful result, but how does Shnatsel frame this?
> First things first: it didn’t segfault! I am actually impressed because I had really low expectations going into this.
Come on! The blog post is full of these kinds of snide zingers which are totally unfounded in the face of the actual evidence presented. When you have someone writing in this disingenuous, meanspirited way about open source maintainers and then putting this in front of a groupthinking rage machine community like Reddit, of course you're going to get harassment. This behavior is totally unacceptable, and it's very sad to see it promoted in the Rust community.
That article from Shnatsel is really stunning, just everything about is negative and snarky- to a comical point. His avatar is a cartoon pony doing a facepalm! I find it kind of funny that the author of this has singled out reddit for criticism when seemingly prominent members of the Rust community are perfectly willing to be so openly toxic.
Being negative and snarky about crappy, hacked-together code that might cause security issues all over the place is not a bad thing. Yes, most of the snark there is about 'http' crate with its weird custom implementation of basic data structures.
'Crappy' and 'Hacked-together' are literally just insults, they don't carry real technical criticism and frankly they just continue to undermine the credibility of the author. The fact is that 'crappy' code is the most used http crate by far. Clearly everyone in the rust community must be an idiot to be using it. I'm sure the author went off and wrote a totally safe replacement for it. See, now we all get to be dicks to each other rather than actually working to fix stuff.
The first step in working to fix stuff is being aware of the issues - there's nothing "insulting" about that. Unsafe blocks without any clearly-stated rationale and safety analysis is quite subpar quality. Just because it's common and a "most used crate" does not make it OK, leftpad was widely used too.
With all due respect, you are not the universal arbiter of "crappy"-ness and "hacked-together"-ness, nor is anybody else. My guess is that you have concerns about the library, and you are absolutely within your right to say so. Your concerns may even have significant substance to them; or perhaps there is additional perspective that could change your mind.
Unfortunately, presenting your view and judgement as a statement of objective fact not only shuts down potentially valid counter-argumentation, but it's also inflammatory/insultory because it misrepresents your agency in the situation, and diminishes the efforts/personhood of those who contributed to the library.
It's understandable that you might want to use such phrasing as short-hand from time to time, but saying it out loud, especially in a public venue is regrettably demonstrative of a lack of self-awareness. It drowns out your message regardless of its merit, and pushes other people away. It's a reductive way of thinking, and it does everybody (including yourself) a disservice.
In the words of Groucho Marx "I don't want to belong to any club that would have me as a member."
We could all choose be sour and jaded, fighting over scraps in a futile search for self-worth –OR– we could each take it upon ourselves to be the standard-bearers of what was, and hopefully will remain, a warm and inviting place for meritorious debate. I choose the latter.
It is absolutely a bad thing to be negative and snarky. It sews resentment within the community and taints potentially valid points. One can express legitimate concerns and be heard without putting oneself on a pedestal or diminishing the self-worth of others. We as a community must take responsibility for fostering a merit-based discourse, and reject pettiness and snark.
What sews resentment is being repeatedly dismissive of serious problems, as we're seeing with this hackish use of 'unsafe' features - that's an attitude that diminishes the self-worth of others, if I ever saw any. If anything, a bit of irony and snark can be helpful in underlining a sharp, pointed critique, while de-escalating the sorts of petty, antagonizing attitudes that we've seen in this case.
I think I understand what you are getting at. Though there may be disagreement on exactly where the lines should be drawn, I think it’s important to underscore the value of patience on the part of everyone in the conversation.
The rust community has a position on these things, but perhaps it isn’t as well-documented or refined as it could be.
No doubt, having to contend with this issue is frustrating for all parties. I think it’s important however to help people feel heard and respected even if we don’t agree with them. One of the burdens as a pillar of the community is the necessity to rise above that frustration (articulating it in an emotionally intelligent manner when necessary) – to focus on educating, and hearing constructive criticism.
That said, candidly it’s not helpful to talk about using snark as a means of being heard. It’s like saying “I’ll put down my weapons after you put down yours”
Let’s not bury the lede here: people are coming to Rust because they see, and are excited by its potential. They are passionate about making the language better. I think we should hear them, search for patience, and disagree without being disagreeable. We will go a LOT further together through collaboration than we can through divisive rhetoric.
Didn't the Rust stdlib just gain a high-performance hash map recently? Is there a real case for using a custom implementation in reqwest? I could see Shnatsel getting a bit frustrated wrt. that choice.
The reason the `http` crate has a "bespoke hashmap implementation" is not because "performance", it's because _HTTP headers call for different behavior than `std::collections::HashMap`_. Some of the differences in behavior include:
- The map needs to be a multimap.
- Insertion order needs to be preserved.
While `std::collections::HashMap` is an excellent HashMap, it is not a good fit for HTTP headers.
High performance relative to what? In the highest performance code, you will always want your own data structures, ones specialized for your particular use case. Generic facilities will never be quite as good. A general purpose language must support this kind of specialization.
The syntax is different but Rust has this feature- its called auto traits, and its how we handle traits related to threadsafety. Its not a solution for what Pin solves because the whole notion that a type might be moveable or not moveable is actually a misstatement of the problem that Pin is solving.
The name for that reference type is a shared reference. It's literally the default reference type. It's not a sufficient guarantee for the use cases Pin supports.
No one has ever proposed a solution - first class or not - which wouldn't be significantly more invasive and infectious. Does it sound fun in code that doesn't have anything to do with futures to spend time worrying if this generic should be `T: ?Move` or not?
How on earth do you write your own `Future` type without understanding pinning ? Or your own combinator? Or executor ? Or how do you understand the error messages when trying to Box or Rc a future? Or write an `async` abstraction over some C FFI component that you need to use? Or...
If the goal was for users to just use high-level web frameworks, and when there is something that the framework cannot do (which is currently a lot), the expectation is that users should just shrug their shoulders and tell their bosses "Sorry boss, Rust can't do that", then sure, there is no need to understand pinning. But such users do not need to use Rust either, an pretty much any other programming language like Go or Python would have been a much better fit for them.
There is a huge gradient of use cases between "high level web frameworks" and "writing everything from scratch" and we've thought about the impact on users at every step of that gradient. But I'm not about to explain it to you, because you are being disingenuous and aggressive.
And even for the user that only uses high level web frameworks, the moment that user needs to debug its program (which might have a bug in its executor), it needs to understand pinning.
I don't see any use case in which understanding pinning is not necessary.
There's miles between understanding pinning well enough to call `Box::pin` (a safe API) and understanding pinning well enough to call `Pin::get_mut_unchecked` (an unsafe API). When we talk about understanding pinning, we mean the latter.
Since (to a first approximation) every individual who has the expertise and contextual knowledge to really evaluate this issue is a poster on internals.rust-lang.org, its pretty surprising to find this thread on the front page of Hacker News. I imagine some Hacker News users who upvoted this link did so out of technical interest, but I suspect a large portion of the attention comes from some combination of these misconceptions:
- The misconception that this could have a practical impact on users (the code being discussed on the thread is all obviously pathological & contrived).
- The misconception that Rust's type system and standard library never contain soundness issues and that this is an exceptional event (in fact we have a number of longstanding soundness issues).
We have a policy of fixing all soundness issues, so this issue will be fixed. In the meantime, while we decide the best solution, it will have no practical impact on Rust users. And none of the solutions we are considering would involve significant breakage to users, or invalidate any real code.
At a high level: the soundness issue occurs because the Pin API was designed based on certain reasoning about the behavior of pointers. This reasoning would be sound but for the fact that we have allowed certain exceptions in relationship to pointers to what are called the "orphan rules" (which usually enable local reasoning like this). These exceptions allow users to introduce code which, while contrived, allows them to violate the guarantees of the Pin API. Such is life.
I think this is really the bottom line here. This "Pin soundness" matter really seems analogous to the process that any compiler project would go through to identify and fix obscure compiler bugs. I don't think it's quite so deserving of the amount of attention it's getting.
> The misconception that this could have a practical impact on users (the code being discussed on the thread is all obviously pathological & contrived).
Famous last words?
I mean I'm not an expert in Rust and even less in Pin, but I've seen my share of theoretical bugs thought of not possibly having any impact in the real world because of too theoretical. In other areas, when you debug a triple segfault and you understand the crazy conditions that lead to it, or when you render a piece of C++ code conforming instead of technically UB and it then starts to crash when in its UB form it worked perfectly, you start to consider that everything is possible :)
This is exactly the kind of comment that makes me dislike Hacker News so strongly, because I find it endemic here.
The idea that someone could think an issue is theoretical and then discover it is practically significant is obvious and no insight at all - it reduces to "people are sometimes wrong." I am declaring based on my significant relevant expertise that this issue is not practically important. Your comment contributes nothing but baseless contradiction.
Because we didn't see any of this. We can't moderate what we don't see. That's why the site guidelines ask you to flag comments that shouldn't be on HN, and in egregious cases to email hn@ycombinator.com. To flag a comment, click on its timestamp to go to its page, then click 'flag' at the top. (You need karma > 30 before flag links appear.)
Please assume good faith and discuss the topics directly. What is it that you find debatable about my comment? I've always made good faith efforts at productive conversation whether someone agrees or disagrees with me on something
What evidence of bad faith are you referring to? Interesting how you and "other people" going through my comments have no actual objections to the content of what I say. Hard to have a productive conversation when all you have are dismissive remarks and insults.
You go after Jewish people as a class, rather than specific institutions controlled by Jewish people, which I think is bad faith argumentation. If you criticized Israel, for example, or some specific set of banks I would be ok with that.
I admit I would not have the same reaction against another person criticizing “white people” as a class. But I believe I am justified in that because whiteness is not an ethnicity, it is the practice of denial of ethnicity. That makes it a specific institution not a group of people.
I am vulnerable also to accusations of hypocrisy because I sometimes criticize men as a class. Maybe that should be out of bounds, (probably is out of bounds on HN) as men are clearly a real biological group and not a concept.
However, masculinity (as opposed to manhood) is not a biological reality, it is a system of identity constructed to hold childbearing women in a sex class (i.e. submitting to control of their bodies). So to the extent that men identify with masculinity as opposed to just having a penis and some hormones, I would say we are also open to being attacked as a class, and lose our protected status as an actual “tribe” of people.
Although as a side note, I do suspect that when masculinity was originally invented it was quite possibly an identity constructed for the protection of men as an actual ethnographic class of underserved people. Pregnancy does confer actual power and patriarchy I suspect was invented to counterbalance that. However I don’t believe it functions that way today.
I suppose you could argue that Jewishness has crossed that rubicon but I don’t see how you could credibly do without getting into holocaust denial which I would also put in the bad faith category.
Serious apologies to any Jewish people reading this who may feel by engaging these questions I am being blasé about the threat of antisemitism. I really don’t want to do that, but I also want to hold a hand out for people who are having a hard time understanding the reasoning behind the rules of liberal discourse.
With respect, and despite being completely off topic:
> But I believe I am justified in that because whiteness is not an ethnicity, it is the practice of denial of ethnicity.
I fundamentally do not understand what that means. Surely I don't have a choice but to be white? What am I denying? It's true that whiteness is not an ethnicity, since Russians and white Americans are surely ethnically distinct - more so, I think, than white and Asian Americans or white and Asian Russians. But beyond that I can't make sense of your sentence. I wonder if it depends on a particular national interpretation of "white" that cannot be accessed by all people who might want to describe someone as "white". But it seems opaque to me.
>Sure, I will try to unpack my thoughts.
>You go after Jewish people as a class, rather than specific institutions controlled by Jewish people, which I think is bad faith argumentation. If you criticized Israel, for example, or some specific set of banks I would be ok with that.
Just to revisit my original comment you are referring to:
>Yeah it is very peculiar how the British elites was able to establish the Mandate for Palestine to give Jewish people their own state but haven't been able to establish something giving British people their own state even after it was explicitly requested.
I am referring to the state of Israel here, though I don't have a disagreement with Israel's existence nor with the Jewish people themselves in whole or in part, instead I am criticizing that the British elites do seem to have the ability to give a people their own state (as they did in the example of Israel here) however they are reluctant to extend that same work towards the British people. So here my discussion of Israel and Jewish people is not an objection to them in whole or part, but rather an example of what can be accomplished, so as to acknowledge but stave off some arguments that throw Brexit up as if it were somehow not possible. In hindsight, I see how the explanation I gave here would have more effectively prevented any interpretation of animosity, so thank you for making that thought clear.
>I admit I would not have the same reaction against another person criticizing “white people” as a class. But I believe I am justified in that because whiteness is not an ethnicity, it is the practice of denial of ethnicity. That makes it a specific institution not a group of people.
I think I agree about what whiteness is but for different reasons, mainly because the only concrete definition of "whiteness" that I can surmise from how "white people" are treated is "white people are the only unprotected class in law and are the only ethnic group of people ineligible for any race or ethnic protections or ethnic/racially based government programs, academic opportunities, or civil rights protections". Whiteness is imposed on to "white people" by society who say that they can not apply for any protection or benefit based on their white status while all other race or ethnic groups can.
>I am vulnerable also to accusations of hypocrisy because I sometimes criticize men as a class. Maybe that should be out of bounds, (probably is out of bounds on HN) as men are clearly a real biological group and not a concept.
>However, masculinity (as opposed to manhood) is not a biological reality, it is a system of identity constructed to hold childbearing women in a sex class (i.e. submitting to control of their bodies). So to the extent that men identify with masculinity as opposed to just having a penis and some hormones, I would say we are also open to being attacked as a class, and lose our protected status as an actual “tribe” of people.
I think you've brought this up as an example of talking about people in groups and how some groups can lose protected status even if membership in that group is determined biologically. While I agree that there are biological reasons that women and men sometimes need different accommodations in both private and public institutions, if some group is being declared unworthy of "protected status" then I am weary of what you mean by that.
>Although as a side note, I do suspect that when masculinity was originally invented it was quite possibly an identity constructed for the protection of men as an actual ethnographic class of underserved people. Pregnancy does confer actual power and patriarchy I suspect was invented to counterbalance that. However I don’t believe it functions that way today.
That's an interesting analysis, clearly a Marxian perspective in how you tally up what attributes give who power and how social institutions can be counterweights to that power. However if you historically look at the founding of major institutions the fundamental units are the families/communities counterbalancing the power of other families/communities, not sexes counterbalancing each other. Democrats vs Republicans, Labor Union vs Management, Army vs Army, State vs State, Religious institution vs Vice, Religious Institution vs State, Religious Institution vs rival Religious Institution, Academy vs Ignorance, Media viewpoint vs Media viewpoint, etc. etc. etc. The logistics to even support the idea of men and women wandering freely such as stable and safe states plus enough economic opportunity for men and women to wander around freely independently from the families and communities they grew up in is a fairly recent and I think still tenuous phenomenon.
>I suppose you could argue that Jewishness has crossed that rubicon but I don’t see how you could credibly do without getting into holocaust denial which I would also put in the bad faith category.
I don't think Jews should lose any "protected status", instead I am usually point out that Jews are showing us what is possible for a community to accomplish and other groups should be able to follow the same roadmap.
>Serious apologies to any Jewish people reading this who may feel by engaging these questions I am being blasé about the threat of antisemitism. I really don’t want to do that, but I also want to hold a hand out for people who are having a hard time understanding the reasoning behind the rules of liberal discourse.
Why do you feel you should apologize to the Jews for engaging in the free discourse of your ideas? Do you feel they may enact retribution on you? Do you feel the same about other groups like the men you discussed before?
Variables don't have a "lifetime," references do. When we talk about lifetimes we talk about the lifetime of references to variables, during which time the variable cannot be moved, dropped, etc. The "lifetime of the variable must be greater than the lifetime of the reference" is a common mental model but this lifetime of the variable doesn't really come into play in reality.
Your 1) and 2) always coincide in the abstract model, though the compiler may optimize out memcpys that have no impact. When you move a pointer around, you don't move the object it points to.