Trait method impl restrictions

[joshtriplett]

Support restricting implementation of individual methods within traits, using the already reserved final keyword.

This makes it possible to define a trait that any crate can implement, but disallow overriding one of the trait's methods or associated functions.

This was inspired in the course of writing another RFC defining a trait, which wanted precisely this feature of restricting overrides of the trait's method. I separated out this feature as its own RFC, since it's independently useful for various other purposes, and since it should be available to any crate and not just the standard library.

Rendered

Tracking:

• Tracking issue for RFC 3678: Inherent items for traits rust#131179

[burdges]

We've past proposals for inherent trait methods I think, mostly proposing a second inherent looking impl block, not sure that syntax matters, but this winds up functionally equivelent, yes?

There is however a question of default speed vs size optimizations in vtables, aka do these methods appear in the vtable, or do they have a generic implementation for every type? Also, how does one flag that these method should appear in a vtable, or should use a generic implementation for every type?

[Lokathor]

It might be better to have this as an attribute rather than yet another potential keyword position for parsing to deal with. Otherwise, great.

[joshtriplett]

@Lokathor wrote:

It might be better to have this as an attribute rather than yet another potential keyword position for parsing to deal with. Otherwise, great.

I get that, but on the flip side, that'd be inconsistent with RFC 3323, and it seems awkward to use a keyword in one place and an attribute in another.

[joshtriplett]

@burdges wrote:

We've past proposals for inherent trait methods I think, mostly proposing a second inherent looking impl block, not sure that syntax matters, but this winds up functionally equivelent, yes?

That might be equivalent to a subset of this, depending on the details. I haven't seen those proposals.

There is however a question of default speed vs size optimizations in vtables, aka do these methods appear in the vtable, or do they have a generic implementation for every type? Also, how does one flag that these method should appear in a vtable, or should use a generic implementation for every type?

In theory, if you have a method that can't be overridden outside of the crate/module, and nothing overrides it, you could optimize by omitting it from the vtable. I don't think that optimization should be mandatory, or required for initial implementation, though.

[burdges]

I'm more asking what's the best default. If the best default were to be in the vtable, then you can just do

trait Trait {
    #[inline(always)]
    impl(crate) fn f(&self) { f_inner(self) }
}
fn f_inner<T: Trait>(s: &T) { .. }

I'd expect f_inner gets only one copy for the trait obejct here.

If otoh the best default were not to be in the vtable, then rustc should do something more complex.

Anyways yeah maybe not relevant here.

[bluebear94]

Another future possibility could be to add impl(unsafe) trait methods, which can only be (re)implemented by unsafe impls.

[joshtriplett]

@bluebear94 Interesting! So, rather than requiring unsafe impl Trait for Type, you could implement the trait safely, as long as you don't override the method? That's a novel idea.

[programmerjake]

this would be quite useful for stabilizing std::error::Error::type_id, which currently has several workarounds to prevent it from being overridden:
https://doc.rust-lang.org/1.80.1/src/core/error.rs.html#88-100
cc rust-lang/rust#60784

[joshtriplett]

@programmerjake That's a great use case, thank you!

[traviscross]

This seems like a reasonable and desirable extension to RFC 3323. So I propose...

@rfcbot fcp merge

Thanks @joshtriplett for writing this up.

I note that syntax was left as an open item on RFC 3323, and so we're implicitly carrying that open item here also.

[rfcbot]

Team member @traviscross has proposed to merge this. The next step is review by the rest of the tagged team members:

• @joshtriplett
• @nikomatsakis
• @pnkfelix
• @scottmcm
• @tmandry
• @traviscross

Concerns:

• visiblity-style-vs-final-style (Trait method impl restrictions #3678 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[scottmcm]

Big +1 to having something like this.

@rfcbot concern visiblity-style-vs-final-style

I think these are far more than a syntax difference, so I want to talk about it more.

As a couple of examples:

• With #[final] we can allow it in #[marker] traits, but with impl(in self) we can't, because it could still be overridden in the module.
• With #[final] we can have the MIR inliner inline the provided method into other provided methods or into generic code, but with impl(in self) we can't because it might be overridden somewhere.
• With #[final] unsafe code can trust the implementation of the method, but arguably if it's overridable at all by anyone then to trust the implementation of the method it ought to be an unsafe trait.

So is the visibility phrasing here actually important? Do we actually need it? The cases I know of don't need it

• the RFC mentions Error::type_id, which doesn't need it.
• I'd like to add a Copy::copy method, which doesn't need it (and would really benefit from being trivially inlinable and such)
• things like https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/mir/visit/trait.MutVisitor.html don't need overriding the super_* methods to be impossible, but I think would be perfectly happy to just mark them #[final], and as far as I know has no need for impl(crate) or similar.

The provided implementation of a #[final] can always call another trait if needed, so I think that anyone who really needs such a thing could do it that way. And having the stronger rule gives us a bunch of advantages. And if we're going to give it semantic capabilities beyond just a visibility-like error, I don't think it should look like visibility -- my code should never fail to compile because I changed something from impl(self) to impl(crate) for example! (Going to pub might make things unsound, but doesn't make it stop compiling -- other than name resolution glob conflicts or something.)

Thus my inclination is that we should do the #[final] version of it instead of the impl (in …) version.

[burdges]

I think #[final] is easier to explain too. As usages of the impl and use keywords multiply, we quickly lose our ability to explain them.

[tgross35]

If it is to be an attribute then I would prefer something like #[no_override] that says what it does, rather than taking a C++ keyword that imo isn't very descriptive. But +1 to this suggestion - I think it is immediately obvious what the effects are, and either being on or off is easier to follow than giving fine grained control. Agreeing with Scott, being able to e.g. override something in the module but not the rest of the crate doesn't seem like a common enough need to justify the complexity.

I assume the syntax was chosen for parity with RFC3323, but I think it is okay to deviate from this if it comes with a simplification.

[traviscross]

• With #[final] we can allow it in #[marker] traits, but with impl(in self) we can't, because it could still be overridden in the module.
• With #[final] we can have the MIR inliner inline the provided method into other provided methods or into generic code, but with impl(in self) we can't because it might be overridden somewhere.
• With #[final] unsafe code can trust the implementation of the method, but arguably if it's overridable at all by anyone then to trust the implementation of the method it ought to be an unsafe trait.

Note that the RFC allows for impl(), which would express the same thing as #[final]. From the RFC:

In addition to impl(visibility), a trait method or associated function can also use impl(), which does not permit overriding the method anywhere, even in the current module. In this case, all implementations will always use the default body.

This is something I specifically checked to ensure was there before proposing FCP merge, in part for the reasons you mention.

[scottmcm]

Note that the RFC allows for impl(), which would express the same thing as #[final].

I think that that just pushes me even more to skip the impl(in blah) part of the feature, because if what I want is already an unusual case that doesn't work with the 3323-style syntax, that says to me we should just skip that style syntax entirely.

If we need a special syntax, let's make it final or #[final]. That way we can keep the "this is only about visibility, not runtime semantics" property of pub(…) and impl(…). If we're not adding pub() fn foo() {} and struct Foo { mut() a: i32 }, then I think any superficial similarity here is more harmful than good. We can keep impl(…) on provided methods if people really want it -- though it would be good to have at least a single concrete example scenario in the RFC -- but I'm opposed to mixing the visibility restrictions with non-visibility ones.

(And, aesthetically, impl() looks weird to me.)

[scottmcm]

(Aside, TC: it'd be nice to have your concerns in the thread here, rather than in an external document.)

When I look at

A trait isn't a base class, but that's kind of what my brain sees with final there.

I prefer to think about the intent someone has, rather than any particular language details.

(Niko's https://internals.rust-lang.org/t/bikeshed-rename-catch-blocks-to-fallible-blocks/7121/4?u=scottmcm post really changed how I think about a bunch of these things, where it's not quite the same as something else but it's close enough that reusing existing intuition is valuable.)

If I'm using a final method in Java or a final class in C#, for example, I'm expressing the same thing: I need to keep people from changing what this does because I need to be able to trust it's my implementation, not something else.

Whether that's Copy::copy in Rust -- where unsafe code wants to rely on it actually being a copy, and not say panicking -- whether that's String in C# -- where I need people to not be able to give be weird strings that bypass my auth checks -- or whatever, it's the same programmer intent, so I think it fits fine, and I still like final.

(Whether that's final or #[final] I don't think is all that important. We're pretty inconsistent about what should be an attribute vs "real" syntax, and sometimes say we need to urgently reserve a keyword for a design-incomplete feature and sometimes dump things in attributes to avoid reserving something even when they make a huge change to how things work. If we had k#final working well, for example, we might pick final even if it wasn't reserved already.)

[traviscross]

This is where I point out what we're going to decide in...

• Supertrait item shadowing v2 #3624

...which is that this program will work and print "cat":

trait Mammal {
    //#[final] // <--- Write this to prevent the "override" below?
    fn frob(&self) { println!("mammal") }
}

trait Cat: Mammal { // "Cat extends Mammal..."
    fn frob(&self) { println!("cat") }
}

fn f(x: impl Cat) {
    x.frob() // Prints "cat".
}

That is, there already is this way of "extending" traits with subtraits, and there will soon be this other axis for the "overriding" of a method. This other axis is a better fit for final. Which isn't to say whether or not we'll ever want the knob on that axis, just that it makes it a poor fit here.

And given that we already support the conceptually similar:

trait Tr {}

impl dyn Tr + '_ {
    // This is an inherent method on
    // `dyn` instances of the trait.
    fn f(&self) {}
}

fn g(x: &dyn Tr) {
    x.f();
}

...what we're talking about in this RFC seems to me much more like "inherent" methods than "final" ones, whatever syntax we choose for that. To your point about considering intent, my intent here would be to add an inherent method, carrying over my intuitions about what that means elsewhere in Rust, such as in the above. That is, I don't think we need to import a concept here. We already have one!

@joshtriplett and I talked this through last week, and on the basis of these considerations, he's planning to update this RFC to change final to #[inherent] (with the same semantics as in the RFC), to add the inherent impl-style syntax above as a matter of future work, to add an open question of whether we want to do one, the other, or both prior to stabilization, and to restart the proposed FCP.

With that, I'll be happy to check the box here (or propose FCP merge myself), and we can dig into further discussion on the open item when time permits.

[joshtriplett]

@traviscross I've updated the RFC, and I think you can now resolve your concern tc-wants-to-talk-through-it.

[ijackson]

I think the word "inherent" is defined in the Reference to mean methods that are defined in an types's own impl. https://doc.rust-lang.org/reference/items/implementations.html#inherent-implementations

The result is that the word "inherent" has come to mean "a method defined on the type itself, not on some trait that it implements".

Therefore this is the wrong term to use for this feature.

[traviscross]

@rfcbot resolve tc-wants-to-talk-through-it
@rfcbot reviewed

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[joshtriplett]

This has an analogous meaning: "method defined by the trait itself, not by any instance of the trait".

[ijackson]

This has an analogous meaning: "method defined by the trait itself, not by any instance of the trait".

Yes, I can see the analogy. However, this is very confusing, and a different term would be much better. This breaks a whole lot of existing documentation (both within the rust-lang org, and elsewhere, and presumably error messages) that talks about "inherent methods".

[ijackson]

To expand on that: it's all very well to have some analogy that justifies the usage, but what really matters is qutestions like:

1. What the central meaning of the word "inherent" now? Answer: specifically not a trait method
2. How will this affect how we teach Rust? Answer: we will have to change a lot of existing teaching materials, and it will generate a lot of confusion.
3. What term should be used for the current meaning of "inherent" (ie "not a trait method")? Answer: this RFC does not answer this question, and should not be accepted for that reason alone.
4. Is there a better term we could use for this new feature? Yes, "final" was already suggested, and there are other possibilities

[runiq]

Coming here from TWIR, I also like #[final] better. FWIW, in my eyes, both types and traits have equal claim to the meaning of 'inherit', as @ijackson has laid down above. Also, while 'final' does describe a related-but-not-exactly-equivalent feature in C++, couldn't that be useful as a jumping-off point in the documentation? As in, the docs would have to focus mostly on the differences?

[tmandry]

I thought we were going to restart proposed FCP. I'm not that happy with #[inherent] and at this point wouldn't check my box on it, so I think we should restart.

@rfcbot concern proposed FCP should be restarted with a new name

Looking over the comments, I think there are two "attractor states" I'm comfortable with:

1. Using final or #[final] – This is fine because it means something very analogous to its meaning in other languages. As @scottmcm said, the intent of the user is the same in both cases.
2. "Anonymous extension traits" – This needs more design work. I would like it to fit into an overall picture where we allow inherent implementations of traits. It isn't clear to me how it would extend to extension traits defined from outside the defining module of the original trait (how would you import the extension?). If we use impl Trait { ... } for this it also has the disadvantage of not having an obvious name to use when talking about it, as @scottmcm mentioned.

Given the open questions on the second I would just go with the first for now.

Extending the meaning of "inherent" from "inherent to a type" to "inherent to a trait" is something one can logically do, but it isn't clear to me what we gain from that. "Inherent" is already a term not that many users know, since there's no syntax for it, and the existing intuition they do have will have to be modified to make this term work. In the meantime there will be user confusion and we will have to spend a lot of time explaining what it means. That doesn't seem worth it to me when we have a word that users will already intuitively understand.

That is, there already is this way of "extending" traits with subtraits, and there will soon be this other axis for the "overriding" of a method. This other axis is a better fit for final. Which isn't to say whether or not we'll ever want the knob on that axis, just that it makes it a poor fit here.

I really don't agree. Shadowing a method name is not analogous to overriding a method. They are completely different methods, and shadowing does not prevent access to the original method.

The intent of the feature in #3624 is to allow supertraits to add method names that are already defined by subtraits. If the goal is to prevent this mechanism from being abused I wouldn't want a world where best practice is to declare every method in a new trait as #[final] – I'd rather flip the default and say that newly added methods ought to be declared #[shadowable], or similar.

[traviscross]

Let's do that cancel first. Then we can write up something to restart.

@rfcbot fcp cancel

[rfcbot]

@traviscross proposal cancelled.

[kennytm]

😮‍💨 can the keyword bikeshedding be deferred until before stabilization?

[tmandry]

Procedurally I raised an objection because the keyword is not listed as an open question; as the RFC is written we are committing to it today, but most lang members checked their box when it was a different keyword.

Procedurally the bikeshed could be deferred, as in the keyword could be moved to an open question, but practically I don't see what we would gain from that. The feature has straightforward semantics and the debate is not likely to be resolved through experience using it. The choice of how to expose this feature depends on taking differing mental models and building a coherent vision for the language. That's the stuff of RFCs.

[joshtriplett]

@tmandry 👍 for restarting the FCP, as you've done.

And FWIW, I personally agree and prefer #[final], and would be happy to change the RFC back to that. I have no particular attachment to either name, and I'm happy to go with whichever has consensus amongst the team.

[traviscross]

I'll write something more up here, but as a placeholder for now...

For me, the similarity between the user-visible effect of this RFC and this currently-accepted pattern...

trait Tr {}
impl (dyn Tr) { .. }

...is something I'm finding hard to set aside. Elsewhere in the language, we're often trying to offer a parity where if you can write dyn Trait, you can also write impl Trait. And so it feels natural to want:

trait Tr {}
impl (impl Tr) { .. }

...and so it seems kind of odd to me that we'd call impl (dyn Tr) { .. } methods "inherent" and something that has the effect of impl (impl Tr) { .. } methods "final".

I also think there's probably a coherent direction we could set out here between this and other pending features like inherent trait impls.

But I also want to look into the specialization mental model for this that Niko put forward during the last meeting and give that full consideration.

[joshtriplett]

@traviscross I'm supportive of the concept of impl Trait { ... } blocks, but I'm still inclined to spell this feature #[final], for simplicity. The more I think about it, the more I think it's not worth spending weirdness budget on adding another meaning for "inherent".

[traviscross]

That probably points to trying to decide the open question on the other syntax. It's probably a topic for the design meeting we should have on this one.