-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deprecate public constructors of module_ class #2552
Deprecate public constructors of module_ class #2552
Conversation
19c3079
to
81dd20e
Compare
81dd20e
to
4be6029
Compare
575cdf0
to
7e51331
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I mildly prefer the public name - macros like PYBIND11_MODULE
tend to invariably have a few power users that need to avoid them for some special reason, so then this would be a requirement in that case. But we could wait and see, changing it to a public method in 1.6.1 if requested?
I believe the future direction is that we can better align with CPython's tools (like PyModule_New
) in the future, possibly supporting multi-phase initialization eventually.
👍 That's kind of what I was aiming at, yes. I had/have actually changed it in #2551 (which has become a lot smaller, after rebasing onto this PR :-) ), since it makes sense in that context (i.e., vanilla
Yessssss! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @YannickJadoul, I agree this is better, I really like the deprecation. Please feel free to merge.
Optional, only if you feel it has merit: there is a lot of PY2 vs PY3 #if
branching. Maybe it could be reduced via something like:
#if PY_MAJOR_VERSION >= 3
using module_def = PyModuleDef;
#else
struct module_def {};
#endif
Then the code for PY2 code could be mostly identical to PY3 code.
If this is feasible, there will be a universal create_top_level_module()
. I'd definitely make it public in that case. I'm less certain about making it public if there are different versions for PY2 and PY3, because it will force downstream users to also have #if
, unless they are certain only PY3 is needed.
include/pybind11/pybind11.h
Outdated
}; | ||
auto m = PyModule_Create(def); | ||
if (m == nullptr) | ||
pybind11_fail("Internal error in detail::create_top_level_module()"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Idea 1: does throw error_already_set()
work?
Idea 2: somehow include name
in the error message, to be helpful?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Answered in the main thread, but that was maybe not a great idea:
I'm now just looking at the error message, to finish this PR, but I can't find if a nullptr mean the error indicator is already set?
https://docs.python.org/3/c-api/module.html#c.PyModule_CreateEDIT: If it definitely does, we should throw py::error_already_set to be fully correct!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at the 3.7 sources, I see some code paths set an error for sure, but it's difficult to know if that's always the case. The bomb-proof solution would be something like if (PyErr_Occurred()) throw error_already_set();
and pybind11_fail()
as a fallback, with name
in the error message.
I kind of feel it's worth it: in case this fires it's most likely a difficult to debug situation, and showing the original error (root cause) or the module name
could save a lot of time troubleshooting.
Unless we can convince the Python developers to document that a NULL return guarantees the error is set ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might not need to add name to the error message, if it usually or hopefully always works with error_already_set()
, as that makes pybind11_fail()
a last-ditch fallback.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Assuming error_already_set would show the name?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Myeah, or we just trust that this part of the API works in the same way? An error_already_set
won't cause things to be messed up too badly.
I can't both throw error_already_set
ánd have the name
in there, though. So that's only as a fall-back?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Assuming error_already_set would show the name?)
I don't know; we don't control that, since that the message is already set by Python. It might be something as cryptic as "didn't manage to create a str
from this const char*
"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, didn't add the name, as the pybind11_fail
is only our fallback option. We can add it, if we find out that (and when!) the control flow can reach that statement?
Great idea, yes! That's exactly what's holding me back. I had been thinking about a typedef with |
It's only one per module, which is almost certainly below the noise level, but also only for the un-dead Python 2: IMO nothing to worry about at all. |
We are definitely at the point where we should design for Python 3, and if it makes Python 2 a little ugly, I don't think that's something to worry about. Python 3 will be around a lot longer. :) |
OK, perfect! I agree, but wanted to have it mentioned :-) I'm now just looking at the error message, to finish this PR, but I can't find if a EDIT: If it definitely does, we should throw |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, I just have some minor polishing suggestions.
include/pybind11/pybind11.h
Outdated
auto m = Py_InitModule3(name, nullptr, options::show_user_defined_docstrings() ? doc : nullptr); | ||
if (m == nullptr) | ||
pybind11_fail("Internal error in module_::create_extension_module()"); | ||
return reinterpret_borrow<module_>(m); | ||
#endif |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like you could move this #endif
and the #elif
above up, and not duplicate the 3 code lines?
Did you see my question about pybind11_fail
vs throw error_already_set()
from yesterday?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about that, but decided not to because of the // TODO: Should be reinterpret_steal, but Python also steals it again when returned from PyInit_...
comment (which only holds for Python 3; in Python 2, it should be reinterpret_borrow
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you see my question about
pybind11_fail
vsthrow error_already_set()
from yesterday?
Yes. Wait, let me reply in that thread.
…ule, and unify Python 2 and 3 signature again
7d17263
to
a65a38b
Compare
b5b4c25
to
0d0b596
Compare
…ge of module_::module_
try { \ | ||
PYBIND11_CONCAT(pybind11_init_, name)(m); \ | ||
return m.ptr(); \ | ||
} catch (::pybind11::error_already_set &e) { \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use PYBIND11_CATCH_INIT_EXCEPTIONS here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll pour that into another PR, yes.
Looks all good to me, just a minor note about the repeated exception handling code in the macro declaration. |
OK, so, this does a few things. A summary:
Main goal
module_
constructors (see discussion in Unify Python 2 & 3 py::module constructor, and make contructor with pre-allocated PyModuleDef private #2534 and static allocation for PyModuleDef, to avoid leak check errors. #2413 on how they actually are not/should not be used). The useful public constructor for Python 2 is moved todetail::create_top_level_module
.static PyModuleDef
) also forPYBIND11_EMBEDDED_MODULE
. (This is kind of a consequence of the previous bullet, since it the deprecation warnings make it clear this wasn't changed yet in static allocation for PyModuleDef, to avoid leak check errors. #2413.)Minor extra cleanup
pybind11
namespace by using::pybind11
in these module-related macros.module_::add_object
into a valid doxygen-documentation of the function (and teach Doxygen thatPYBIND11_NOINLINE
isn't really a relevant part of the signature to generate docs).Two more questions to be answered:
Should this still be part of 2.6.0? I'd argue "yes", since
a) I don't see a lot of uses for a plain
py::module_
constructor (@rwgk also found out thatmodule_
's constructor is hardly used in Google's code base, which strengthens the belief that there's not a lot of use for that constructor), andb) this will allow us to maybe reuse this constructor later? (cfr. Use PyModule_New in module_ public constructor #2551), and
c) what's the point of postponing this, and delaying the pain (if any). That's what deprecation is for, no?
Should we have
pybind11::detail::create_top_level_module
orstatic pybind11::module_::create_extension_module
? I.e., is this a public interface or an internal one? I know I was the one arguing for the latter, before, but I think this being a public, but "named" constructor would work for me, as well.EDIT:
pybind11::detail::create_top_level_module
was moved tostatic pybind11::module_::create_extension_module
, with a unified signature, though @rwkg's suggestion of a typedef!