Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to share data between nodes and controller #769

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

pehala
Copy link

@pehala pehala commented Mar 25, 2022

Why?

Once the nodes are branched from the controller, there is no way to transmit information between them, only through pytest events, which is not always enough (e.g. collecting info from fixtures that are run on nodes).

How?

I created a simple mechanism that will send back dict of serializable data when the node is being shutdown. This means that you have access to all the shared data from nodes in the session_finish hook on the controller. Example of how it works can be seen in the simple test I wrote.

Appendix

This is a proof of concept and is in no way finished, the point of this PR is to start the discussion if this feature (or something along those lines) is even wanted and if you agree that yes, I will finish the PR and polish it.

@pehala pehala marked this pull request as draft March 25, 2022 09:17
@pehala pehala force-pushed the shared_data branch 2 times, most recently from 25ccfa1 to 48b7e2f Compare March 25, 2022 09:24
Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first glance this looks like a disconnected data dump lacking clarity

Unless there is way more clarity as to what this is supposed to support, we can't accept the new api

@pehala
Copy link
Author

pehala commented Mar 25, 2022

One of the issues that feature like this would solve is pytest-dev/pytest#7767, where there exists a hack how to use that function but it has to be run on the controller if you need to use this from any fixture, it will not do anything as it is executed on the worker node, which doesn't have any way how to synchronize the data with the master.

Basically, I think that it would be good to have some sort of API that would enable sharing of data between nodes and controller in any way. It doesn't have to be this API (If we agree on how the API should look I will gladly implement it in a better way), this is just PoC that something like it can exist and can work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants