-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Function to guarantee PUB/SUB channel in SYNC_PUB #151
base: master
Are you sure you want to change the base?
Changes from all commits
5a092be
e2e812f
393087d
85fa7c6
0834e34
35ef0df
20e908a
747e57b
97d204e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -36,6 +36,7 @@ | |
from .address import AgentAddressKind | ||
from .address import AgentAddressSerializer | ||
from .address import AgentChannel | ||
from .address import AgentChannelKind | ||
from .address import address_to_host_port | ||
from .address import guess_kind | ||
from .proxy import Proxy | ||
|
@@ -543,6 +544,15 @@ def register(self, socket, address, alias=None, handler=None): | |
self.poller.register(socket, zmq.POLLIN) | ||
self._set_handler(socket, handler) | ||
|
||
def get_handler(self, alias): | ||
""" | ||
Get the handler associated to a socket given the socket alias. | ||
|
||
Ideally, this should only be called for alias that represent a | ||
SUB socket. | ||
""" | ||
return self.handler[self.socket[alias]] | ||
|
||
def _set_handler(self, socket, handlers): | ||
""" | ||
Set the socket handler(s). | ||
|
@@ -647,7 +657,7 @@ def _bind_address(self, kind, alias=None, handler=None, addr=None, | |
self.register(socket, server_address, alias, handler) | ||
# SUB sockets are a special case | ||
if kind == 'SUB': | ||
self._subscribe(server_address, handler) | ||
self.subscribe(server_address, handler) | ||
return server_address | ||
|
||
def _bind_channel(self, kind, alias=None, handler=None, addr=None, | ||
|
@@ -783,7 +793,7 @@ def _connect_address(self, server_address, alias=None, handler=None): | |
if client_address.kind == 'SUB': | ||
if not alias: | ||
alias = client_address | ||
self._subscribe(alias, handler) | ||
self.subscribe(alias, handler) | ||
return client_address | ||
|
||
def _connect_channel(self, channel, alias=None, handler=None): | ||
|
@@ -912,7 +922,7 @@ def _handle_async_requests(self, data): | |
else: | ||
handler(self, response) | ||
|
||
def _subscribe(self, alias: str, handlers: Dict[Union[bytes, str], Any]): | ||
def subscribe(self, alias: str, handlers: Dict[Union[bytes, str], Any]): | ||
""" | ||
Subscribe the agent to another agent. | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be great to complete de docstring here as well now that the method is "user accessible". |
||
|
@@ -965,6 +975,9 @@ def set_attr(self, **kwargs): | |
def get_attr(self, name): | ||
return getattr(self, name) | ||
|
||
def del_attr(self, name): | ||
delattr(self, name) | ||
|
||
def set_method(self, *args, **kwargs): | ||
""" | ||
Set object methods. | ||
|
@@ -1508,6 +1521,20 @@ def close_sockets(self): | |
for sock in self.get_unique_external_zmq_sockets(): | ||
sock.close(linger=get_linger()) | ||
|
||
def get_uuid_used_as_alias_for_sub_in_sync_pub(self, client_alias): | ||
""" | ||
Return the uuid that was used as the alias for the SUB socket | ||
when a connection to a SYNC_PUB channel was made. | ||
""" | ||
channel = self.addr(client_alias) | ||
if channel.kind != AgentChannelKind('SYNC_SUB'): | ||
raise ValueError('Incorrect channel kind: {}'.format(channel.kind)) | ||
client_addr = channel.twin().sender.twin() | ||
addr_to_access_uuid = self.addr(client_addr) | ||
uuid = self._async_req_uuid[addr_to_access_uuid] | ||
|
||
return uuid | ||
|
||
def ping(self): | ||
""" | ||
A test method to check the readiness of the agent. Used for testing | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -192,3 +192,36 @@ def wait_agent_attr(agent, name='received', length=None, data=None, value=None, | |
break | ||
time.sleep(0.01) | ||
return False | ||
|
||
|
||
def synchronize_sync_pub(server, server_alias, client, client_alias): | ||
''' | ||
Create a SYNC_PUB/SYNC_SUB channel and connect both agents. | ||
|
||
Make sure they have stablished the PUB/SUB communication within the | ||
SYNC_PUB/SYNC_SUB channel before returning. This will guarantee that | ||
no PUB messages are lost. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Missing "Parameters" section from the docstring. |
||
''' | ||
def assert_receive(agent, message, topic=None): | ||
try: | ||
agent.get_attr('_tmp_attr') | ||
agent.set_attr(_tmp_attr=True) | ||
except AttributeError: # Attribute already deleted | ||
pass | ||
|
||
uuid = client.get_uuid_used_as_alias_for_sub_in_sync_pub(client_alias) | ||
|
||
# Set a temporary custom handler | ||
client.set_attr(_tmp_attr=False) | ||
original_handler = client.get_handler(uuid) | ||
client.subscribe(uuid, assert_receive) | ||
|
||
# Send messages through the PUB socket until the client receives them | ||
server.each(0.1, 'send', server_alias, 'Synchronize', alias='_tmp_timer') | ||
assert wait_agent_attr(client, name='_tmp_attr', value=True, timeout=5) | ||
server.stop_timer('_tmp_timer') | ||
|
||
# Restore the original handler, now that the connection is guaranteed | ||
client.subscribe(uuid, original_handler) | ||
|
||
client.del_attr('_tmp_attr') | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I am not sure about this implementation. Worst-case there might messages not received by the client when you change the handler. That is really bad. Also, that requires the Would it make sense to ensure synchronization with SYNC-PUB requests? i.e.:
You achieve synchronization while avoiding double You may want to specify an What do you think? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That seems interesting, I will take a look into it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it would not be possible to synchronize them through requests. I am thinking about the case in which a request might change the internal state of the SYNC_PUB agent, and for those cases, an extra hack around would have to be made, I think (changing the handler of the SYNC_PUB momentarily for the requests). Perhaps what we could do is explain that some messages might be lost in the process or that it should only be used for testing purposes. Now I'm thinking that perhaps the function does not even belong there. In the end, I came up with it to make some tests more consistent. Any thoughts? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmmmmm. Yeah, one possibility would be to take that away from there and keep it just for the tests. Another possibility would be to make all SYNC_PUB sockets implement heart-beating. This is something we should do at some point anyway, hopefully Soon™. Heart-beating would mean that a SYNC_PUB socket would publish a "heart-beat" (like an "I'm alive" message) periodically, to all subscribed agents. Clients can use this to know if they should reconnect (maybe the server went down). What we could do for now is to implement just a part of the heart-beating: allow clients to request a heart-beat at any time (i.e.: "are you alive?"). This would be a request from the SYNC_SUB, although it would be treated as a special case and all SYNC_PUB channels should handle it the same way, without taking into account the user-defined handler for other requests. This means adding a "special case" in the protocol, but I don't think it is bad, as requests do not need to be very fast and are infrequent. We could then use those heart-beat requests from the tests to ensure synchronization. If you feel like that is too much for now, just take it away from there and open a new issue so we try to improve it for version 1.0.0. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe heart-beat messages could be published with a special topic (so that clients may decide to subscribe or not to this heart-beat). Maybe we could use that for now for the tests: from the publisher, we add a timer that publishes heart-beats with a special topic, from the subscriber, when we want to make sure we are fully connected, we just subscribe to this topic and wait for the first heart-beat. What you think? Maybe we should anyway open an issue and move this discussion there (we're probably going to discuss these things again when we start working on a good heart-beat implementation). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Docstring
Parameters
andReturns
sections are missing. 😉