Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis PUBSUB connection issue after idle period #52

Open
gliatsos opened this issue Dec 20, 2017 · 2 comments
Open

Redis PUBSUB connection issue after idle period #52

gliatsos opened this issue Dec 20, 2017 · 2 comments

Comments

@gliatsos
Copy link

I am using version 0.5.0 of the library and I have a pretty similar code as in the PUBSUB example. My application subscribes to a channel and receives messages.
What I am facing is that every Monday, the application is not being able to receive messages from Redis.

Is there any timeout that I should handle in case the connection remains idle during the weekend? Shall I configure something extra in my application or in Redis to bypass this?

Thank you in advance,
George

@arrtchiu
Copy link

I have the same issue here except using BLPOP with no timeout rather than publish/subscribe. If I stop pushing events to the queue for ~10 mins usually the most recent BLPOP doesn't fire my async callback.

Have not researched it yet, but at the moment I'm suspecting the underlying socket has timed out but the library hasn't been notified or fails to call the callback. It also does not call the callback installed by installErrorHandler.

Currently worked around it by setting a BLPOP timeout and re-sending the command each time. Kinda like an application-layer tcp-keepalive ;). You could add an extra "ping" message to your pub/sub but I can think of a lot of reasons why you probably wouldn't want to do that.

Will try get to the bottom of this... I don't like this kind of workaround!

In case it's useful to anyone else debugging this issue: I think I've only seen it occur on Linux. Mac doesn't seem to behave the same way.

@mremski
Copy link

mremski commented Feb 21, 2019

@arrtchiu I've also seen that behavior on BLPOP using the sync client, our application uses very long lived connection in a few instances. I believe that the library defaults not sending tcp keepalives (you can set them), we tried setting a timeout on the BLPOP command itself and could still get "hung".
What seems to work for us: setCommandTimeout() on the client to 15 mins, BLPOP with timeout of 0. That definitely terminates the command at the 15 minute mark, and the error returned is End Of File (make sure you add the error parameter on the call to command). Unfortunately if you kill the connection to the server in the that timeout you get the same End Of File, so we treat timeout the same as socket close and call disconnect() on the client and go through a reconnect sequence.
We're running on Linux, both Fedora and Ubuntu, have seen the problem on both.

Oh, I just want to say Thanks to Alex for providing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants