Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add retryable #359

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat: add retryable #359

wants to merge 5 commits into from

Conversation

bshaffer
Copy link
Contributor

@bshaffer bshaffer commented Aug 26, 2021

Add automatic retriables for certain error codes. The feature would need to be implemented in google/gax to take effect. This is better for Backwards Compatibility and a better design in general. Other considerations:

  • add exponential backoff
  • add support for retryUnavailable option, which defaults to true. Some clients will want to turn this off.

@bshaffer bshaffer requested a review from a team as a code owner August 26, 2021 18:34
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Aug 26, 2021
@bshaffer bshaffer added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Aug 26, 2021
@bshaffer bshaffer requested a review from TimurSadykov April 29, 2022 04:38
@bshaffer bshaffer changed the title feat: [DO NOT MERGE] add retryable feat: add retryable Mar 2, 2023
@bshaffer bshaffer removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Mar 2, 2023
@bshaffer
Copy link
Contributor Author

bshaffer commented Mar 2, 2023

@TimurSadykov we are getting requests for this behavior to be incorporated for certain spanner errors (see googleapis/google-cloud-php#5473). Can we go ahead and merge it?

@TimurSadykov
Copy link

TimurSadykov commented Mar 3, 2023

As we discussed at some point, we don't want to add retries, just return errors (exceptions), that indicate the error is retryable. If we can transition this into something like that - that would be great.

@bshaffer
Copy link
Contributor Author

bshaffer commented Mar 3, 2023

@TimurSadykov Interesting, that sounds good. But if we indicate it's retryable, wouldn't we also want to allow those requests to be automatically retry?

@TimurSadykov
Copy link

@TimurSadykov Interesting, that sounds good. But if we indicate it's retryable, wouldn't we also want to allow those requests to be automatically retry?

There was a long debate whether we should :) In short, the main reason not to retry is that every client can and should retry anyways. That is how few our languages historically do not have retries and do ok. However, for some of the languages client retries related to specifically auth are blocked by gRPC (described in go/auth-correct-retry). This could be perceived as retries layer is missing. Therefore we are fixing that first with retryable and gRPC fix.

We can always revisit that and add retries later down the road.

@bshaffer
Copy link
Contributor Author

bshaffer commented Mar 3, 2023

@TimurSadykov thank you for the thorough reply! So in the case mentioned by our customers (internal server errors returned by the Spanner service, NOT auth or grpc-specific errors), would it be safe or desirable to implement retry logic?

For reference, Java has implemented this here: googleapis/java-spanner#2111

@TimurSadykov
Copy link

TimurSadykov commented Mar 10, 2023

@bshaffer The overall retry approach is to rely on client side retries. Those are transparent and configurable. Java Spanner example sems to be inline with that. Those are client side retries. Not sure if default retry configuration applies to those retries though. But that would be minor, fixable deviation.

Basically, client knows what to consider retryable from its own backend or transport. For Auth it is a bit different and clients don't know auth specifics. Therefore auth libraries encapsulates the logic of what is considered retriable and communicate to client via Retryable interface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants