-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add region
kw argument to login
.
#581
Add region
kw argument to login
.
#581
Conversation
src/tiledb/cloud/client.py
Outdated
region_url = urllib3.util.Url( | ||
scheme=orig_url.scheme, | ||
# TODO: Validate region. | ||
host=region + ".aws." + orig_url.host, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we make the region be <region>.<cloud_provider>
so its future proofed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, good point. I turned the region into an enum, which also helps with validating the given region.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don’t want to make this AWS-specific; since this is just going to replace the host
parameter in the endpoint, would it make sense to have a set of constants that the user can pass in?
AWS_US_EAST_1 = "https://us-east-1.aws.api.tiledb.com"
# etc.
# Or namespace it with classes:
class AWS:
US_EAST_1 = "https://us-east-1.aws.api.tiledb.com"
class OtherCloud:
SOME_REGION: "https://some-region.other.api.tiledb.com"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My thinking with the enum was that we simply add another one as we support new cloud providers. That said, if y'all prefer constants, then that's fine with me.
Also, in de2950f, I revised the code to expose a separate function instead, tiledb.cloud.login_to_region()
. This feels more natural to use.
3ca5333
to
52252f1
Compare
region
kw argument to login
.region
kw argument to login
.
src/tiledb/cloud/client.py
Outdated
:param region: the cloud region to run code in | ||
:return: | ||
""" | ||
if not region in AWS: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this should be if region not in AWS
instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They're semantically identical but your suggestions is more readable, so I addressed this in 5af2601.
src/tiledb/cloud/client.py
Outdated
AP_SOUTHEAST_1 = Asia, Singapore | ||
""" | ||
|
||
US_EAST_1 = "https://us-east-1.aws.api.tiledb.com" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can expose a method to the clients to just print all the available regions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My expectation was that users turn to the docs to see what regions are supported. That said, if you see value in such a function then I'm happy to add it.
src/tiledb/cloud/client.py
Outdated
@@ -63,6 +63,36 @@ def Ctx(config=None): | |||
return tiledb.Ctx(Config(config)) | |||
|
|||
|
|||
class AWS(enum.Enum): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would call this AWSRegion or namespace it inside a Region class (so it shows up as Region.AWS.whatever
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 445c41d.
src/tiledb/cloud/client.py
Outdated
:return: | ||
""" | ||
if region not in AWS: | ||
raise Exception("Cloud region is invalid") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
raise Exception("Cloud region is invalid") | |
raise ValueError(f"region {region} is invalid") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 25bccf9.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’m still skeptical of this being its own function which takes enums rather than just a set of string constants that you can pass to host
. Among other things, as we add new regions, existing installs would have a divide where existing regions can use the region type and the login_to_region
function, but new regions would use login
with host=
.
In short, to me,
tiledb.cloud.login(host=tiledb.cloud.Region.AWS.US_EAST_1)
(or maybe make region
into a module, tiledb.cloud.regions
)
seems like a perfectly reasonable thing to write, and doesn’t require any new functions.
9aa1b76
to
e5401a6
Compare
See e5401a6. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one suggestion but looks good overall.
src/tiledb/cloud/region/region.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe put this directly into region/__init__.py
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, addressed in the squashed commit that I just force-pushed. Thanks for all your feedback!
Some users need to log in to specific regions only. This PR exposes host constants that make this easier, like so: import tiledb.cloud from tiledb.cloud.region import AWS tiledb.cloud.login(host=AWS.US_EAST_1) As of today, we support five AWS regions.
3121ee2
to
53ce099
Compare
As mentioned in this story, there are several ways to allow users to specify the region:
tiledb.cloud.client.config.config.host
directly, which is the status quo.region
keyword argument to classes likeDAG
,Delayed
, etc.The first option is confusing to customers, so we need a better approach. I've been playing with the second approach and find it confusing too: It's easy to forget to pass the
region
argument. Besides, if a user wants to run a task graph in region A, odds are that they want all their code to run in region A. The third approach strikes me as the safest and least confusing but it feels odd having to specify the region as part of the login function, like so:Or perhaps we should instead expose a new function, like
tiledb.cloud.set_region()
? Any thoughts on this?