Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

duckdb.connect() config argument doesn't accept any S3 configuration options in Python API #50

Open
nicklausroach opened this issue Aug 9, 2024 · 2 comments

Comments

@nicklausroach
Copy link

nicklausroach commented Aug 9, 2024

When trying to add s3-specific configurations to the connection config, I'm met with any of:

duckdb.duckdb.InvalidInputException: Invalid Input Error: Unrecognized configuration property "s3_access_key_id"
duckdb.duckdb.InvalidInputException: Invalid Input Error: Unrecognized configuration property "s3_secret_access_key"
duckdb.duckdb.InvalidInputException: Invalid Input Error: Unrecognized configuration property "s3_region"

I could be doing something wrong here, but all other config options seem to work other than the s3-related options. Example snippet:

>>> duckdb.connect(database=':memory:', read_only=False, config={'s3_secret_access_key': ''})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
duckdb.duckdb.InvalidInputException: Invalid Input Error: Unrecognized configuration property "s3_secret_access_key"
@samansmink
Copy link
Collaborator

This indeed does not work because at startup the httpfs extension is not yet loaded!
please use secrets instead!

@nicklausroach
Copy link
Author

Ah I knew it was something stupid I was doing! That may be worth calling out in the docs, I assumed I could add those s3 configurations at connection time and then load the httpfs extension. Regardless, thank you for your help @samansmink!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants