Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connector creating table in Default schema instead of specified bucket for automatic created tables #300

Open
kevingomez93 opened this issue Oct 7, 2024 · 0 comments

Comments

@kevingomez93
Copy link

I have configured an Iceberg Sink Connector with the following configuration:

connector.class=io.tabular.iceberg.connect.IcebergSinkConnector
iceberg.tables.evolve-schema-enabled=true
tasks.max=2
iceberg.tables.schema-override.event_date.type=Date
iceberg.catalog.client.region=us-east-1
iceberg.tables.auto-create-enabled=true
iceberg.control.commit.timeout-ms=1480000
iceberg.tables=schema.some_table
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter=org.apache.kafka.connect.storage.StringConverter
iceberg.tables.default-partition-by=event_date
topics=some_topic
iceberg.catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO
iceberg.control.commit.interval-ms=120000
iceberg.catalog.uri=thrift://some-adress:9083
value.converter.schemas.enable=false
iceberg.catalog.warehouse=s3a://streaming-bucket/warehouse
iceberg.catalog.type=hive
iceberg.catalog.s3.path-style-access=true
iceberg.catalog.s3.bucket.name=streaming-bucket

Even though the configuration explicitly specifies the iceberg.catalog.warehouse parameter and iceberg.catalog.s3.bucket.name as s3a://streaming-bucket/warehouse and the bucket name as streaming-bucket, the connector automatically creates the Iceberg table in a default bucket rather than the specified bucket and schema.

Do you know if is possible to set the bucket where the table in S3 should be placed?
Is there something that I am missing in the configuration?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant