describe on tables can be slower than equivalent query on information_schema.columns #443
Open
1 task done
Labels
enhancement
New feature or request
Describe the feature
At least with Iceberg and Hive tables, we have observed that
describe <table>
is consistently slower than the equivalent query againstinformation_schema.columns
to retrieve column information:Given how often
describe
statements are run, we would like to switch to using the faster query againstinformation_schema.columns
.Describe alternatives you've considered
We considered enabling metastore caching in trino. This is disabled in Iceberg and cannot be enabled so we don't consider it a valid option.
Why the
describe
statement is consistently slower in trino is an unknown. We feel updating the dbt-trino adapter is a quicker solution than tracking the down the issue in trino.Who will benefit?
This will speed up dbt pipelines without requiring any trino configuration changes to enable any caching.
Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: