Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[async] Support different materializations for BQ #1261

Open
5 tasks
tatiana opened this issue Oct 21, 2024 · 0 comments
Open
5 tasks

[async] Support different materializations for BQ #1261

tatiana opened this issue Oct 21, 2024 · 0 comments
Labels
area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc dbt:run Primarily related to dbt run command or functionality execution:async Related to the Async execution mode profile:bigquery Related to BigQuery ProfileConfig
Milestone
Cosmos 1.9.0

Comments

@tatiana
Copy link
Collaborator

tatiana commented Oct 21, 2024 •
edited
Loading

Context

https://docs.getdbt.com/docs/build/materializations

Materializations are strategies for persisting dbt models in a warehouse. There are five types of materializations built into dbt. They are:

  • table
  • view
  • incremental
  • ephemeral
  • materialized view

As of Cosmos 1.7, we only support tables with --full-refresh. We should learn what the queries run by with the BQ adaptor and the conditions for those and implement those in Cosmos.

Acceptance criteria

All the following materialisations work the same way when running dbt run and Cosmos using ExecutionMode.AIRFLOW_ASYNC with BQ:

  • table
  • view
  • incremental
  • ephemeral
  • materialized view

We'll likely want to break this ticket into smaller ones.
@tatiana tatiana added the execution:async Related to the Async execution mode label Oct 21, 2024
@dosubot dosubot bot added area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc dbt:run Primarily related to dbt run command or functionality profile:bigquery Related to BigQuery ProfileConfig labels Oct 21, 2024
@tatiana tatiana mentioned this issue Oct 21, 2024
Merged
2 tasks
@tatiana tatiana added this to the Cosmos 1.9.0 milestone Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:execution Related to the execution environment/mode, like Docker, Kubernetes, Local, VirtualEnv, etc dbt:run Primarily related to dbt run command or functionality execution:async Related to the Async execution mode profile:bigquery Related to BigQuery ProfileConfig
Projects
None yet
Milestone
Cosmos 1.9.0
Development

No branches or pull requests
1 participant
@tatiana