Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new spec to references pages #6190

Merged
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
b70d801
update configs
mirnawong1 Oct 1, 2024
66f65b5
add properites adn configs
mirnawong1 Oct 1, 2024
c699609
update all content under snapshots
mirnawong1 Oct 1, 2024
b81da15
Update check_cols.md
mirnawong1 Oct 1, 2024
dc6b25e
Update check_cols.md
mirnawong1 Oct 1, 2024
261dcff
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 1, 2024
966cf41
Update check_cols.md
mirnawong1 Oct 1, 2024
78706d6
Update pre-hook-post-hook.md
mirnawong1 Oct 1, 2024
1a3ff6d
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 1, 2024
d93d65a
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 1, 2024
5ef956a
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 1, 2024
7f600ba
update and triple check
mirnawong1 Oct 1, 2024
8aa509d
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 1, 2024
16974cc
Merge branch 'add/new-snapshot-spec-reference' of github.com:dbt-labs…
mirnawong1 Oct 1, 2024
b9c2937
update
mirnawong1 Oct 1, 2024
73a13b0
update ref to downstream
mirnawong1 Oct 1, 2024
4252fa4
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 1, 2024
b523a38
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 2, 2024
6b11685
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 2, 2024
c9f9bbf
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 2, 2024
e21535c
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 2, 2024
98ad8ac
Update website/docs/reference/resource-configs/strategy.md
mirnawong1 Oct 3, 2024
b0191c6
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 3, 2024
2076f26
Update website/docs/reference/resource-configs/unique_key.md
mirnawong1 Oct 3, 2024
18826b7
Update website/docs/reference/resource-configs/updated_at.md
mirnawong1 Oct 3, 2024
7fbb2fe
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 3, 2024
d99fb72
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 3, 2024
23bb87a
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 3, 2024
2dbdeed
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 3, 2024
4882c66
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 3, 2024
7906a09
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 3, 2024
ffc8b3f
Update website/docs/reference/snapshot-configs.md
mirnawong1 Oct 3, 2024
dfa67f9
doug's feedback
mirnawong1 Oct 3, 2024
e0d4277
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 3, 2024
3ac664e
Merge branch 'add/new-snapshot-spec-main' into add/new-snapshot-spec-…
mirnawong1 Oct 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion website/docs/reference/configs-and-properties.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,18 @@ Whereas you can use **configurations** to:

Depending on the resource type, configurations can be defined in the dbt project and also in an installed package by:

<VersionBlock firstVersion="1.9">

1. Using a [`config` property](/reference/resource-properties/config) in a `.yml` file in the `models/`, `snapshots/`, or `tests/` directory
2. From the [`dbt_project.yml` file](dbt_project.yml), under the corresponding resource key (`models:`, `snapshots:`, `tests:`, etc)
</VersionBlock>

<VersionBlock lastVersion="1.8">

1. Using a [`config()` Jinja macro](/reference/dbt-jinja-functions/config) within a `model`, `snapshot`, or `test` SQL file
2. Using a [`config` property](/reference/resource-properties/config) in a `.yml` file
2. Using a [`config` property](/reference/resource-properties/config) in a `.yml` file in the `models/`, `snapshots/`, or `tests/` directory.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
3. From the [`dbt_project.yml` file](dbt_project.yml), under the corresponding resource key (`models:`, `snapshots:`, `tests:`, etc)
</VersionBlock>

### Config inheritance

Expand Down
76 changes: 75 additions & 1 deletion website/docs/reference/resource-configs/check_cols.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,32 @@ resource_types: [snapshots]
description: "Read this guide to understand the check_cols configuration in dbt."
datatype: "[column_name] | all"
---

<VersionBlock firstVersion="1.9">
<File name="snapshots/<filename>.yml">

```yml
snapshots:
- name: snapshot_name
relation: source('jaffle_shop', 'orders')
config:
schema: string
unique_key: column_name_or_expression
strategy: check
check_cols:
- column_name
```

</File>
</VersionBlock>

<VersionBlock lastVersion="1.8">

:::info Use the latest snapshot syntax

In Versionless and dbt v1.9 and later, snapshots are defined in an updated syntax using a YAML file within your `snapshots/` directory (as defined by the [`snapshot-paths` config](/reference/project-configs/snapshot-paths)). For faster and more efficient management, consider the updated snapshot YAML syntax, [available in Versionless](/docs/dbt-versions/versionless-cloud) or [dbt Core v1.9 and later](/docs/dbt-versions/core).
:::
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

<File name='snapshots/<filename>.sql'>

```jinja2
Expand All @@ -14,7 +40,7 @@ datatype: "[column_name] | all"
```

</File>

</VersionBlock>

<File name='dbt_project.yml'>

Expand Down Expand Up @@ -42,6 +68,29 @@ No default is provided.

### Check a list of columns for changes

<VersionBlock firstVersion="1.9">

<File name="snapshots/orders_snapshot_check.yml">

```yaml
snapshots:
- name: orders_snapshot_check
relation: source('jaffle_shop', 'orders')
config:
schema: snapshots
unique_key: id
strategy: check
check_cols:
- status
- is_cancelled
```
</File>

To select from this snapshot in a downstream model: `select * from {{ source('jaffle_shop', 'orders') }}`
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
</VersionBlock>

<VersionBlock lastVersion="1.8">

```sql
{% snapshot orders_snapshot_check %}

Expand All @@ -58,8 +107,32 @@ No default is provided.
{% endsnapshot %}
```

</VersionBlock>

### Check all columns for changes

<VersionBlock firstVersion="1.9">

<File name="orders_snapshot_check.yml">

```yaml
snapshots:
- name: orders_snapshot_check
relation: source('jaffle_shop', 'orders')
config:
schema: snapshots
unique_key: id
strategy: check
check_cols:
- all
```
</File>

To select from this snapshot in a downstream model: `select * from {{ source('jaffle_shop', 'orders') }}`
</VersionBlock>

<VersionBlock lastVersion="1.8">

```sql
{% snapshot orders_snapshot_check %}

Expand All @@ -75,3 +148,4 @@ No default is provided.

{% endsnapshot %}
```
</VersionBlock>
48 changes: 48 additions & 0 deletions website/docs/reference/resource-configs/invalidate_hard_deletes.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,32 @@ description: "Invalidate_hard_deletes - Read this in-depth guide to learn about
datatype: column_name
---


<VersionBlock firstVersion="1.9">

<File name='snapshots/<filename>.yml'>

```yaml
snapshots:
- name: snapshot
relation: source('my_source', 'my_table')
[config](/reference/snapshot-configs):
strategy: timestamp
invalidate_hard_deletes: true | false
```

</File>


</VersionBlock>

<VersionBlock lastVersion="1.8">

:::info Use the latest snapshot syntax

In Versionless and dbt v1.9 and later, snapshots are defined in an updated syntax using a YAML file within your `snapshots/` directory (as defined by the [`snapshot-paths` config](/reference/project-configs/snapshot-paths)). For faster and more efficient management, consider the updated snapshot YAML syntax, [available in Versionless](/docs/dbt-versions/versionless-cloud) or [dbt Core v1.9 and later](/docs/dbt-versions/core).
:::

<File name='snapshots/<filename>.sql'>

```jinja2
Expand All @@ -17,6 +43,7 @@ datatype: column_name
```

</File>
</VersionBlock>

<File name='dbt_project.yml'>

Expand All @@ -39,6 +66,26 @@ By default the feature is disabled.

## Example

<VersionBlock firstVersion="1.9">
<File name='snapshots/orders.yml'>

```yaml
snapshots:
- name: orders_snapshot
relation: source('jaffle_shop', 'orders')
config:
schema: snapshots
database: analytics
unique_key: id
strategy: timestamp
updated_at: updated_at
invalidate_hard_deletes: true
```
</File>

</VersionBlock>

<VersionBlock lastVersion="1.8">
<File name='snapshots/orders.sql'>

```sql
Expand All @@ -60,3 +107,4 @@ By default the feature is disabled.
```

</File>
</VersionBlock>
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ snapshots:

</File>

<VersionBlock lastVersion="1.8">

<File name='snapshots/<filename>.sql'>

```sql
Expand All @@ -125,13 +127,15 @@ select ...
```

</File>
</VersionBlock>

<File name='snapshots/properties.yml'>

```yml
snapshots:
- name: [<snapshot_name>]
config:
[config](/reference/resource-properties/config):
[<snapshot_config>](/reference/snapshot-configs): <config_value>
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
[pre_hook](/reference/resource-configs/pre-hook-post-hook): <sql-statement> | [<sql-statement>]
[post_hook](/reference/resource-configs/pre-hook-post-hook): <sql-statement> | [<sql-statement>]
```
Expand Down
51 changes: 50 additions & 1 deletion website/docs/reference/resource-configs/snapshot_name.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't realize this page existed until this week!

We should consider an action item for later if we'd want to keep this page or retire it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, now I see some other pages that link here: using a link to /reference/resource-configs/snapshot_name.

Still might be worth a 2nd look later since I don't think we have similar model_name, seed_name, etc. pages.

Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,27 @@
description: "Snapshot-name - Read this in-depth guide to learn about configurations in dbt."
---

<VersionBlock firstVersion="1.9">
<File name='snapshots/<filename>.yml'>

```yaml
snapshots:
- name: snapshot_name
relation: source('my_source', 'my_table')
config:
schema: string
database: string
unique_key: column_name_or_expression
strategy: timestamp | check
updated_at: column_name # Required if strategy is 'timestamp'

```

</File>
</VersionBlock>

<VersionBlock lastVersion="1.8">

<File name='snapshots/<filename>.sql'>

```jinja2
Expand All @@ -13,9 +34,16 @@ description: "Snapshot-name - Read this in-depth guide to learn about configurat

</File>

:::info Use the latest snapshot syntax

In Versionless and dbt v1.9 and later, snapshots are defined in an updated syntax using a YAML file within your `snapshots/` directory (as defined by the [`snapshot-paths` config](/reference/project-configs/snapshot-paths)). For faster and more efficient management, consider the updated snapshot YAML syntax, [available in Versionless](/docs/dbt-versions/versionless-cloud) or [dbt Core v1.9 and later](/docs/dbt-versions/core).
:::
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

</VersionBlock>

## Description

The name of a snapshot, as defined in the `{% snapshot %}` block header. This name is used when selecting from a snapshot using the [`ref` function](/reference/dbt-jinja-functions/ref)
The name of a snapshot, which is used when selecting from a snapshot using the [`ref` function](/reference/dbt-jinja-functions/ref)

This name must not conflict with the name of any other "refable" resource (models, seeds, other snapshots) defined in this project or package.

Expand All @@ -24,6 +52,26 @@ The name does not need to match the file name. As a result, snapshot filenames d
## Examples
### Name a snapshot `order_snapshot`

<VersionBlock firstVersion="1.9">
<File name='snapshots/order_snapshot.yml'>


```yaml
snapshots:
- name: order_snapshot
relation: source('my_source', 'my_table')
config:
schema: string
database: string
unique_key: column_name_or_expression
strategy: timestamp | check
updated_at: column_name # Required if strategy is 'timestamp'
```
</File>

</VersionBlock>

<VersionBlock lastVersion="1.8">
<File name='snapshots/orders.sql'>

```jinja2
Expand All @@ -35,6 +83,7 @@ The name does not need to match the file name. As a result, snapshot filenames d

</File>

</VersionBlock>

To select from this snapshot in a downstream model:

Expand Down
Loading
Loading