Skip to content

Commit

Permalink
docs(automations): Add new doc for Glossary Term Propagation Automati…
Browse files Browse the repository at this point in the history
…on, other docs cleanup (datahub-project#11851)

Co-authored-by: John Joyce <[email protected]>
  • Loading branch information
jjoyce0510 and John Joyce authored Nov 14, 2024
1 parent 64e9114 commit 62bb5d7
Show file tree
Hide file tree
Showing 4 changed files with 66 additions and 43 deletions.
5 changes: 5 additions & 0 deletions docs-website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,11 @@ module.exports = {
type: "doc",
id: "docs/automations/docs-propagation",
},
{
label: "Glossary Term Propagation",
type: "doc",
id: "docs/automations/glossary-term-propagation",
},
{
label: "BigQuery Metadata Sync",
type: "doc",
Expand Down
22 changes: 0 additions & 22 deletions docs/automations/docs-propagation.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ This feature is enabled by default in Open Source DataHub.
| Column-Level Docs Propagation | ✔️ | ✔️ |
| Asset-Level Docs Propagation | ✔️ | ✔️ |
| Downstream Lineage + Siblings | ✔️ | ✔️ |
| Propagation Rollback (Undo) || ✔️ |
| Historical Backfilling || ✔️ |

## Enabling Documentation Propagation
Expand Down Expand Up @@ -95,27 +94,6 @@ and then click "Initialize".
This one-time step will kick off the back-filling process for existing descriptions. If you only want to begin propagating
descriptions going forward, you can skip this step.

## Rolling Back Propagated Descriptions (DataHub Cloud Only)

In DataHub Cloud, you can rollback all descriptions that have been propagated historically.

This feature allows you to "clean up" or "undo" any accidental propagation that may have occurred automatically, in the case
that you no longer want propagated descriptions to be visible.

To do this, navigate to the Automation you created in Step 3 above, click the 3-dot "More" menu

<p align="left">
<img width="15%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/automation-more-menu.png"/>
</p>

and then click "Rollback".

<p align="left">
<img width="15%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/automation-rollback.png"/>
</p>

This one-time step will remove all propagated tags and glossary terms from Snowflake. To simply stop propagating new tags, you can disable the automation.

## Viewing Propagated Descriptions

Once the automation is enabled, you'll be able to recognize propagated descriptions as those with the thunderbolt icon next to them:
Expand Down
61 changes: 61 additions & 0 deletions docs/automations/glossary-term-propagation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Glossary Term Propagation Automation

<FeatureAvailability saasOnly />

## Introduction

Glossary Term Propagation is an automation feature that propagates classification labels (Glossary Terms) across column and assets based on downstream lineage and sibling relationships.
This automation simplifies metadata management by ensuring consistent term classification and reducing manual effort in categorizing data assets, aiding Data Governance & Compliance, and enhancing Data Discovery.

## Capabilities

- **Column-Level Glossary Term Propagation**: Automatically propagate Glossary Terms to all downstream lineage columns and sibling columns.
- **Asset-Level Glossary Term Propagation**: Automatically propagate Glossary Terms to all downstream lineage assets & sibling assets.
- **Select Terms & Term Groups**: Select specific Glossary Terms & Term Groups to propagate, e.g. to propagate only sensitive or important labels.

Note that Asset-level propagation is currently only support for **Datasets** (Tables, Views, Topics, etc), and not for other asset types including
Charts, Dashboards, Data Pipelines, Data Tasks.

## Enabling Glossary Term Propagation

1. **Navigate to Automations**: Go to 'Govern' > 'Automations' in the navigation bar.

<p align="center">
<img width="20%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/automations-nav-link.png"/>
</p>


2. **Create An Automation**: Select 'Glossary Term Propagation' from the automation types.

<p align="center">
<img width="30%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/glossary-term-propagation/automation-type.png"/>
</p>

3. **Configure Automation**: Complete the required fields and select 'Save and Run' to activate the automation.

<p align="center">
<img width="60%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/glossary-term-propagation/automation-form.png"/>
</p>

## Propagating for Existing Assets

In DataHub Cloud, you can back-fill historical data to ensure existing Glossary Terms are consistently propagated across downstream relationships. To begin, access the Automation created in Step 3, click the 3-dot "more" menu, and choose "Initialize." This will kick off the backfill process.

<p align="left">
<img width="15%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/automation-more-menu.png"/>
</p>

and then click "Initialize".

<p align="left">
<img width="15%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/automation-initialize.png"/>
</p>


## Viewing Propagated Glossary Terms

Once enabled, propagated Glossary Terms will display a thunderbolt icon, indicating the origin of the term and any intermediate lineage hops used in propagation.

<p align="center">
<img width="50%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/glossary-term-propagation/view-propagated-terms.png"/>
</p>
21 changes: 0 additions & 21 deletions docs/automations/snowflake-tag-propagation.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,27 +57,6 @@ and then click "Initialize".
This one-time step will kick off the back-filling process for existing descriptions. If you only want to begin propagating
descriptions going forward, you can skip this step.

## Rolling Back Propagated Tags

You can rollback all tags and glossary terms that have been propagated historically.

This feature allows you to "clean up" or "undo" any accidental propagation that may have occurred automatically, in the case
that you no longer want propagated descriptions to be visible.

To do this, navigate to the Automation you created in Step 3 above, click the 3-dot "More" menu

<p align="left">
<img width="20%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/automation-more-menu.png"/>
</p>

and then click "Rollback".

<p align="left">
<img width="20%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/automation/saas/automation-rollback.png"/>
</p>

This one-time step will remove all propagated tags and glossary terms from Snowflake. To simply stop propagating new tags, you can disable the automation.

## Viewing Propagated Tags

You can view propagated Tags (and corresponding DataHub URNs) inside the Snowflake UI to confirm the automation is working as expected.
Expand Down

0 comments on commit 62bb5d7

Please sign in to comment.