Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref and source functions both return a Relation object #6331

Open
1 task done
dbeatty10 opened this issue Oct 21, 2024 · 0 comments
Open
1 task done

ref and source functions both return a Relation object #6331

dbeatty10 opened this issue Oct 21, 2024 · 0 comments
Labels
content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear

Comments

@dbeatty10
Copy link
Contributor

dbeatty10 commented Oct 21, 2024

Contributions

  • I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates

It seems to me that there is an overarching concept that encompasses the ref and source functions. But we don't have a name for this concept or a page that covers it.

See the docs for the relation config in snapshots as an example of a config that can take either a ref or a source as an argument. This same thing applies to given inputs in unit tests (although we don't currently have any examples of using source). See also depends_on for exposures.

Spitballing, maybe "relation functions" is a decent name? The docs for exposures call these "refable", but sources and metrics aren't in the list of refable node types.

Here's a brainstorm of alternative names:

  • reference
  • dependency
  • connector
  • resolver
  • binder
  • link
  • anchor
  • bridge
  • context
  • map
  • gateway
  • lookup
  • path
  • access
  • pointer
  • relay
  • fetch
  • tether
  • dependency link

What part(s) of the page would you like to see updated?

The ref and source functions have very similar behavior.

Here's a couple similarities:

  • Both return a Relation
  • Both compile to the full object name in the database
  • Both create dependencies between the referenced node and the current model, which is useful for documentation and node selection

Here's some differences:

  • ref is for a model, seed, or snapshot whereas source is just for sources
  • ref has a minimum of one argument has optional arguments for project/package and version whereas source always has exactly two arguments.
  • ref get its database and schema from the generate_database_name and generate_schema_name macros (which use target.database and target.schema from the target variables) whereas source has database and schema as part of its definition in YAML.

Additional information

This issue is mainly inspired by the relation config in snapshots that can be either source or ref:

snapshots:
  - name: string
    relation: relation  # source('my_source', 'my_table') or ref('my_model')

But could also help address this question: https://stackoverflow.com/questions/73784913/dbt-ref-vs-source

Other tie-ins

@dbeatty10 dbeatty10 added content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear labels Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear
Projects
None yet
Development

No branches or pull requests

1 participant