get redshift external tables (#753)

* get redshift external tables redshift external tables are not listed in information_schema.tables but we can extract the same info from svv_external tables and union the results. also quote database name as redshift allows dashes which require quotes * add changelog * snowflake doesn't accept quotes --------- Co-authored-by: Anders <[email protected]>
dbt-labs · Nov 27, 2023 · 23ddc9e · 23ddc9e
1 parent 6ba7b66
commit 23ddc9e
Show file tree

Hide file tree

Showing 2 changed files with 32 additions and 0 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -12,10 +12,16 @@
 ## Fixes
 - deduplicate macro for Databricks now uses the QUALIFY clause, which fixes NULL columns issues from the default natural join logic
 - deduplicate macro for Redshift now uses the QUALIFY clause, which fixes NULL columns issues from the default natural join logic
+- get_tables_by_pattern_sql will now:
+  - return redshift external tables ([#752](https://github.com/dbt-labs/dbt-utils/issues/752)
+  - work with valid redshift database names that contain dashes
+## Under the hood
+- created a new dispatch redshift__get_tables_by_pattern which unions the result of the default macro and querying svv_external_tables for the same conditions (schema name, pattern, exclude pattern).
 
 ## Contributors:
 [@graciegoheen](https://github.com/graciegoheen)
 [@yauhen-sobaleu](https://github.com/yauhen-sobaleu)
+[@brendan-cook-87](https://github.com/brendan-cook-87)
 
 # dbt utils v1.1.1
 ## New features

diff --git a/macros/sql/get_tables_by_pattern_sql.sql b/macros/sql/get_tables_by_pattern_sql.sql
@@ -16,6 +16,32 @@
 
 {% endmacro %}
 
+{% macro redshift__get_tables_by_pattern_sql(schema_pattern, table_pattern, exclude='', database=target.database) %}
+
+    {% set sql %}
+        select distinct
+            table_schema as {{ adapter.quote('table_schema') }},
+            table_name as {{ adapter.quote('table_name') }},
+            {{ dbt_utils.get_table_types_sql() }}
+        from "{{ database }}"."information_schema"."tables"
+        where table_schema ilike '{{ schema_pattern }}'
+        and table_name ilike '{{ table_pattern }}'
+        and table_name not ilike '{{ exclude }}'
+        union all
+        select distinct
+            schemaname as {{ adapter.quote('table_schema') }},
+            tablename as {{ adapter.quote('table_name') }},
+            'external' as {{ adapter.quote('table_type') }}
+        from svv_external_tables
+        where redshift_database_name = '{{ database }}'
+        and schemaname ilike '{{ schema_pattern }}'
+        and table_name ilike '{{ table_pattern }}'
+        and table_name not ilike '{{ exclude }}'
+    {% endset %}
+
+    {{ return(sql) }}
+{% endmacro %}
+
 
 {% macro bigquery__get_tables_by_pattern_sql(schema_pattern, table_pattern, exclude='', database=target.database) %}