-
Notifications
You must be signed in to change notification settings - Fork 155
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] Error when loading NULL result from JSON column #1101
Comments
Thanks for reporting this @Gilby216! TLDRI think #1061 will fix this. Root causeWe've had a couple issues that look similar to this, namely #972 and #1055. It looks like the underlying issue for this one is within our fix added in #974. When I tried the following monkey patch instead, it worked: def _json_from_json(value, _):
"""NOOP string -> string coercion"""
return None if value is None else json.loads(value) SolutionThe fix might be covered by #1061, which removes our monkey patch in favor of relying on googleapis/python-bigquery#1587 instead. We should add the following reproduction case to a test case within #1061 to confirm. When I removed the monkey patch in my local environment, it worked. ReprexRun the following while using dbt-bigquery: dbt show --inline 'select parse_json(null)' Get this error:
WorkaroundIn the meantime, @jeremyyeo mentioned within #1055 that you can coerce it into Select ifnull(parse_json(NULL), to_json(null)) |
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days. |
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers. |
Is this a new bug in dbt-bigquery?
Current Behavior
When I run the following query in dbt Cloud I get an error:
Select parse_json(NULL)
Error encountered:
Runtime Error the JSON object must be str, bytes or bytearray, not NoneType
However that same query runs successfully when I run it directly in the BigQuery console. It outputs a SQL NULL value.
The query above is the simplest reproduce case. If it's useful, the actual situation I am trying to solve for here is to query into a JSON column and extract certain fields, but sometimes those fields are missing. To illustrate this more concretely, the following SQL also produces the same error in DBT Cloud:
Select parsed_json_data.ex1 from ( Select parse_json('{"ex2":"example"}') AS parsed_json_data ) AS t1
But again, that query above runs successfully directly in BigQuery console and outputs a NULL value.
If you change this query to extract the field that does exist in the JSON then everything works fine in both DBT Cloud and BigQuery console:
Select parsed_json_data.ex2 from ( Select parse_json('{"ex2":"example"}') AS parsed_json_data ) AS t1
Expected Behavior
NULL values returned from JSON columns should not error, they should output NULL.
Steps To Reproduce
In DBT Cloud, preview the following query/model:
Select parse_json(NULL)
Relevant log output
Environment
Additional Context
No response
The text was updated successfully, but these errors were encountered: