You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
basin3d doesn't support a common missing value. If a datasource's missing value is a numeric, it gets passed thru as if it is a result value.
For consideration:
have a basin3d missing value vocabulary
map missing value from data source to basin3d
catch and translate the datasource missing value
potentially: filter by missing value
Examples from USGS. Note, need to figure out what happens with non-numeric values like NaN, NA.
Using django-basin3d: ~/measurement_tvp_timeseries/?monitoring_features=USGS-09147500&observed_property_variables=RDC&start_date=2023-02-01&end_date=2023-02-10
Return: USGS missing value -999999 is converted to -28316.818683153
'''
"statistic": "MEAN",
"result_points": [
["2023-02-01T00:00:00.000",-28316.818683153],
["2023-02-02T00:00:00.000",-28316.818683153],
["2023-02-03T00:00:00.000",-28316.818683153],
["2023-02-04T00:00:00.000",-28316.818683153],
["2023-02-05T00:00:00.000",-28316.818683153],
["2023-02-06T00:00:00.000",1.5149513145],
["2023-02-07T00:00:00.000",1.5800800625999998],
["2023-02-08T00:00:00.000",-28316.818683153],
["2023-02-09T00:00:00.000",1.614060279],
["2023-02-10T00:00:00.000",-28316.818683153]
],
'''
"""
{"name":"ns1:timeSeriesResponseType","declaredType":"org.cuahsi.waterml.TimeSeriesResponseType","scope":"javax.xml.bind.JAXBElement$GlobalScope","value":{"queryInfo":{"queryURL":"http://waterservices.usgs.gov/nwis/dv/parameterCd=00060&startDT=2023-02-01&endDT=2023-02-10&sites=09147500&format=json&siteType=ST&siteStatus=all","criteria":{"locationParam":"[ALL:09147500]","variableParam":"[00060]","timeParam":{"beginDateTime":"2023-02-01T00:00:00.000","endDateTime":"2023-02-10T00:00:00.000"},"parameter":[]},"note":[{"value":"[ALL:09147500]","title":"filter:sites"},{"value":"[ST]","title":"filter:siteType"},{"value":"[mode=RANGE, modifiedSince=null] interval={INTERVAL[2023-02-01T00:00:00.000-05:00/2023-02-10T00:00:00.000-05:00]}","title":"filter:timeRange"},{"value":"methodIds=[ALL]","title":"filter:methodId"},{"value":"2023-04-21T15:38:38.319Z","title":"requestDT"},{"value":"95cb98e0-e05a-11ed-9320-2cea7f58f5ca","title":"requestId"},{"value":"Provisional data are subject to revision. Go to http://waterdata.usgs.gov/nwis/help/?provisional for more information.","title":"disclaimer"},{"value":"vaas01","title":"server"}]},"timeSeries":[{"sourceInfo":{"siteName":"UNCOMPAHGRE RIVER AT COLONA, CO","siteCode":[{"value":"09147500","network":"NWIS","agencyCode":"USGS"}],"timeZoneInfo":{"defaultTimeZone":{"zoneOffset":"-07:00","zoneAbbreviation":"MST"},"daylightSavingsTimeZone":{"zoneOffset":"-06:00","zoneAbbreviation":"MDT"},"siteUsesDaylightSavingsTime":true},"geoLocation":{"geogLocation":{"srs":"EPSG:4326","latitude":38.33143056,"longitude":-107.779225},"localSiteXY":[]},"note":[],"siteType":[],"siteProperty":[{"value":"ST","name":"siteTypeCd"},{"value":"14020006","name":"hucCd"},{"value":"08","name":"stateCd"},{"value":"08091","name":"countyCd"}]},"variable":{"variableCode":[{"value":"00060","network":"NWIS","vocabulary":"NWIS:UnitValues","variableID":45807197,"default":true}],"variableName":"Streamflow, ft³/s","variableDescription":"Discharge, cubic feet per second","valueType":"Derived Value","unit":{"unitCode":"ft3/s"},"options":{"option":[{"value":"Mean","name":"Statistic","optionCode":"00003"}]},"note":[],"noDataValue":-999999.0,"variableProperty":[],"oid":"45807197"},"values":[{"value":[{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-01T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-02T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-03T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-04T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-05T00:00:00.000"},{"value":"53.5","qualifiers":["P"],"dateTime":"2023-02-06T00:00:00.000"},{"value":"55.8","qualifiers":["P"],"dateTime":"2023-02-07T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-08T00:00:00.000"},{"value":"57.0","qualifiers":["P"],"dateTime":"2023-02-09T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-10T00:00:00.000"}],"qualifier":[{"qualifierCode":"Ice","qualifierDescription":"Value is affected by ice at the measurement site.","qualifierID":0,"network":"NWIS","vocabulary":"uv_rmk_cd"},{"qualifierCode":"P","qualifierDescription":"Provisional data subject to revision.","qualifierID":1,"network":"NWIS","vocabulary":"uv_rmk_cd"}],"qualityControlLevel":[],"method":[{"methodDescription":"","methodID":18989}],"source":[],"offset":[],"sample":[],"censorCode":[]}],"name":"USGS:09147500:00060:00003"}]},"nil":false,"globalScope":true,"typeSubstituted":false}
"""
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
basin3d doesn't support a common missing value. If a datasource's missing value is a numeric, it gets passed thru as if it is a result value.
For consideration:
Examples from USGS. Note, need to figure out what happens with non-numeric values like NaN, NA.
Using django-basin3d: ~/measurement_tvp_timeseries/?monitoring_features=USGS-09147500&observed_property_variables=RDC&start_date=2023-02-01&end_date=2023-02-10
Return: USGS missing value -999999 is converted to -28316.818683153
'''
"statistic": "MEAN",
"result_points": [
["2023-02-01T00:00:00.000",-28316.818683153],
["2023-02-02T00:00:00.000",-28316.818683153],
["2023-02-03T00:00:00.000",-28316.818683153],
["2023-02-04T00:00:00.000",-28316.818683153],
["2023-02-05T00:00:00.000",-28316.818683153],
["2023-02-06T00:00:00.000",1.5149513145],
["2023-02-07T00:00:00.000",1.5800800625999998],
["2023-02-08T00:00:00.000",-28316.818683153],
["2023-02-09T00:00:00.000",1.614060279],
["2023-02-10T00:00:00.000",-28316.818683153]
],
'''
Corresponding USGS direct query:
https://waterservices.usgs.gov/nwis/dv/?parameterCd=00060&startDT=2023-02-01&endDT=2023-02-10&sites=09147500&format=json&siteType=ST&siteStatus=all
"""
{"name":"ns1:timeSeriesResponseType","declaredType":"org.cuahsi.waterml.TimeSeriesResponseType","scope":"javax.xml.bind.JAXBElement$GlobalScope","value":{"queryInfo":{"queryURL":"http://waterservices.usgs.gov/nwis/dv/parameterCd=00060&startDT=2023-02-01&endDT=2023-02-10&sites=09147500&format=json&siteType=ST&siteStatus=all","criteria":{"locationParam":"[ALL:09147500]","variableParam":"[00060]","timeParam":{"beginDateTime":"2023-02-01T00:00:00.000","endDateTime":"2023-02-10T00:00:00.000"},"parameter":[]},"note":[{"value":"[ALL:09147500]","title":"filter:sites"},{"value":"[ST]","title":"filter:siteType"},{"value":"[mode=RANGE, modifiedSince=null] interval={INTERVAL[2023-02-01T00:00:00.000-05:00/2023-02-10T00:00:00.000-05:00]}","title":"filter:timeRange"},{"value":"methodIds=[ALL]","title":"filter:methodId"},{"value":"2023-04-21T15:38:38.319Z","title":"requestDT"},{"value":"95cb98e0-e05a-11ed-9320-2cea7f58f5ca","title":"requestId"},{"value":"Provisional data are subject to revision. Go to http://waterdata.usgs.gov/nwis/help/?provisional for more information.","title":"disclaimer"},{"value":"vaas01","title":"server"}]},"timeSeries":[{"sourceInfo":{"siteName":"UNCOMPAHGRE RIVER AT COLONA, CO","siteCode":[{"value":"09147500","network":"NWIS","agencyCode":"USGS"}],"timeZoneInfo":{"defaultTimeZone":{"zoneOffset":"-07:00","zoneAbbreviation":"MST"},"daylightSavingsTimeZone":{"zoneOffset":"-06:00","zoneAbbreviation":"MDT"},"siteUsesDaylightSavingsTime":true},"geoLocation":{"geogLocation":{"srs":"EPSG:4326","latitude":38.33143056,"longitude":-107.779225},"localSiteXY":[]},"note":[],"siteType":[],"siteProperty":[{"value":"ST","name":"siteTypeCd"},{"value":"14020006","name":"hucCd"},{"value":"08","name":"stateCd"},{"value":"08091","name":"countyCd"}]},"variable":{"variableCode":[{"value":"00060","network":"NWIS","vocabulary":"NWIS:UnitValues","variableID":45807197,"default":true}],"variableName":"Streamflow, ft³/s","variableDescription":"Discharge, cubic feet per second","valueType":"Derived Value","unit":{"unitCode":"ft3/s"},"options":{"option":[{"value":"Mean","name":"Statistic","optionCode":"00003"}]},"note":[],"noDataValue":-999999.0,"variableProperty":[],"oid":"45807197"},"values":[{"value":[{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-01T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-02T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-03T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-04T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-05T00:00:00.000"},{"value":"53.5","qualifiers":["P"],"dateTime":"2023-02-06T00:00:00.000"},{"value":"55.8","qualifiers":["P"],"dateTime":"2023-02-07T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-08T00:00:00.000"},{"value":"57.0","qualifiers":["P"],"dateTime":"2023-02-09T00:00:00.000"},{"value":"-999999","qualifiers":["P","Ice"],"dateTime":"2023-02-10T00:00:00.000"}],"qualifier":[{"qualifierCode":"Ice","qualifierDescription":"Value is affected by ice at the measurement site.","qualifierID":0,"network":"NWIS","vocabulary":"uv_rmk_cd"},{"qualifierCode":"P","qualifierDescription":"Provisional data subject to revision.","qualifierID":1,"network":"NWIS","vocabulary":"uv_rmk_cd"}],"qualityControlLevel":[],"method":[{"methodDescription":"","methodID":18989}],"source":[],"offset":[],"sample":[],"censorCode":[]}],"name":"USGS:09147500:00060:00003"}]},"nil":false,"globalScope":true,"typeSubstituted":false}
"""
The text was updated successfully, but these errors were encountered: