You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Would be nice to be able to be able to be able to handle 403 errors and do some sort of rate limiting for NOMADs requests. Ran into an issue where I got blocked by the NWS's abusive Behavior policy when requesting data from NOMADs.
Abusive Behavior Policy can be found here
Info on how to handle retries on error codes can be found here
and rate limiting in requests can be done using requests-ratelimiter python package.
At the minimum the ability to pass in a requests.session variable would be helpful because then we could set up our own rate limiting/error handling
EDIT: Just wanted to add the issue is especially prevalent when requesting a forecast that hasn't yet been generated, as it causes Herbie to mass-ignore 403 error codes for files that don't yet exist and quickly results in a ban
The text was updated successfully, but these errors were encountered:
Would be nice to be able to be able to be able to handle 403 errors and do some sort of rate limiting for NOMADs requests. Ran into an issue where I got blocked by the NWS's abusive Behavior policy when requesting data from NOMADs.
Abusive Behavior Policy can be found here
Info on how to handle retries on error codes can be found here
and rate limiting in requests can be done using requests-ratelimiter python package.
At the minimum the ability to pass in a requests.session variable would be helpful because then we could set up our own rate limiting/error handling
EDIT: Just wanted to add the issue is especially prevalent when requesting a forecast that hasn't yet been generated, as it causes Herbie to mass-ignore 403 error codes for files that don't yet exist and quickly results in a ban
The text was updated successfully, but these errors were encountered: