PresRat correction #215

Checking some edge cases that were wrong before.

This should avoid singularities while not changing the averages.

It should help to follow the logic of the tests applied.

Rolling back to pass the tests before moving again to the newer formulation.

Hopefuly we won't need to use it, but let's lock what is working now to help to figure out where is the problem.

Fixed this test. The issue is that gridpoints missing historical references are skipped, and then there is an implicit interpolation resulting in approximated values that fail precise comparisons.

Saving mf and mh explicitly since we should be able to simplify the operation. For now, saving monthly but it should transitioned later.

Coordinate and time are decoupled from the data. To be able to run monthly means we shall include it. To keep consistent with the rest of the library, let's add a couple more arguments.

The non-finite values should be ignored in the statistics.

It's actually only used in the prototype of the monthly means replacement, but let's add it already anyways.

This avoids too many explicit arguments. For instance, PresRat requires variables related to the K parameter.

Pass on all (current) tests. Recording it before refactoring.

It's safer to hold for now and wait for the ecosystem to stabilize before moving.

Despite using the very same file, sup3r manipulate the dataset and expect some patterns without reinforcing it, leading to quiet errors. The modifications here match the expected dataset by sup3r so we can validate results.

Recording it before the refactoring and integration with QDM.

It looks like a bug in pylinter, but it's easier to just avoid it.

Checking more details in the output.

sup3r expects data in a certain way and implicitly apply some transformations, so it is required to conform with that to be able to compare answers.

Figuring out the minimum requirements to run.

It should work fine with any Python 3.11.

@bnb32

Thanks to @bnb32 for noticing that!

Initializing documentation but more work to be done here.

Simple enough to have now doubts, so take advantage of `--fix`.

@bnb32

As suggested by @bnb32.

Future implementation for moving windows that are not ready yet.

Creating support for custom datasets for training created on the fly.

All related to synthetic data for testing. I must improve those before the final PR.

And sliently does the wrong thing otherwise.

Whenever an assumption is required, it must be reinforced to avoid silent errros and wasted time debugging those.

Unfortunately we can't do much about that while using numpy.

At some point moved to serial to help debugging but forgot that.

Moved to use synthetic data, so reference (global) variables are not based anymore in a fixed sample data file, but instead are prescribed and govern the training data.

Related to the migration to synthetic sample data.

It will be needed later, but let's keep it clean for now.

Removing old hardcoded testing variables in favor of global general test guiding variables.

It was missing custom inputs for `dist` and `relative`.

This is a different interpretation where x's denominator is also corrected, i.e. mh-corrected, which by definition is approximated to oh.

I lost that change by mistake while rebasing.

Althought `PresRat` is indeed derived from `QuantileDeltaMappingCorrection`, it is best to refer to `bias.PresRat` even if that will require the user to finally reach to `QuantileDeltaMappingCorrection` for more details.

Returning too many variables. Better do it at least groupped in a dictionary.

Also, reinforcing named arguments to avoid errors.

@grantbuster

New threshold is zero as requested by @grantbuster.

A lot of redundancy. Make it work correctly first, optimize later.

We might not need that anymore with the new formulation.

With the time dependent QDM, it would require a larger testing data to pass on this one. We might want to pay the price of a larger training data since the actual data shouldn't have such gap.

The fp_fut_cc() will now refer to a synthetic dataset.

Let's take as new standard to estimate on 24 times along the year with time windows of 30, i.e. every 15 days a monthly scale estimate.

It sliently error returning a stripe instead of a tile.

Get bias_transforms._get_factors() assume lat is descending and longitude to be ascending, and sliently fails if otherwise. This checks the order and fails if not satisfied. Without this protection it silently fails and have misleaded before where is the error.

A decision to be made here. With the time dependent QDM/PresRat, we require a larger testing data to guarantee coverage and avoid NaN.

With a set it guarantee to run only on valid times.

Offset based on the characteristics of the dataset itself.

Re-organizing tests, combining redundant, and cleaning.

Now with a time dependent QDM, we need a full year coverage or we get NaN CDFs.

With the time dependent QDM, it requires a full coverage sample to avoid NaN.

Making clear who are the CC and FUT_CC datasets.

QDM changed the argument order to keep consistency with the rest of the library, thus presrat should also change.

@grantbuster

Error noticed by @grantbuster

Get factors was a chunk of original get_spatial_bc_factors() stripped out to be re-used and avoid redundancy. It is identical to original code to avoid breaking other parts of the package.

Documentation was outdated. With PresRat, there are even more parameters to recover, so to keep consistency this was changed to return a dictionary of parameters.

It was missing to force K=1, otherwise any difference in the means would result in a K that would change the unbiased.

@grantbuster

Removing apply_zero_precipitation_rate() as requested by @grantbuster

@grantbuster

If no_trend is True, skip zero rate and k-factor corrections. As requested by @grantbuster

@grantbuster

Thanks to @grantbuster who noticed that.

Some information for future refactoring.

@grantbuster

I missed this method from an early stage implementation. Thanks to @grantbuster who noticed that! Also defining default `zero_rate_threshold` to zero as requested by Grant.

@grantbuster

The rate should be <= otherwise we would miss completely the case of a threshold equal to zero. Thanks to @grantbuster who noticed that.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PresRat correction #215

PresRat correction #215

Commits on Jul 19, 2024

Commits on Jul 22, 2024

Commits on Jul 23, 2024