Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check the proper behavior of probability weighting normalization #44

Open
sami6mz opened this issue Aug 30, 2023 · 3 comments
Open

Check the proper behavior of probability weighting normalization #44

sami6mz opened this issue Aug 30, 2023 · 3 comments

Comments

@sami6mz
Copy link
Contributor

sami6mz commented Aug 30, 2023

As expressed in #42 and #43, we would like to check if normalization has a proper behavior.

"Normalization" refers to probability weighting normalization, as described in "Identifying causal mechanisms (primarily) based on inverse probability weighting", Huber (2014), DOI: 10.1002/jae.2341

We normalize the probabilities involved in score functions by dividing by their average on all the sample, respectively for treated and non-treated units. I describe that in my internship report page 22.

As far as I understood, it is a trick supposed to eliminate selection bias and make the estimation robust to extreme probability values.

A possible test would be to compare normalized and non-normalized estimators, or just to check the relative error like all other estimators (just add it to the tolerance dictionary).

As for now, normalization is implemented in med_dml and multiply_robust, which both give consistent results.
Normalization also is the very idea behind IPW estimator formulas.

@sami6mz
Copy link
Contributor Author

sami6mz commented Aug 30, 2023

By the way, @bthirion suggested to implement normalization into a function.
It is a bit tricky because it really depends on the estimator, but I guess it is doable.

@bthirion
Copy link
Collaborator

Sorry, my suggestion was actually a bit different (sorry if that was unclear): when you have 10-15 lines of code running a computation (any computation), you should cut that into an ancillary function. The point is that you can test that small function independently + it makes code more readable.

@judithabk6
Copy link
Owner

this has been implemented, and included in the tests as normalize is the default behavior, so not sure extra exploration or tests are needed as tests are already too long (#52). Probably should be included in a general discussion on tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants