Skip to content

Commit

Permalink
add illustration of approximate vs exact p-vals
Browse files Browse the repository at this point in the history
  • Loading branch information
qbarthelemy committed Jul 26, 2021
1 parent 82cdab6 commit cf65d9c
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 5 deletions.
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,13 @@ Ordinal and numerical
In permutation tests, the null distribution is sampled by random permutations
of measurements. Making no hypothesis on the statistic distribution, the
p-value is defined as the proportion of the null distribution with test
statistic greater (or lower, depending on the test) than or equal to the test
statistic of the observed data.
statistic greater (or, for certain test statistic, lower) than or equal to the
test statistic of the observed data.
When all permutations are evaluated, these permutation tests give
[exact tests](https://en.wikipedia.org/wiki/Exact_test).

![](/doc/fig_approx_vs_exact.png)

In case of several variables, they naturally perform a correction for
[multiple tests](https://en.wikipedia.org/wiki/Multiple_comparisons_problem):
- rmax permutation test from Pearson r correlations,
Expand Down Expand Up @@ -93,7 +95,7 @@ Hall/CRC, 2007.

## Documentation

TODO: put doc on RTD.
TODO: put documentation on RTD.

## Installation

Expand All @@ -116,5 +118,5 @@ Use `pytest`.

## Authors

Quentin Barthélemy, an anonymous researcher and Louis Mayaud,
Quentin Barthélemy, an anonymous researcher and Louis Mayaud,
under the wisdom of [Marco Congedo](https://github.com/Marco-Congedo).
Binary file added doc/fig_approx_vs_exact.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion examples/compute_exact_ttest_ind.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
plt.legend(loc='upper left')
ax2 = fig.add_subplot(122, title='Exact p-value', xlabel='t', sharey=ax1)
ax2.axvline(x=t_exact, c='r', label='Real statistic t')
y, x, _ = ax2.hist(t_dist, 50, density=True, alpha=0.4,
y, x, _ = ax2.hist(t_dist, 50, density=True, histtype='step',
label='Exact t-distribution')
ax2.fill_between(x[1:], y, where=(x[1:]>=t_exact), facecolor='r', step='pre',
label='Exact p-value')
Expand Down

0 comments on commit cf65d9c

Please sign in to comment.