diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 8db5bdebe..2adeb4843 100755 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -53,11 +53,11 @@ pages: - user_guide/data/wine_data.md - evaluate: - user_guide/evaluate/bootstrap.md + - user_guide/evaluate/BootstrapOutOufBag.md - user_guide/evaluate/confusion_matrix.md - user_guide/evaluate/lift_score.md - user_guide/evaluate/mcnemar_table.md - user_guide/evaluate/mcnemar.md - - user_guide/evaluate/OutOufBagBootstrap.md - user_guide/evaluate/permutation_test.md - user_guide/evaluate/scoring.md - feature_extraction: diff --git a/docs/sources/CHANGELOG.md b/docs/sources/CHANGELOG.md index fbfbed19e..6796f60af 100755 --- a/docs/sources/CHANGELOG.md +++ b/docs/sources/CHANGELOG.md @@ -20,7 +20,7 @@ The CHANGELOG for the current development version is available at - Added a `loadings_` attribute to `PrincipalComponentAnalysis` to compute the factor loadings of the features on the principal components. [#251](https://github.com/rasbt/mlxtend/pull/251) - Allow grid search over classifiers/regressors in ensemble and stacking estimators [#259](https://github.com/rasbt/mlxtend/pull/259) - New `make_multiplexer_dataset` function that creates a dataset generated by a n-bit Boolean multiplexer for evaluating supervised learning algorithms [#263](https://github.com/rasbt/mlxtend/pull/263) -- Added a new `OutOfBagBootstrap` class, an implementation of the out-of-bag bootstrap to evaluate supervised learning algorithms [#265](https://github.com/rasbt/mlxtend/pull/265) +- Added a new `BootstrapOutOfBag` class, an implementation of the out-of-bag bootstrap to evaluate supervised learning algorithms [#265](https://github.com/rasbt/mlxtend/pull/265) ##### Changes diff --git a/docs/sources/USER_GUIDE_INDEX.md b/docs/sources/USER_GUIDE_INDEX.md index 3bbc04a8d..f35a0cd60 100755 --- a/docs/sources/USER_GUIDE_INDEX.md +++ b/docs/sources/USER_GUIDE_INDEX.md @@ -25,11 +25,11 @@ ## `evaluate` - [bootstrap](user_guide/evaluate/bootstrap.md) +- [BootstrapOutOfBag](user_guide/evaluate/BootstrapOutOfBag.md) - [confusion_matrix](user_guide/evaluate/confusion_matrix.md) - [lift_score](user_guide/evaluate/lift_score.md) - [mcnemar_table](user_guide/evaluate/mcnemar_table.md) - [mcnemar](user_guide/evaluate/mcnemar.md) -- [OutOufBagBootstrap](user_guide/evaluate/OutOfBagBootstrap.md) - [permutation_test](user_guide/evaluate/permutation_test.md) - [scoring](user_guide/evaluate/scoring.md) diff --git a/docs/sources/user_guide/evaluate/OutOfBagBootstrap.ipynb b/docs/sources/user_guide/evaluate/BootstrapOutOfBag.ipynb similarity index 95% rename from docs/sources/user_guide/evaluate/OutOfBagBootstrap.ipynb rename to docs/sources/user_guide/evaluate/BootstrapOutOfBag.ipynb index 202bd3f16..dd5bfa3b8 100644 --- a/docs/sources/user_guide/evaluate/OutOfBagBootstrap.ipynb +++ b/docs/sources/user_guide/evaluate/BootstrapOutOfBag.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# OutOfBagBootstrap" + "# BootstrapOutOfBag" ] }, { @@ -18,7 +18,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "> `from mlxtend.evaluate import OutOfBagBootstrap` " + "> `from mlxtend.evaluate import BootstrapOutOfBag` " ] }, { @@ -35,7 +35,7 @@ "Originally, the bootstrap method aims to determine the statistical properties of an estimator when the underlying distribution was unknown and additional samples are not available. Now, in order to exploit this method for the evaluation of predictive models, such as hypotheses for classification and regression, we may prefer a slightly different approach to bootstrapping using the so-called Out-Of-Bag (OOB) or Leave-One-Out Bootstrap (LOOB) technique. Here, we use out-of-bag samples as test sets for evaluation instead of evaluating the model on the training data. Out-of-bag samples are the unique sets of instances that are not used for model fitting as shown in the figure below [1].\n", "\n", "\n", - "![](OutOfBagBootstrap_files/bootrap_concept.png)\n", + "![](BootstrapOutOfBag_files/bootrap_concept.png)\n", "\n", "\n", "The figure above illustrates how three random bootstrap samples drawn from an exemplary ten-sample dataset ($X_1,X_2, ..., X_{10}$) and their out-of-bag sample for testing may look like. In practice, Bradley Efron and Robert Tibshirani recommend drawing 50 to 200 bootstrap samples as being sufficient for reliable estimates [2]." @@ -62,7 +62,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The `OutOfBagBootstrap` class mimics the behavior of scikit-learn's cross-validation classes, e.g., `KFold`:" + "The `BootstrapOutOfBag` class mimics the behavior of scikit-learn's cross-validation classes, e.g., `KFold`:" ] }, { @@ -81,11 +81,11 @@ } ], "source": [ - "from mlxtend.evaluate import OutOfBagBootstrap\n", + "from mlxtend.evaluate import BootstrapOutOfBag\n", "import numpy as np\n", "\n", "\n", - "oob = OutOfBagBootstrap(n_splits=3)\n", + "oob = BootstrapOutOfBag(n_splits=3)\n", "for train, test in oob.split(np.array([1, 2, 3, 4, 5])):\n", " print(train, test)" ] @@ -94,7 +94,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Consequently, we can use `OutOfBagBootstrap` objects via the `cross_val_score` method:" + "Consequently, we can use `BootstrapOutOfBag` objects via the `cross_val_score` method:" ] }, { @@ -137,7 +137,7 @@ } ], "source": [ - "print(cross_val_score(lr, X, y, cv=OutOfBagBootstrap(n_splits=3, random_seed=456)))" + "print(cross_val_score(lr, X, y, cv=BootstrapOutOfBag(n_splits=3, random_seed=456)))" ] }, { @@ -162,7 +162,7 @@ ], "source": [ "print('Mean accuracy: %.1f%%' % np.mean(100*cross_val_score(\n", - " lr, X, y, cv=OutOfBagBootstrap(n_splits=200, random_seed=456))))" + " lr, X, y, cv=BootstrapOutOfBag(n_splits=200, random_seed=456))))" ] }, { @@ -206,7 +206,7 @@ } ], "source": [ - "accuracies = cross_val_score(lr, X, y, cv=OutOfBagBootstrap(n_splits=1000, random_seed=456))\n", + "accuracies = cross_val_score(lr, X, y, cv=BootstrapOutOfBag(n_splits=1000, random_seed=456))\n", "mean = np.mean(accuracies)\n", "\n", "lower = np.percentile(accuracies, 2.5)\n", @@ -243,9 +243,9 @@ "name": "stdout", "output_type": "stream", "text": [ - "## OutOfBagBootstrap\n", + "## BootstrapOutOfBag\n", "\n", - "*OutOfBagBootstrap(n_splits=200, random_seed=None)*\n", + "*BootstrapOutOfBag(n_splits=200, random_seed=None)*\n", "\n", "**Parameters**\n", "\n", @@ -328,7 +328,7 @@ } ], "source": [ - "with open('../../api_modules/mlxtend.evaluate/OutOfBagBootstrap.md', 'r') as f:\n", + "with open('../../api_modules/mlxtend.evaluate/BootstrapOutOfBag.md', 'r') as f:\n", " s = f.read() \n", "print(s)" ] diff --git a/mlxtend/evaluate/__init__.py b/mlxtend/evaluate/__init__.py index 388051f3d..329c86666 100644 --- a/mlxtend/evaluate/__init__.py +++ b/mlxtend/evaluate/__init__.py @@ -10,10 +10,10 @@ from .mcnemar import mcnemar_table from .mcnemar import mcnemar from .bootstrap import bootstrap -from .outofbag_bootstrap import OutOfBagBootstrap +from .bootstrap_outofbag import BootstrapOutOfBag from .permutation import permutation_test __all__ = ["scoring", "confusion_matrix", "mcnemar_table", "mcnemar", "lift_score", - "bootstrap", "permutation_test", "OutOfBagBootstrap"] + "bootstrap", "permutation_test", "BootstrapOutOfBag"] diff --git a/mlxtend/evaluate/outofbag_bootstrap.py b/mlxtend/evaluate/bootstrap_outofbag.py similarity index 98% rename from mlxtend/evaluate/outofbag_bootstrap.py rename to mlxtend/evaluate/bootstrap_outofbag.py index 2b1241bd4..15c172337 100644 --- a/mlxtend/evaluate/outofbag_bootstrap.py +++ b/mlxtend/evaluate/bootstrap_outofbag.py @@ -9,7 +9,7 @@ import numpy as np -class OutOfBagBootstrap(object): +class BootstrapOutOfBag(object): """ Parameters ---------- diff --git a/mlxtend/evaluate/tests/test_outofbag_bootstrap.py b/mlxtend/evaluate/tests/test_bootstrap_outofbag.py similarity index 79% rename from mlxtend/evaluate/tests/test_outofbag_bootstrap.py rename to mlxtend/evaluate/tests/test_bootstrap_outofbag.py index ec5ab0c0d..6ff82af4f 100644 --- a/mlxtend/evaluate/tests/test_outofbag_bootstrap.py +++ b/mlxtend/evaluate/tests/test_bootstrap_outofbag.py @@ -5,18 +5,18 @@ # License: BSD 3 clause import numpy as np -from mlxtend.evaluate import OutOfBagBootstrap +from mlxtend.evaluate import BootstrapOutOfBag from mlxtend.utils import assert_raises def test_defaults(): - oob = OutOfBagBootstrap() + oob = BootstrapOutOfBag() results = list(oob.split(np.array([1, 2, 3, 4, 5]))) assert len(results) == 200 def test_splits(): - oob = OutOfBagBootstrap(n_splits=3, random_seed=123) + oob = BootstrapOutOfBag(n_splits=3, random_seed=123) results = list(oob.split(np.array([1, 2, 3, 4, 5]))) assert len(results) == 3 assert np.array_equal(results[0][0], np.array([2, 4, 2, 1, 3])) @@ -28,10 +28,10 @@ def test_splits(): def test_invalid_splits(): assert_raises(ValueError, 'Number of splits must be greater than 1.', - OutOfBagBootstrap, + BootstrapOutOfBag, 0) def test_get_n_splits(): - oob = OutOfBagBootstrap(n_splits=3, random_seed=123) + oob = BootstrapOutOfBag(n_splits=3, random_seed=123) assert oob.n_splits == 3