Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving the efficiency of the Gaussian and (Gaussian) copula methods #366

Merged
merged 64 commits into from
Jan 15, 2024
Merged
Show file tree
Hide file tree
Changes from 53 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
d637952
Comparing different versions of Gaussian method
LHBO Dec 4, 2023
4c6bead
Hashed out large `n_samples`
LHBO Dec 4, 2023
cbf53a2
Commented out shapr
LHBO Dec 4, 2023
8045102
Merge master into branch
LHBO Dec 4, 2023
87fdc54
Moved file to `inst/scripts/` as tests fails
LHBO Dec 4, 2023
a91e3ad
adding rnorm alternative for version 5
martinju Dec 8, 2023
0d67b17
Added new version of v5_rnorm without sweep. 20% faster
LHBO Dec 12, 2023
9d0ef5b
Added gaussian cpp code
LHBO Dec 18, 2023
7737587
added gaussian cpp in comparison file
LHBO Dec 18, 2023
9d493a5
update cpp supporting files
LHBO Dec 18, 2023
6d2eb56
Removed redudant parameters
LHBO Dec 18, 2023
ed5b87a
logical error used `n_rows` instead of `n_cols`
LHBO Dec 18, 2023
530c816
Prøvde å gjøre koden raskere, men blitt tregere(?)
LHBO Dec 18, 2023
1e8c593
typo
LHBO Dec 18, 2023
e4eb606
Tried to see if I could make it speed it up.
LHBO Dec 20, 2023
3f038eb
Here we add different cpp versions to find the fastest one
LHBO Jan 4, 2024
b0c2180
Arma::cube with efficient indicing is the fastest
LHBO Jan 4, 2024
75aea94
Added cpp functions here so we have them in the future (if needed)
LHBO Jan 5, 2024
b1baed1
Made cube_v2 the main cpp version and removed others
LHBO Jan 5, 2024
4162364
Updated `approach_gaussian()` to use Cpp code. Verified that we get t…
LHBO Jan 5, 2024
adf57e5
Moved function to inst/compare_gaussian.R so it is still runable.
LHBO Jan 5, 2024
74b1ae2
Added `#' @inheritParams default_doc`
LHBO Jan 5, 2024
9590605
Changed order of parameters
LHBO Jan 5, 2024
df9777d
Removed `if (!is.null(index_features))` in `approach_gaussian()`.
LHBO Jan 5, 2024
11819e8
Need to clean code. Push in case something happens with my computer.
LHBO Jan 5, 2024
92ccff3
approach_gaussian: removed not used variables + stylr + added `data.t…
LHBO Jan 6, 2024
ed41f91
Removed quotation marks in dt
LHBO Jan 6, 2024
8833bb7
Start to clean up copula.cpp
LHBO Jan 6, 2024
5a20f69
deleted not used functions in copula.cpp
LHBO Jan 6, 2024
63a911f
Roxygen gaussian
LHBO Jan 7, 2024
24ec723
Roxygen
LHBO Jan 7, 2024
b7ece67
Cleaned up Copula.cpp
LHBO Jan 7, 2024
bc3df32
Updated approach_copula.R
LHBO Jan 7, 2024
0ad95d2
Roxygen
LHBO Jan 7, 2024
ffba299
Deleted unnecessary/old R code from approach_copula.R
LHBO Jan 7, 2024
cf9a5b4
Updated RcppExports
LHBO Jan 7, 2024
3a14985
Renamed illustration script
LHBO Jan 7, 2024
da8f03c
Adding script to compare the R and C++ versions.
LHBO Jan 7, 2024
ff0b2b4
lintr + stylr
LHBO Jan 7, 2024
1de33a1
typo
LHBO Jan 7, 2024
6d6d7bc
Added a copula version that uses cpp, but r for quantile
LHBO Jan 7, 2024
8eec712
Updated comparison file
LHBO Jan 7, 2024
a0f6132
Ran code several time to get more CPU times
LHBO Jan 7, 2024
5bd7be6
stylr
LHBO Jan 7, 2024
19a92be
Updated manuals/documentation
LHBO Jan 7, 2024
cbb06ed
Updated versions with working and efficient C++ code
LHBO Jan 9, 2024
a1e1767
Updated manuals and NAMESPACE
LHBO Jan 9, 2024
f2aa827
Added comparison of shapr and rcpp compile
LHBO Jan 11, 2024
e0a784b
Added to ignore .DS_Store (mac file created when looking at folder el…
LHBO Jan 11, 2024
2613e6e
Fixed TODO tasks in approach_copula.R
LHBO Jan 11, 2024
cfd0b24
Stylr
LHBO Jan 11, 2024
ccfbd59
Removed debug funciton
LHBO Jan 11, 2024
71b8f40
Added comparisons to when we compile with Rcpp::sourceCpp in all exam…
LHBO Jan 11, 2024
33d7391
Added test that shows that cpp and R gives same values when n_samples…
LHBO Jan 12, 2024
7634b38
Go back to using R for gaussian_transform and gaussian_transform_sepa…
LHBO Jan 12, 2024
4db7aac
Typo in Copula.cpp
LHBO Jan 12, 2024
e969bd9
Added code that Shows that C++ code produce equivalent Shapley values…
LHBO Jan 12, 2024
0bd1f84
Updated RcppExports
LHBO Jan 12, 2024
8f111a6
Removed unneeded variable
LHBO Jan 12, 2024
df1c276
man
martinju Jan 15, 2024
e9c5d4f
test files
martinju Jan 15, 2024
21d0599
clear unused packages
martinju Jan 15, 2024
5b394d5
Merge remote-tracking branch 'origin/master' into Improve_Gaussian_Lars
martinju Jan 15, 2024
3ea3d9d
rerun tests after master merge
martinju Jan 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 2 additions & 12 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,22 @@
Rprof.out
src/*.o
src/*.so

*.o

*.so

tmp/

\.RDataTmp

*.out

*.Rout

src-i386/

src-x64/

paper_experiments/res/single_res/

paper_experiments/res/single_res_old/
inst/doc

docs/*
doc
Meta
docs
/doc/
/Meta/
.idea
.idea
.DS_Store
2 changes: 2 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ export(mahalanobis_distance_cpp)
export(observation_impute_cpp)
export(predict_model)
export(prepare_data)
export(prepare_data_copula_cpp)
export(prepare_data_gaussian_cpp)
export(rss_cpp)
export(setup)
export(setup_approach)
Expand Down
109 changes: 109 additions & 0 deletions R/RcppExports.R
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,115 @@ aicc_full_cpp <- function(h, X_list, mcov_list, S_scale_dist, y_list, negative)
.Call(`_shapr_aicc_full_cpp`, h, X_list, mcov_list, S_scale_dist, y_list, negative)
}

#' Compute the quantiles using quantile type seven
#'
#' @details Using quantile type number seven from stats::quantile in R.
#'
#' @param x arma::vec. Numeric vector whose sample quantiles are wanted.
#' @param probs arma::vec. Numeric vector of probabilities with values between zero and one.
#'
#' @return A vector of length `length(probs)` with the quantiles is returned.
#'
#' @keywords internal
#' @author Lars Henry Berge Olsen
quantile_type7_cpp <- function(x, probs) {
.Call(`_shapr_quantile_type7_cpp`, x, probs)
}

#' Transforms new data to a standardized normal distribution
#'
#' @details The function uses `arma::quantile(...)` which corresponds to R's `stats::quantile(..., type = 5)`.
#'
#' @param z arma::mat. The data are the Gaussian Monte Carlos samples to transform.
#' @param x arma::mat. The data with the original transformation. Used to conduct the transformation of `z`.
#'
#' @return arma::mat of the same dimension as `z`
#'
#' @keywords internal
#' @author Lars Henry Berge Olsen
inv_gaussian_transform_cpp <- function(z, x) {
.Call(`_shapr_inv_gaussian_transform_cpp`, z, x)
}

#' Generate (Gaussian) Copula MC samples
#'
#' @param MC_samples_mat arma::mat. Matrix of dimension (`n_samples`, `n_features`) containing samples from the
#' univariate standard normal.
#' @param x_explain_mat arma::mat. Matrix of dimension (`n_explain`, `n_features`) containing the observations
#' to explain on the original scale.
#' @param x_explain_gaussian_mat arma::mat. Matrix of dimension (`n_explain`, `n_features`) containing the
#' observations to explain after being transformed using the Gaussian transform, i.e., the samples have been
#' transformed to a standardized normal distribution.
#' @param x_train_mat arma::mat. Matrix of dimension (`n_train`, `n_features`) containing the training observations.
#' @param S arma::mat. Matrix of dimension (`n_combinations`, `n_features`) containing binary representations of
#' the used coalitions. S cannot contain the empty or grand coalition, i.e., a row containing only zeros or ones.
#' This is not a problem internally in shapr as the empty and grand coalitions treated differently.
#' @param mu arma::vec. Vector of length `n_features` containing the mean of each feature after being transformed
#' using the Gaussian transform, i.e., the samples have been transformed to a standardized normal distribution.
#' @param cov_mat arma::mat. Matrix of dimension (`n_features`, `n_features`) containing the pairwise covariance
#' between all pairs of features after being transformed using the Gaussian transform, i.e., the samples have been
#' transformed to a standardized normal distribution.
#'
#' @return An arma::cube/3D array of dimension (`n_samples`, `n_explain` * `n_coalitions`, `n_features`), where
#' the columns (_,j,_) are matrices of dimension (`n_samples`, `n_features`) containing the conditional Gaussian
#' copula MC samples for each explicand and coalition on the original scale.
#'
#' @export
#' @keywords internal
#' @author Lars Henry Berge Olsen
prepare_data_copula_cpp <- function(MC_samples_mat, x_explain_mat, x_explain_gaussian_mat, x_train_mat, S, mu, cov_mat) {
.Call(`_shapr_prepare_data_copula_cpp`, MC_samples_mat, x_explain_mat, x_explain_gaussian_mat, x_train_mat, S, mu, cov_mat)
}

#' Transforms a sample to standardized normal distribution
#'
#' @param x Numeric matrix. The data which should be transformed to a standard normal distribution.
#'
#' @return Numeric matrix of dimension `dim(x)`
#'
#' @keywords internal
#' @author Lars Henry Berge Olsen
gaussian_transform_cpp <- function(x) {
.Call(`_shapr_gaussian_transform_cpp`, x)
}

#' Transforms new data to standardized normal (column-wise) based on other data transformations
#'
#' @param y arma::mat. A numeric matrix containing the data that is to be transformed.
#' @param x arma::mat. A numeric matrix containing the data of the original transformation.
#'
#' @return An arma::mat matrix of the same dimension as `y` containing the back-transformed Gaussian data.
#'
#' @keywords internal
#' @author Lars Henry Berge Olsen, Martin Jullum
gaussian_transform_separate_cpp <- function(y, x) {
.Call(`_shapr_gaussian_transform_separate_cpp`, y, x)
}

#' Generate Gaussian MC samples
#'
#' @param MC_samples_mat arma::mat. Matrix of dimension (`n_samples`, `n_features`) containing samples from the
#' univariate standard normal.
#' @param x_explain_mat arma::mat. Matrix of dimension (`n_explain`, `n_features`) containing the observations
#' to explain.
#' @param S arma::mat. Matrix of dimension (`n_combinations`, `n_features`) containing binary representations of
#' the used coalitions. S cannot contain the empty or grand coalition, i.e., a row containing only zeros or ones.
#' This is not a problem internally in shapr as the empty and grand coalitions treated differently.
#' @param mu arma::vec. Vector of length `n_features` containing the mean of each feature.
#' @param cov_mat arma::mat. Matrix of dimension (`n_features`, `n_features`) containing the pairwise covariance
#' between all pairs of features.
#'
#' @return An arma::cube/3D array of dimension (`n_samples`, `n_explain` * `n_coalitions`, `n_features`), where
#' the columns (_,j,_) are matrices of dimension (`n_samples`, `n_features`) containing the conditional Gaussian
#' MC samples for each explicand and coalition.
#'
#' @export
#' @keywords internal
#' @author Lars Henry Berge Olsen
prepare_data_gaussian_cpp <- function(MC_samples_mat, x_explain_mat, S, mu, cov_mat) {
.Call(`_shapr_prepare_data_gaussian_cpp`, MC_samples_mat, x_explain_mat, S, mu, cov_mat)
}

#' (Generalized) Mahalanobis distance
#'
#' Used to get the Euclidean distance as well by setting \code{mcov} = \code{diag(m)}.
Expand Down
Loading
Loading