Skip to content

Commit

Permalink
minor changes
Browse files Browse the repository at this point in the history
  • Loading branch information
enriquea committed Jan 16, 2024
1 parent 5c84bd2 commit e567d9b
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions fsspark/fs/multivariate.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ def multivariate_correlation_selector(fsdf: FSDataFrame,
Compute the correlation matrix (Pearson) among input features and select those below a specified threshold.
:param fsdf: Input FSDataFrame
:param strict: If True (default), apply hard filtering (strict) to remove highly related features.
:param strict: If True (default), apply hard filtering (strict) to remove highly correlated features.
Otherwise, find the maximal independent set of highly correlated features (approximate method).
The approximate method is experimental.
`Warning`: The approximate method is experimental.
:param corr_threshold: Minimal correlation threshold to consider two features correlated.
:param method: One of `pearson` (default) or `spearman`.
Expand Down
4 changes: 2 additions & 2 deletions fsspark/pipeline/fs_corr_rf.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from fsspark.fs.ml import cv_rf_classification, get_accuracy, get_predictions, get_feature_scores
from fsspark.fs.multivariate import multivariate_filter
from fsspark.fs.univariate import univariate_filter
from fsspark.fs.utils import filter_missingness_rate, impute_missing
from fsspark.fs.utils import remove_features_by_missingness_rate, impute_missing
from fsspark.utils.datasets import get_tnbc_data_path
from fsspark.utils.io import import_table_as_psdf

Expand All @@ -26,7 +26,7 @@
# Step 1. Data pre-processing.

# a) Filter missingness rate
fsdf = filter_missingness_rate(fsdf, threshold=0.1)
fsdf = remove_features_by_missingness_rate(fsdf, threshold=0.1)

# b) Impute data frame
fsdf = impute_missing(fsdf)
Expand Down

0 comments on commit e567d9b

Please sign in to comment.