Learn R Programming

familiar (version 1.5.0)

End-to-End Automated Machine Learning and Model Evaluation

Description

Single unified interface for end-to-end modelling of regression, categorical and time-to-event (survival) outcomes. Models created using familiar are self-containing, and their use does not require additional information such as baseline survival, feature clustering, or feature transformation and normalisation parameters. Model performance, calibration, risk group stratification, (permutation) variable importance, individual conditional expectation, partial dependence, and more, are assessed automatically as part of the evaluation process and exported in tabular format and plotted, and may also be computed manually using export and plot functions. Where possible, metrics and values obtained during the evaluation process come with confidence intervals.

Copy Link

Version

Install

install.packages('familiar')

Monthly Downloads

320

Version

1.5.0

License

EUPL

Issues

Pull Requests

Stars

Forks

Maintainer

Alex Zwanenburg

Last Published

September 23rd, 2024

Functions in familiar (1.5.0)

.check_plot_splitting_variables

Checks and sanitizes splitting variables for plotting.
.check_input_plot_args

Internal checks on common plot input arguments
.impute_outcome_type

Internal imputation function for the outcome type.
.check_outcome_type_plausibility

Internal function for checking if the outcome type fits well to the data
.check_input_identifier_column

Internal function for checking consistency of the identifier columns
.check_survival_time_plausibility

Internal function to test plausibility of provided survival times.
.get_default_sign_size

Internal function for obtaining a default signature size parameter
.load_iterations

Internal function for loading iteration data from the file system
.get_iteration_data

Internal function for creating or retrieving iteration data
.finish_data_preparation

Internal function for finalising generic data processing
.parse_file_paths

Internal function for parsing file paths
.parse_integer_features

Internal function for converting integer features
.parse_model_development_settings

Internal function for parsing settings related to model development
.parse_experiment_settings

Internal function for parsing settings related to the experimental setup
.parse_categorical_features

Internal function for setting categorical features
.parse_evaluation_settings

Internal function for parsing settings related to model evaluation
.parse_feature_selection_settings

Internal function for parsing settings related to feature selection
.parse_initial_settings

Internal function for parsing settings required to parse the input data and define the experiment
.parse_hyperparameter_optimisation_settings

Internal function for parsing settings related to hyperparameter optimisation
.parse_general_settings

Internal function for parsing settings that configure various aspects of the worklow
.prepare_familiar_data_sets

Prepare familiarData objects for evaluation at runtime.
export_all

Extract and export all data.
experimentData-class

Experiment data
.update_experimental_design_settings

Internal function to check batch assignment to development and validation
.plot_univariate_importance

Internal plotting function for univariate plots
.plot_permutation_variable_importance

Internal plotting function for permutation variable importance plots
.parse_setup_settings

Internal function for parsing settings related to the computational setup
.update_initial_settings

Internal check and update of settings related to data set parsing
.parse_preprocessing_settings

Internal function for parsing settings related to preprocessing
encapsulate_path

Encapsulate path
export_decision_curve_analysis_data

Extract and export decision curve analysis data.
export_auc_data

Extract and export ROC and Precision-Recall curves.
export_calibration_info

Extract and export calibration information.
export_confusion_matrix_data

Extract and export confusion matrices.
export_feature_similarity

Extract and export mutual correlation between features.
export_fs_vimp

Extract and export feature selection variable importance.
export_hyperparameters

Extract and export model hyperparameters.
export_calibration_data

Extract and export calibration and goodness-of-fit tests.
export_ice_data

Extract and export individual conditional expectation data.
export_feature_expressions

Extract and export feature expressions.
export_model_performance

Extract and export metrics for model performance.
export_univariate_analysis_data

Extract and export univariate analysis data of features.
export_risk_stratification_info

Extract and export cut-off values for risk group stratification.
export_prediction_data

Extract and export predicted values.
export_model_vimp

Extract and export model-based variable importance.
export_sample_similarity

Extract and export mutual correlation between features.
export_risk_stratification_data

Extract and export sample risk group stratification and associated tests.
extract_auc_data

Internal function to extract area under the ROC curve information.
export_partial_dependence_data

Extract and export partial dependence data.
export_permutation_vimp

Extract and export permutation variable importance.
extract_confusion_matrix

Internal function to extract the confusion matrix.
extract_calibration_data

Internal function to extract calibration data.
extract_calibration_info

Internal function to extract calibration info from data.
extract_decision_curve_data

Internal function to extract decision curve analysis data.
extract_feature_expression

Internal function to extract feature expressions.
extract_experimental_setup

Parse experimental design
extract_data

Internal function to create a familiarData object.
extract_hyperparameters

Internal function to extract hyperparameters from models.
familiarNoveltyDetector-class

Novelty detector.
extract_dispatcher,familiarEnsemble,familiarDataElement-method

Internal function to dispatch extraction functions.
familiarVimpMethod-class

Variable importance method object.
get_learner_names,familiarCollection-method

Get current learner name labels
extract_performance

Internal function to extract performance metrics.
familiarModel-class

Familiar model.
familiarMetric-class

Model performance metric.
extract_model_vimp

Internal function to extract variable importance from models.
extract_permutation_vimp

Internal function to extract permutation variable importance.
extract_sample_similarity

Internal function to extract the sample distance table.
extract_univariate_analysis

Internal function to extract data from a univariate analysis.
featureInfoParameters-class

Feature information parameters object.
familiar

familiar: Fully Automated Machine Learning with Interpretable Analysis of Results
featureInfo-class

Feature information object.
extract_ice

Internal function to extract data for individual conditional expectation plots.
extract_predictions

Internal function to extract predicted values from models.
get_risk_group_names,familiarCollection-method

Get current risk group labels
familiarDataElement-class

Data container for evaluation data.
get_data_set_names,familiarCollection-method

Get current name of datasets
familiarData-class

Dataset obtained after evaluating models on a dataset.
get_class_names,familiarCollection-method

Get outcome class labels
get_vimp_table

Extract variable importance table.
get_xml_config

Create an empty xml configuration file
familiarCollection-class

Collection of familiar data.
extract_feature_similarity

Internal function to extract the feature distance table.
extract_fs_vimp

Internal function to extract feature selection variable importance.
extract_risk_stratification_data

Internal function to extract stratification data.
familiarEnsemble-class

Ensemble of familiar models.
outcomeInfo-class

Outcome information object.
plot_auc_precision_recall_curve

Plot the precision-recall curve.
extract_risk_stratification_info

Internal function to extract risk stratification info from data.
plot_auc_roc_curve

Plot the receiver operating characteristic curve.
is.waive

Internal test to see if an object is a waiver
plot_calibration_data

Plot calibration figures.
is.encapsulated_path

Internal test for encapsulated_path
familiarHyperparameterLearner-class

Hyperparameter learner.
get_feature_names,familiarCollection-method

Get current feature labels
plot_decision_curve

Plot decision curves.
plot_confusion_matrix

Plot confusion matrix.
plot_feature_similarity

Plot heatmaps for pairwise similarity between features.
get_fs_method_names,familiarCollection-method

Get current feature selection method name labels
plot_ice

Plot individual conditional expectation plots.
create_randomised_groups

Create randomised groups Creates randomised groups, e.g. for tests that depend on splitting (continuous) data into groups, such as the Hosmer-Lemeshow test
dataObject-class

Data object
aggregate_vimp_table

Aggregate variable importance from multiple variable importance objects.
as_familiar_collection

Conversion to familiarCollection object.
as_familiar_data

Conversion to familiarData object.
as_familiar_ensemble

Conversion to familiarEnsemble object.
as_data_object

Creates a valid data object from input data.
coef

Extract model coefficients
.check_feature_availability

Internal function to check whether feature columns are found in the data
.check_class_level_plausibility

Internal function to test plausibility of provided class levels