Skip to contents

Experiment

Functions for creating and running a simulation experiment.

Experiment
R6 class representing a simulation experiment.
create_experiment()
Create a new Experiment.
run_experiment()
Run the full Experiment pipeline (fitting, evaluating, and visualizing).
generate_data()
Generate data from each DGP in the Experiment.
fit_experiment()
Fit an Experiment.
evaluate_experiment()
Evaluate an Experiment.
visualize_experiment()
Visualize results of an Experiment.
add_dgp() add_method() add_evaluator() add_visualizer()
Helper functions for adding components to an Experiment.
update_dgp() update_method() update_evaluator() update_visualizer()
Helper functions for updating components of an Experiment.
remove_dgp() remove_method() remove_evaluator() remove_visualizer()
Helper functions for removing components of an Experiment.
get_dgps() get_methods() get_evaluators() get_visualizers()
Helper functions for getting components in an Experiment.
add_vary_across() update_vary_across() remove_vary_across() get_vary_across()
Varying across parameters in an Experiment.
clear_cache()
Clear cached results from Experiment.
get_cached_results()
Retrieve cached results from previously saved Experiment.
get_save_dir()
Get results directory for an Experiment.
set_save_dir()
Set results directory for an Experiment.
save_experiment()
Save an Experiment.
export_visualizers()
Export cached Visualizer results to image.

Simulation Experiment Directory Setup

Functions for automatically setting up a simulation experiment directory.

create_sim()
Create a simulation project

R Markdown Report

Functions for summarizing and visualizing results from a simulation experiment in a knitted R Markdown report.

set_doc_options()
Set R Markdown options for Evaluator and Visualizer outputs in summary report.
init_docs()
Initialize documentation template for the R Markdown results report.
render_docs()
Render an R Markdown file summarizing the results of an Experiment or set of Experiments.
create_rmd()
Render an R Markdown file summarizing the results of an Experiment or set of Experiments.

DGP

Functions for creating a DGP (data-generating process).

DGP
R6 class representing a data-generating process
create_dgp()
Create a new DGP (data-generating process)

Method

Functions for creating a Method.

Method
R6 class representing a method
create_method()
Create a new Method

Evaluator

Functions for creating an Evaluator.

Evaluator
R6 class representing an evaluator
create_evaluator()
Create a new Evaluator

Visualizer

Functions for creating a Visualizer.

Visualizer
R6 class representing a visualizer
create_visualizer()
Create a new Visualizer

Evaluator Library: Prediction

A library of common functions for evaluating prediction-focused simulation experiments.

eval_pred_err() summarize_pred_err()
Evaluate and/or summarize prediction errors.
eval_pred_curve() summarize_pred_curve()
Evaluate and/or summarize ROC or PR curves.

Evaluator Library: Feature Selection

A library of common functions for evaluating feature-selection-focused simulation experiments.

eval_feature_selection_err() summarize_feature_selection_err()
Evaluate and/or summarize feature selection errors.
eval_feature_selection_curve() summarize_feature_selection_curve()
Evaluate and/or summarize ROC or PR curves for feature selection.
eval_feature_importance() summarize_feature_importance()
Evaluate and/or summarize feature importance scores.

Evaluator Library: Inference

A library of common functions for evaluating inference-focused simulation experiments.

eval_testing_err() summarize_testing_err()
Evaluate and/or summarize error metrics when conducting multiple hypothesis tests.
eval_testing_curve() summarize_testing_curve()
Evaluate and/or summarize ROC or PR curves for feature rankings, ranked by p-value.
eval_reject_prob()
Evaluate the rejection probability of a hypothesis test.

Evaluator Library: Additional Yardstick Metrics

A library of additional yardstick metrics for evaluating simulation experiments.

fp() fp_vec()
Number of false positives
tp() tp_vec()
Number of true positives
neg() neg_vec()
Number of estimated negative cases
pos() pos_vec()
Number of estimated positive cases

Visualizer Library: Prediction

A library of common functions for visualizing prediction-focused simulation experiments.

plot_pred_err()
Plot prediction error according to various metrics.
plot_pred_curve()
Plot ROC/PR curves.

Visualizer Library: Feature Selection

A library of common functions for visualizing feature-selection-focused simulation experiments.

plot_feature_selection_err()
Plot feature selection error according to various metrics.
plot_feature_selection_curve()
Plot ROC/PR curves for feature selection.
plot_feature_importance()
Plot feature importances.

Visualizer Library: Inference

A library of common functions for visualizing inference-focused simulation experiments.

plot_testing_err()
Plot testing error evaluation results according to various metrics.
plot_testing_curve()
Plot ROC/PR curves for feature rankings, ranked by p-values.
plot_reject_prob()
Plot the rejection probability of a hypothesis test.

Boilerplate Templates

A library of minimalistic templates for running common types of simulation experiments.

use_prediction_template() use_feature_selection_template() use_inference_template()
Functions to create boilerplate code for specific types of experiments.

Testing Functions

Functions for test-driven simulation studies.

run_tests()
Run Tests
test_sim_dir()
Test simChef Ingredients

Developer Functions

Helper functions for developers to create their own simulation experiment components.

eval_constructor()
Developer function to construct basic Evaluator.
eval_summarizer()
Developer function for summarizing evaluation results.
plot_eval_constructor()
Developer function for plotting results from Evaluator.
plot_fit_constructor()
Developer function for plotting results from particular replicate(s) in the Experiment fit.
get_dot_args()
Get dot (...) arguments.
get_plot_data()
Developer function to get data for plotting in Visualizers.
list_col_to_chr()
Convert a list column to a readable character column.
load_all()
Load All Simulation Functions in R/
add_na_counts()
Developer function to add number of NAs to evaluator results