Function reference • simChef

Experiment

Functions for creating and running a simulation experiment.

Experiment: R6 class representing a simulation experiment.

create_experiment(): Create a new Experiment.

run_experiment(): Run the full Experiment pipeline (fitting, evaluating, and visualizing).

generate_data(): Generate data from each DGP in the Experiment.

fit_experiment(): Fit an Experiment.

evaluate_experiment(): Evaluate an Experiment.

visualize_experiment(): Visualize results of an Experiment.

add_dgp() add_method() add_evaluator() add_visualizer(): Helper functions for adding components to an Experiment.

update_dgp() update_method() update_evaluator() update_visualizer(): Helper functions for updating components of an Experiment.

remove_dgp() remove_method() remove_evaluator() remove_visualizer(): Helper functions for removing components of an Experiment.

get_dgps() get_methods() get_evaluators() get_visualizers(): Helper functions for getting components in an Experiment.

add_vary_across() update_vary_across() remove_vary_across() get_vary_across(): Varying across parameters in an Experiment.

clear_cache(): Clear cached results from Experiment.

get_cached_results(): Retrieve cached results from previously saved Experiment.

get_save_dir(): Get results directory for an Experiment.

set_save_dir(): Set results directory for an Experiment.

save_experiment(): Save an Experiment.

export_visualizers(): Export cached Visualizer results to image.

Simulation Experiment Directory Setup

Functions for automatically setting up a simulation experiment directory.

create_sim(): Create a simulation project

R Markdown Report

Functions for summarizing and visualizing results from a simulation experiment in a knitted R Markdown report.

set_doc_options(): Set R Markdown options for Evaluator and Visualizer outputs in summary report.

init_docs(): Initialize documentation template for the R Markdown results report.

render_docs(): Render an R Markdown file summarizing the results of an Experiment or set of Experiments.

create_rmd(): Render an R Markdown file summarizing the results of an Experiment or set of Experiments.

DGP

Functions for creating a DGP (data-generating process).

DGP: R6 class representing a data-generating process

create_dgp(): Create a new DGP (data-generating process)

Method

Functions for creating a Method.

Method: R6 class representing a method

create_method(): Create a new Method

Evaluator

Functions for creating an Evaluator.

Evaluator: R6 class representing an evaluator

create_evaluator(): Create a new Evaluator

Visualizer

Functions for creating a Visualizer.

Visualizer: R6 class representing a visualizer

create_visualizer(): Create a new Visualizer

Evaluator Library: Prediction

A library of common functions for evaluating prediction-focused simulation experiments.

eval_pred_err() summarize_pred_err(): Evaluate and/or summarize prediction errors.

eval_pred_curve() summarize_pred_curve(): Evaluate and/or summarize ROC or PR curves.

Evaluator Library: Feature Selection

A library of common functions for evaluating feature-selection-focused simulation experiments.

eval_feature_selection_err() summarize_feature_selection_err(): Evaluate and/or summarize feature selection errors.

eval_feature_selection_curve() summarize_feature_selection_curve(): Evaluate and/or summarize ROC or PR curves for feature selection.

eval_feature_importance() summarize_feature_importance(): Evaluate and/or summarize feature importance scores.

Evaluator Library: Inference

A library of common functions for evaluating inference-focused simulation experiments.

eval_testing_err() summarize_testing_err(): Evaluate and/or summarize error metrics when conducting multiple hypothesis tests.

eval_testing_curve() summarize_testing_curve(): Evaluate and/or summarize ROC or PR curves for feature rankings, ranked by p-value.

eval_reject_prob(): Evaluate the rejection probability of a hypothesis test.

Evaluator Library: Additional Yardstick Metrics

A library of additional yardstick metrics for evaluating simulation experiments.

fp() fp_vec(): Number of false positives

tp() tp_vec(): Number of true positives

neg() neg_vec(): Number of estimated negative cases

pos() pos_vec(): Number of estimated positive cases

Visualizer Library: Prediction

A library of common functions for visualizing prediction-focused simulation experiments.

plot_pred_err(): Plot prediction error according to various metrics.

plot_pred_curve(): Plot ROC/PR curves.

Visualizer Library: Feature Selection

A library of common functions for visualizing feature-selection-focused simulation experiments.

plot_feature_selection_err(): Plot feature selection error according to various metrics.

plot_feature_selection_curve(): Plot ROC/PR curves for feature selection.

plot_feature_importance(): Plot feature importances.

Visualizer Library: Inference

A library of common functions for visualizing inference-focused simulation experiments.

plot_testing_err(): Plot testing error evaluation results according to various metrics.

plot_testing_curve(): Plot ROC/PR curves for feature rankings, ranked by p-values.

plot_reject_prob(): Plot the rejection probability of a hypothesis test.

Boilerplate Templates

A library of minimalistic templates for running common types of simulation experiments.

use_prediction_template() use_feature_selection_template() use_inference_template(): Functions to create boilerplate code for specific types of experiments.

Testing Functions

Functions for test-driven simulation studies.

run_tests(): Run Tests

test_sim_dir(): Test simChef Ingredients

Developer Functions

Helper functions for developers to create their own simulation experiment components.

eval_constructor(): Developer function to construct basic Evaluator.

eval_summarizer(): Developer function for summarizing evaluation results.

plot_eval_constructor(): Developer function for plotting results from Evaluator.

plot_fit_constructor(): Developer function for plotting results from particular replicate(s) in the Experiment fit.

get_dot_args(): Get dot (...) arguments.

get_plot_data(): Developer function to get data for plotting in Visualizers.

list_col_to_chr(): Convert a list column to a readable character column.

load_all(): Load All Simulation Functions in R/

add_na_counts(): Developer function to add number of NAs to evaluator results