Run the full Experiment
pipeline (fitting, evaluating, and visualizing).
Source: R/experiment-helpers.R
run_experiment.Rd
Run the full Experiment
pipeline (fitting, evaluating, and visualizing).
Usage
run_experiment(
experiment,
n_reps = 1,
parallel_strategy = c("reps"),
future.globals = NULL,
future.packages = NULL,
future.seed = TRUE,
use_cached = FALSE,
return_all_cached_reps = FALSE,
save = FALSE,
checkpoint_n_reps = 0,
verbose = 1,
...
)
Arguments
- experiment
An
Experiment
object.- n_reps
The number of replicates of the
Experiment
for this run.- parallel_strategy
A vector with some combination of "reps", "dgps", or "methods". Determines how computation will be distributed across available resources. Default is "reps".
- future.globals
Character vector of names in the global environment to pass to parallel workers. Passed as the argument of the same name to
future.apply::future_lapply
and related functions. To set for all runs of the experiment, use the same argument during initialization.- future.packages
Character vector of packages required by parallel workers. Passed as the argument of the same name to
future.apply::future_lapply
and related functions. To set for all runs of the experiment, use the same argument during initialization.- future.seed
Passed as the argument of the same name in
future.apply::future_apply
.- use_cached
Logical. If
TRUE
, find and return previously saved results. If cached results cannot be found, continue as ifuse_cached
wasFALSE
.- return_all_cached_reps
Logical. If
FALSE
(default), returns only the fit results for the requestedn_reps
. IfTRUE
, returns fit results for the requestedn_reps
plus any additional cached replicates from the (DGP
,Method
) combinations in theExperiment
. Note that even ifreturn_all_cached_reps = TRUE
, only then_reps
replicates are used when evaluating and visualizing theExperiment
.- save
If
TRUE
, save outputs to disk.- checkpoint_n_reps
The number of experiment replicates to compute before saving results to disk. If 0 (the default), no checkpoints are saved.
- verbose
Level of verbosity. Default is 1, which prints out messages after major checkpoints in the experiment. If 2, prints additional debugging information for warnings and messages from user-defined functions (in addition to error debugging information). If 0, no messages are printed other than user-defined function error debugging information.
- ...
Not used.
Value
A list of results from the simulation experiment.
- fit_results
A tibble containing results from the
fit
method. In addition to results columns, has columns named '.rep', '.dgp_name', '.method_name', and thevary_across
parameter names if applicable.- eval_results
A list of tibbles containing results from the
evaluate
method, which evaluates eachEvaluator
in theExperiment
. Length of list is equivalent to the number ofEvaluators
.- viz_results
A list of tibbles containing results from the
visualize
method, which visualizes eachVisualizer
in theExperiment
. Length of list is equivalent to the number ofVisualizers
.