Run the full Experiment pipeline (fitting, evaluating, and visualizing).

Run the full Experiment pipeline (fitting, evaluating, and visualizing).

Usage

run_experiment(
  experiment,
  n_reps = 1,
  parallel_strategy = c("reps"),
  future.globals = NULL,
  future.packages = NULL,
  future.seed = TRUE,
  use_cached = FALSE,
  return_all_cached_reps = FALSE,
  save = FALSE,
  checkpoint_n_reps = 0,
  verbose = 1,
  ...
)

Arguments

experiment: An Experiment object.
n_reps: The number of replicates of the Experiment for this run.
parallel_strategy: A vector with some combination of "reps", "dgps", or "methods". Determines how computation will be distributed across available resources. Default is "reps".
future.globals: Character vector of names in the global environment to pass to parallel workers. Passed as the argument of the same name to future.apply::future_lapply and related functions. To set for all runs of the experiment, use the same argument during initialization.
future.packages: Character vector of packages required by parallel workers. Passed as the argument of the same name to future.apply::future_lapply and related functions. To set for all runs of the experiment, use the same argument during initialization.
future.seed: Passed as the argument of the same name in future.apply::future_apply.
use_cached: Logical. If TRUE, find and return previously saved results. If cached results cannot be found, continue as if use_cached was FALSE.
return_all_cached_reps: Logical. If FALSE (default), returns only the fit results for the requested n_reps. If TRUE, returns fit results for the requested n_reps plus any additional cached replicates from the (DGP, Method) combinations in the Experiment. Note that even if return_all_cached_reps = TRUE, only the n_reps replicates are used when evaluating and visualizing the Experiment.
save: If TRUE, save outputs to disk.
checkpoint_n_reps: The number of experiment replicates to compute before saving results to disk. If 0 (the default), no checkpoints are saved.
verbose: Level of verbosity. Default is 1, which prints out messages after major checkpoints in the experiment. If 2, prints additional debugging information for warnings and messages from user-defined functions (in addition to error debugging information). If 0, no messages are printed other than user-defined function error debugging information.
...: Not used.

Value

A list of results from the simulation experiment.

fit_results: A tibble containing results from the fit method. In addition to results columns, has columns named '.rep', '.dgp_name', '.method_name', and the vary_across parameter names if applicable.
eval_results: A list of tibbles containing results from the evaluate method, which evaluates each Evaluator in the Experiment. Length of list is equivalent to the number of Evaluators.
viz_results: A list of tibbles containing results from the visualize method, which visualizes each Visualizer in the Experiment. Length of list is equivalent to the number of Visualizers.