Skip to contents

Helper function to get the summarized evaluation results tibble for plotting. This function will compute the summarized evaluation results if they have not been computed previously. Otherwise, it will read in the previously computed results and compute and append any new results necessary to construct the specified plot.

Usage

get_eval_tibble(
  fit_results,
  eval_tib = NULL,
  eval_id = NULL,
  eval_fun = paste0("summarize_", eval_id),
  vary_params = NULL,
  show,
  y_str = NULL,
  ...
)

Arguments

fit_results

A tibble, as returned by the fit method.

eval_tib

(Optional) Tibble (typically from the output of eval_summary_constructor) containing the summarized evaluation results to plot. If not provided, the evaluation results will be automatically computed by calling eval_fun(). If the summarized evaluation results have already been computed previously, eval_tib should be specified to avoid duplicate computations.

eval_id

Character string. ID used as the suffix for naming columns in eval_summary_constructor(). Should be the same as the eval_id argument in eval_summary_constructor().

eval_fun

Function used to compute evaluation results summary. This function is only used (and required) if necessary results have not already been computed in eval_tib.

vary_params

A vector of parameter names that are varied across in the Experiment.

show

Character vector with elements being one of "boxplot", "point", "line", "bar", "errorbar", "ribbon" indicating what plot layer(s) to construct.

y_str

(Optional) Name of column in eval_tib to plot on the y-axis if show is anything but "boxplot". Default "auto" chooses what to plot on the y-axis automatically.

...

Additional arguments to pass to eval_fun(). This is only used if necessary results have not already been computed in eval_tib.

Value

A tibble with the summarized evaluation results.

Examples

# generate example fit results data
fit_results <- tibble::tibble(
  .rep = rep(1:2, times = 2),
  .dgp_name = c("DGP1", "DGP1", "DGP2", "DGP2"),
  .method_name = c("Method"),
  # true response
  y = lapply(1:4, FUN = function(x) rnorm(100)),
  # predicted response
  predictions = lapply(1:4, FUN = function(x) rnorm(100))
)

# compute (from scratch) the evaluation results that are necessary for plotting
eval_tib <- get_eval_tibble(fit_results = fit_results, 
                            eval_tib = NULL,
                            eval_id = "pred_err",
                            eval_fun = "summarize_pred_err", 
                            show = c("point", "errorbar"),
                            y_str = NULL,  # or equivalently, "mean_pred_err"
                            # arguments to pass to `eval_fun`
                            truth_col = "y",
                            estimate_col = "predictions")
# this is equivalent to:
eval_tib2 <- summarize_pred_err(
  fit_results = fit_results, 
  truth_col = "y",
  estimate_col = "predictions",
  summary_funs = c("mean", "sd")
)
all.equal(eval_tib, eval_tib2)
#> [1] TRUE

# read in pre-computed evaluation results since it has everything needed for plotting
eval_tib <- get_eval_tibble(fit_results = fit_results,
                            eval_tib = eval_tib, 
                            eval_id = "pred_err",
                            eval_fun = "summarize_pred_err", 
                            show = c("point", "errorbar"),
                            y_str = NULL)  # or equivalently, "mean_pred_err"
all.equal(eval_tib, eval_tib2)
#> [1] TRUE

# if columns that are needed for plotting are missing in `eval_tib`, then
# `get_eval_tibble` will call the `eval_fun` and compute the necessary results
eval_tib <- get_eval_tibble(fit_results = fit_results, 
                            eval_tib = eval_tib %>% dplyr::select(-mean_pred_err),
                            eval_id = "pred_err",
                            eval_fun = "summarize_pred_err", 
                            show = c("point", "errorbar"),
                            y_str = NULL,  # or equivalently, "mean_pred_err"
                            # arguments to pass to `eval_fun`
                            truth_col = "y",
                            estimate_col = "predictions")
all.equal(eval_tib, eval_tib2)
#> [1] TRUE