Plot the rejection probability of a hypothesis test.

Plot the probability of rejecting the null hypothesis across various levels of significance.

Usage

plot_reject_prob(
  fit_results,
  eval_results = NULL,
  evaluator_name = NULL,
  vary_params = NULL,
  feature_col = NULL,
  show_features = NULL,
  show_identity_line = FALSE,
  show = c("line"),
  ...
)

Arguments

fit_results

A tibble, as returned by the fit method.

eval_results

A list of result tibbles, as returned by the evaluate method.

evaluator_name

Name of Evaluator containing results to plot. To compute the evaluation summary results from scratch or if the evaluation summary results have not yet been evaluated, set to NULL.

vary_params

A vector of parameter names that are varied across in the Experiment.

feature_col

A character string identifying the column in fit_results with the feature names or IDs.

show_features

Vector of feature names corresponding to features to display in the plot. If NULL (default), shows all features in the data.

show_identity_line

Logical indicating whether or not to plot the y = x line.

show

Character vector with elements being one of "boxplot", "point", "line", "bar", "errorbar", "ribbon" indicating what plot layer(s) to construct.

...

Additional arguments to pass to plot_eval_summary(). This includes arguments for plotting and for passing into eval_reject_prob().

Value

If interactive = TRUE, returns a plotly object if plot_by is NULL and a list of plotly objects if plot_by is not NULL. If interactive = FALSE, returns a ggplot object if plot_by is NULL and a list of ggplot objects if plot_by is not NULL.

See also

Other inference_funs: eval_reject_prob(), eval_testing_curve_funs, eval_testing_err_funs, plot_testing_curve(), plot_testing_err()

Examples

# generate example fit_results data
fit_results <- tibble::tibble(
  .rep = rep(1:2, times = 2),
  .dgp_name = c("DGP1", "DGP1", "DGP2", "DGP2"),
  .method_name = c("Method"),
  feature_info = lapply(
    1:4,
    FUN = function(i) {
      tibble::tibble(
        # feature names
        feature = c("featureA", "featureB", "featureC"),  
        # estimated p-values
        pval = 10^(sample(-3:0, 3, replace = TRUE))
      )
    }
  )
)

# generate example eval_results data
eval_results <- list(
  `Reject Prob.` = eval_reject_prob(
    fit_results, 
    nested_data = "feature_info",
    feature_col = "feature", 
    pval_col = "pval"
  )
)

# create bar plot using pre-computed evaluation results
plt <- plot_reject_prob(fit_results = fit_results,
                        eval_results = eval_results,
                        evaluator_name = "Reject Prob.",
                        feature_col = "feature")
# or alternatively, create the same plot without pre-computing evaluation results
plt <- plot_reject_prob(fit_results, 
                        nested_data = "feature_info",
                        feature_col = "feature",
                        pval_col = "pval")

# can customize plot (see plot_eval_summary() for possible arguments)
plt <- plot_reject_prob(fit_results = fit_results,
                        eval_results = eval_results,
                        evaluator_name = "Reject Prob.",
                        facet_formula = NULL,
                        plot_by = "feature")