Skip to contents

Varying across parameters in an Experiment.

Source: R/experiment-helpers.R
vary_across.Rd

Helper functions for adding, updating, removing, or getting a vary_across component in an Experiment. When a vary_across component is added and the Experiment is run, the Experiment is systematically varied across values of the specified parameter in the DGP or Method while all other parameters are held constant.

Usage

add_vary_across(.experiment, .dgp, .method, ...)

update_vary_across(.experiment, .dgp, .method, ...)

remove_vary_across(experiment, dgp, method, param_names = NULL)

get_vary_across(experiment)

Arguments

.experiment, experiment

An Experiment object.

.dgp, dgp

Name of DGP to vary in the Experiment. Can also be a DGP object that matches one in the Experiment or even a vector/list of DGP names/objects, assuming they can all take in the specified param_names.

.method, method

Name of Method to vary in the Experiment. Can also be a Method object that matches one in the Experiment or even a vector/list of Method names/objects, assuming they all support the target arguments provided via ....

...

Any number of named arguments where names match an argument in the user-specified DGP or Method function and values are vectors (for scalar parameters) or lists (for arbitrary parameters).

param_names

A character vector of parameter names to remove. If not provided, the entire set of vary_across parameters will be removed for the specified DGP/Method.

Value

In the case of get_vary_across, a nested list with entries "dgp" and "method" that contains the parameters to vary across for each DGP and Method in the Experiment. Otherwise, the original Experiment object passed to *_vary_across().

Details

One of the .dgp or .method arguments (but not both) must be provided when using add_vary_across() and update_vary_across. For remove_vary_across(), if both the dgp and method arguments are not provided, then all vary_across parameters from the experiment are removed.

Examples

# generate data from normal distribution with n samples
normal_dgp <- create_dgp(
  .dgp_fun = function(n) rnorm(n), .name = "Normal DGP", n = 100
)
# generate data from binomial distribution with n samples
bernoulli_dgp <- create_dgp(
  .dgp_fun = function(n) rbinom(n, 1, 0.5), .name = "Bernoulli DGP", n = 100
)

# compute weighted mean of data
mean_method <- create_method(
  .method_fun = function(x, ...) list(mean = mean(x, ...)),
  .name = "Mean"
)

# initialize experiment with toy DGPs and Methods
experiment <- create_experiment(name = "Experiment Name") %>%
  add_dgp(normal_dgp) %>%
  add_dgp(bernoulli_dgp) %>%
  add_method(mean_method)

# vary across n for normal DGP
experiment <- experiment %>%
  add_vary_across(.dgp = "Normal DGP", n = c(100, 200, 300))
get_vary_across(experiment)
#> $dgp
#> $dgp$`Normal DGP`
#> $dgp$`Normal DGP`$n
#> [1] 100 200 300
#> 
#> 
#> 
#> $method
#> list()
#> 
print(experiment)
#> Experiment Name: Experiment Name 
#>    Saved results at: results/Experiment Name 
#>    DGPs: Normal DGP, Bernoulli DGP 
#>    Methods: Mean 
#>    Evaluators:  
#>    Visualizers:  
#>    Vary Across: 
#>       DGP: Normal DGP 
#>          n:  num [1:3] 100 200 300

# remove vary across for normal DGP
experiment <- experiment %>%
  remove_vary_across(dgp = "Normal DGP")
get_vary_across(experiment)
#> $dgp
#> list()
#> 
#> $method
#> list()
#> 
print(experiment)
#> Experiment Name: Experiment Name 
#>    Saved results at: results/Experiment Name 
#>    DGPs: Normal DGP, Bernoulli DGP 
#>    Methods: Mean 
#>    Evaluators:  
#>    Visualizers:  
#>    Vary Across: None

# can add vary across for multiple DGPs at once
experiment <- experiment %>%
  add_vary_across(.dgp = c("Normal DGP", "Bernoulli DGP"), n = c(100, 200, 300))
get_vary_across(experiment)
#> $dgp
#> $dgp$`Normal DGP`
#> $dgp$`Normal DGP`$n
#> [1] 100 200 300
#> 
#> 
#> $dgp$`Bernoulli DGP`
#> $dgp$`Bernoulli DGP`$n
#> [1] 100 200 300
#> 
#> 
#> 
#> $method
#> list()
#> 
print(experiment)
#> Experiment Name: Experiment Name 
#>    Saved results at: results/Experiment Name 
#>    DGPs: Normal DGP, Bernoulli DGP 
#>    Methods: Mean 
#>    Evaluators:  
#>    Visualizers:  
#>    Vary Across: 
#>       DGP: Normal DGP 
#>          n:  num [1:3] 100 200 300
#>       DGP: Bernoulli DGP 
#>          n:  num [1:3] 100 200 300

# can update vary across for DGPs
experiment <- experiment %>%
  update_vary_across(.dgp = "Normal DGP", n = c(100, 300)) %>%
  update_vary_across(.dgp = "Bernoulli DGP", n = c(100, 200))
get_vary_across(experiment)
#> $dgp
#> $dgp$`Normal DGP`
#> $dgp$`Normal DGP`$n
#> [1] 100 300
#> 
#> 
#> $dgp$`Bernoulli DGP`
#> $dgp$`Bernoulli DGP`$n
#> [1] 100 200
#> 
#> 
#> 
#> $method
#> list()
#> 
print(experiment)
#> Experiment Name: Experiment Name 
#>    Saved results at: results/Experiment Name 
#>    DGPs: Normal DGP, Bernoulli DGP 
#>    Methods: Mean 
#>    Evaluators:  
#>    Visualizers:  
#>    Vary Across: 
#>       DGP: Normal DGP 
#>          n:  num [1:2] 100 300
#>       DGP: Bernoulli DGP 
#>          n:  num [1:2] 100 200

# can also add/update/remove vary across for methods
experiment <- experiment %>%
  add_vary_across(.method = "Mean", trim = list(0, 0.1, 0.2))
print(experiment)
#> Experiment Name: Experiment Name 
#>    Saved results at: results/Experiment Name 
#>    DGPs: Normal DGP, Bernoulli DGP 
#>    Methods: Mean 
#>    Evaluators:  
#>    Visualizers:  
#>    Vary Across: 
#>       DGP: Normal DGP 
#>          n:  num [1:2] 100 300
#>       DGP: Bernoulli DGP 
#>          n:  num [1:2] 100 200
#>       Method: Mean 
#>          trim: List of 3
#>            $ : num 0
#>            $ : num 0.1
#>            $ : num 0.2
experiment <- experiment %>%
  update_vary_across(
    .method = "Mean", trim = list(0, 0.1, 0.2, 0.3)
  )
print(experiment)
#> Experiment Name: Experiment Name 
#>    Saved results at: results/Experiment Name 
#>    DGPs: Normal DGP, Bernoulli DGP 
#>    Methods: Mean 
#>    Evaluators:  
#>    Visualizers:  
#>    Vary Across: 
#>       DGP: Normal DGP 
#>          n:  num [1:2] 100 300
#>       DGP: Bernoulli DGP 
#>          n:  num [1:2] 100 200
#>       Method: Mean 
#>          trim: List of 4
#>            $ : num 0
#>            $ : num 0.1
#>            $ : num 0.2
#>            $ : num 0.3
experiment <- experiment %>%
  remove_vary_across(method = "Mean")
print(experiment)
#> Experiment Name: Experiment Name 
#>    Saved results at: results/Experiment Name 
#>    DGPs: Normal DGP, Bernoulli DGP 
#>    Methods: Mean 
#>    Evaluators:  
#>    Visualizers:  
#>    Vary Across: 
#>       DGP: Normal DGP 
#>          n:  num [1:2] 100 300
#>       DGP: Bernoulli DGP 
#>          n:  num [1:2] 100 200

# can remove all vary across parameters
experiment <- experiment %>%
  remove_vary_across()
get_vary_across(experiment)
#> $dgp
#> list()
#> 
#> $method
#> list()
#> 
print(experiment)
#> Experiment Name: Experiment Name 
#>    Saved results at: results/Experiment Name 
#>    DGPs: Normal DGP, Bernoulli DGP 
#>    Methods: Mean 
#>    Evaluators:  
#>    Visualizers:  
#>    Vary Across: None