Setting Up Your Simulation StudySource:
We recommend initializing your simulation directory as you would any new analysis: using a project. Doing so improves reproducibility and replicability; it keeps your code, data, and outputs neatly separated from other analyses.
In the same vein, we highly encourage the use of version control software — git in particular. It helps you keep track of changes, undo snafus, and seamlessly collaborate with your peers. Your git repositories can be linked to GitHub, a hosting service that lets you manage your projects. If you’re unfamiliar with git or GitHub, or haven’t yet set git up on you machine, we suggest reviewing Chapters 4 through 8 of Happy Git and GitHub for the useR.
While you could create a version-controlled R project manually, we
suggest using the
create_sim() function. Briefly, this
function takes as argument a
path indicating the location
of the new directory for your simulation study and creates the necessary
folders and files. It also initializes a new project and a git
repository. This repository can optionally be linked to GitHub using,
If you don’t want to use git, set
init_git argument to
After creating your project with
find the following directories and files at the designated
R/: This folder will contain your simulation study’s R code.
simChefexperiment is assembled in this file.
dgp/: Data-generating process functions are stored here.
method/: Method functions are stored here.
eval/: Evaluator functions are stored here.
viz/: Visualizer functions are stored here.
results/: Your simulation experiment’s output are automatically saved here.
README.md: This markdown file briefly summarizes your simulation study’s goals and results. Some guidance on navigating your project’s directory structure might also be appreciated by your collaborators — and even yourself at some point in the future.
We also suggest incorporating unit tests in your simulation study. Unit tests allow you to test individual units of code to make sure that they are working as expected. The trustworthiness of your simulation study’s results is in turn improved.
simChef integrates with the
testthat R package,
the most commonly used unit testing framework for R software
development. The scaffolding required for testing your simulation
study’s code is generated automatically when setting
tests argument to
TRUE—the default. All of your tests can then be carried out
run_tests() from the root directory of your
Here’s a rundown of the directories and files that are created by
tests/: The folder containing all of the unit testing material.
testthat.R: This script calls
testthat::test_dir()on the following sub-directories to run the tests contained therein. All packages used by the functions in your
R/directory must be included manually in this file.
testthat/dgp-tests/: Tests for data-generating process functions.
testthat/method-tests/: Tests for method functions.
testthat/eval-tests/: Tests for evaluator functions.
testthat/viz-tests/: Tests for visualizer functions.
If you know that your simulation study will require plenty of
computational resources, then you might consider running it on a
high-performance computing environment with a workload managers, like SLURM.
Directories for organizing high-power computing-related files can be
generated by setting
hpc = TRUE when calling
scripts/: This directory folder contains scripts, like Bash files, for initiating simulation studies in high-power computing environments. For added flexibility, we recommend the R package
optparseto parse command-line options from your bash scripts within
logs/: The files output by these workload managers can be stored in this directory.
Again with reproducibility and replicability in mind, we encourage
the use of
for project-local dependency management.
renv can be
initialized when creating your project by setting
init_renv = TRUE in your call to
renv workflow can be found here.