From d4e4e61193f1b4c76bffb664e08138530025a198 Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Fri, 16 Jun 2023 13:06:54 -0400 Subject: [PATCH 1/8] docs: add experiment-runner contribution guide --- docs/contribute/experiment-runner-module.md | 32 +++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 docs/contribute/experiment-runner-module.md diff --git a/docs/contribute/experiment-runner-module.md b/docs/contribute/experiment-runner-module.md new file mode 100644 index 000000000..378efb5ad --- /dev/null +++ b/docs/contribute/experiment-runner-module.md @@ -0,0 +1,32 @@ +# Contribute an experimentalist + +AutoRA experiment runners are designed with to generate observations. They encompass a range of tools for conducting both synthetic and real experiments. By combining the observations with the corresponding experimental conditions, they may provide the inputs for an AutoRA theorist that then formulates a model that elucidates the relationship between the conditions and the observed outcomes. +![Experimentalist Runner Module](../img/experiment_runner.png) + +Experiment runners can be implemented as *synthetic runners*: +To contribute a *synthetic experiment runner* follow the [core](core.md) contribution guide. + +Complete experiment runners are functions that return observations. Experiment runner contributions also encompass tools that help with the automation of conducting experiments. For example, tools that are already implemented are a automated [recruitment manager](https://autoresearch.github.io/autora/user-guide/experiment-runners/recruitment-managers/prolific/) for recruiting participants on [Prolific](prolific.co) and a tool for automating [execution of online experiments](https://autoresearch.github.io/autora/user-guide/experiment-runners/experimentation-managers/firebase/) with [Firebase](https://firebase.google.com/). + +## Repository setup + +For non-synthetic experiment runners, we recommend using the [cookiecutter template](https://github.com/AutoResearch/autora-template-cookiecutter) to set up +a repository for your experiment runner. Alternatively, you can use the +[unguided template](https://github.com/AutoResearch/autora-template). If you choose the cookiecutter template, you can set up your repository using + +```shell +cookiecutter https://github.com/AutoResearch/autora-template-cookiecutter +``` + +Make sure to select the `experiment runner` option when prompted. If you want to design a tool to recruit participants, you may choose the option `recruitment manager`. If you want to design a tool to conduct experiments you may choose the option `experimentation manager`. You can also design a custom tool and name it yourself. You can skip all other prompts pertaining to other modules +(e.g., experimentalists) by pressing enter. + +## Implementation + +If you want to implement a complete experiment runner this should be a function that returns observations. But the experiment runner category also encompasses tools that help with automation of conducting experiments. For a more detailed overview, you can look at the [list of tools](https://autoresearch.github.io/autora/experiment-runner/) that are already implemented. + + +## Next steps: testing, documentation, publishing + +For more information on how to test, document, and publish your experimentalist, please refer to the +[general guideline for module contributions](module.md) . From 61d71fabfc6e31a6d0be649d4aea3dd9031dc2c9 Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Fri, 16 Jun 2023 13:12:53 -0400 Subject: [PATCH 2/8] chore: add experiment-runner contribution to nave --- mkdocs.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/mkdocs.yml b/mkdocs.yml index 44c5c9999..58478c4c7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -130,6 +130,7 @@ nav: - Overview: 'contribute/module.md' - Theorist Contribution: 'contribute/theorist-module.md' - Experimentalist Contribution: 'contribute/experimentalist-module.md' + - Experiment Runner Contribution: 'contribute/experiment-runner-module.md' - Core Contributions: 'contribute/core.md' - Environment Setup: 'contribute/setup.md' - Pre-Commit Hooks: 'contribute/pre-commit-hooks.md' From 912a37127947f9755f0ba59c5fca12fa6f423630 Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Fri, 16 Jun 2023 14:59:38 -0400 Subject: [PATCH 3/8] Update docs/contribute/experiment-runner-module.md Co-authored-by: benwandrew --- docs/contribute/experiment-runner-module.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contribute/experiment-runner-module.md b/docs/contribute/experiment-runner-module.md index 378efb5ad..c0904533f 100644 --- a/docs/contribute/experiment-runner-module.md +++ b/docs/contribute/experiment-runner-module.md @@ -1,6 +1,6 @@ # Contribute an experimentalist -AutoRA experiment runners are designed with to generate observations. They encompass a range of tools for conducting both synthetic and real experiments. By combining the observations with the corresponding experimental conditions, they may provide the inputs for an AutoRA theorist that then formulates a model that elucidates the relationship between the conditions and the observed outcomes. +AutoRA experiment runners are designed to generate observations. They encompass a range of tools for conducting both synthetic and real experiments. By combining observations with corresponding experimental conditions, they provide inputs necessary for an AutoRA theorist to perform model discovery. ![Experimentalist Runner Module](../img/experiment_runner.png) Experiment runners can be implemented as *synthetic runners*: From b744463f509d57454c314b3ce758f70ab76c5615 Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Fri, 16 Jun 2023 15:00:16 -0400 Subject: [PATCH 4/8] Update docs/contribute/experiment-runner-module.md Co-authored-by: benwandrew --- docs/contribute/experiment-runner-module.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contribute/experiment-runner-module.md b/docs/contribute/experiment-runner-module.md index c0904533f..9922c2330 100644 --- a/docs/contribute/experiment-runner-module.md +++ b/docs/contribute/experiment-runner-module.md @@ -6,7 +6,7 @@ AutoRA experiment runners are designed to generate observations. They encompass Experiment runners can be implemented as *synthetic runners*: To contribute a *synthetic experiment runner* follow the [core](core.md) contribution guide. -Complete experiment runners are functions that return observations. Experiment runner contributions also encompass tools that help with the automation of conducting experiments. For example, tools that are already implemented are a automated [recruitment manager](https://autoresearch.github.io/autora/user-guide/experiment-runners/recruitment-managers/prolific/) for recruiting participants on [Prolific](prolific.co) and a tool for automating [execution of online experiments](https://autoresearch.github.io/autora/user-guide/experiment-runners/experimentation-managers/firebase/) with [Firebase](https://firebase.google.com/). +Contributions may be complete experiment runners, which are functions that return observations, or tools that help automate experiments. Examples of such tools that are already implemented include a [recruitment manager](https://autoresearch.github.io/autora/user-guide/experiment-runners/recruitment-managers/prolific/) for recruiting participants on [Prolific](prolific.co) and an [experimentation manager](https://autoresearch.github.io/autora/user-guide/experiment-runners/experimentation-managers/firebase/) for executing online experiments with [Firebase](https://firebase.google.com/). ## Repository setup From fd91bdc5b51c0686d246961ff05dc324b6565dac Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Fri, 16 Jun 2023 15:00:55 -0400 Subject: [PATCH 5/8] Update docs/contribute/experiment-runner-module.md Co-authored-by: benwandrew --- docs/contribute/experiment-runner-module.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contribute/experiment-runner-module.md b/docs/contribute/experiment-runner-module.md index 9922c2330..02492fecd 100644 --- a/docs/contribute/experiment-runner-module.md +++ b/docs/contribute/experiment-runner-module.md @@ -18,7 +18,7 @@ a repository for your experiment runner. Alternatively, you can use the cookiecutter https://github.com/AutoResearch/autora-template-cookiecutter ``` -Make sure to select the `experiment runner` option when prompted. If you want to design a tool to recruit participants, you may choose the option `recruitment manager`. If you want to design a tool to conduct experiments you may choose the option `experimentation manager`. You can also design a custom tool and name it yourself. You can skip all other prompts pertaining to other modules +Make sure to select the `experiment runner` option when prompted. If you want to design a tool to recruit participants, choose the `recruitment manager` option. If you want to design a tool to conduct experiments, choose the `experimentation manager` option. You can also design a custom tool and name it yourself. You can skip all prompts pertaining to other modules (e.g., experimentalists) by pressing enter. ## Implementation From d0aa23644f38f2f31f341a90f3ce4fffe6e53217 Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Fri, 16 Jun 2023 15:01:15 -0400 Subject: [PATCH 6/8] Update docs/contribute/experiment-runner-module.md Co-authored-by: benwandrew --- docs/contribute/experiment-runner-module.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contribute/experiment-runner-module.md b/docs/contribute/experiment-runner-module.md index 02492fecd..b33082aa2 100644 --- a/docs/contribute/experiment-runner-module.md +++ b/docs/contribute/experiment-runner-module.md @@ -23,7 +23,7 @@ Make sure to select the `experiment runner` option when prompted. If you want to ## Implementation -If you want to implement a complete experiment runner this should be a function that returns observations. But the experiment runner category also encompasses tools that help with automation of conducting experiments. For a more detailed overview, you can look at the [list of tools](https://autoresearch.github.io/autora/experiment-runner/) that are already implemented. +To implement a complete experiment runner, be sure to define a function that returns observations. To get an idea for tools that help automate experiments, see the [list of tools](https://autoresearch.github.io/autora/experiment-runner/) that are already implemented. ## Next steps: testing, documentation, publishing From 8191e8a82eb0b2503df4451175f0221be550e212 Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Fri, 16 Jun 2023 15:01:23 -0400 Subject: [PATCH 7/8] Update docs/contribute/experiment-runner-module.md Co-authored-by: benwandrew --- docs/contribute/experiment-runner-module.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contribute/experiment-runner-module.md b/docs/contribute/experiment-runner-module.md index b33082aa2..0777b1c61 100644 --- a/docs/contribute/experiment-runner-module.md +++ b/docs/contribute/experiment-runner-module.md @@ -28,5 +28,5 @@ To implement a complete experiment runner, be sure to define a function that ret ## Next steps: testing, documentation, publishing -For more information on how to test, document, and publish your experimentalist, please refer to the +For more information on how to test, document, and publish your experiment runner, please refer to the [general guideline for module contributions](module.md) . From 03b467515efa5fe3f0882fb5510840e65e01e2f0 Mon Sep 17 00:00:00 2001 From: Younes Strittmatter Date: Fri, 16 Jun 2023 15:01:30 -0400 Subject: [PATCH 8/8] Update mkdocs.yml Co-authored-by: benwandrew --- mkdocs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdocs.yml b/mkdocs.yml index 58478c4c7..d853ead18 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -130,7 +130,7 @@ nav: - Overview: 'contribute/module.md' - Theorist Contribution: 'contribute/theorist-module.md' - Experimentalist Contribution: 'contribute/experimentalist-module.md' - - Experiment Runner Contribution: 'contribute/experiment-runner-module.md' + - Experiment Runner: 'contribute/experiment-runner-module.md' - Core Contributions: 'contribute/core.md' - Environment Setup: 'contribute/setup.md' - Pre-Commit Hooks: 'contribute/pre-commit-hooks.md'