Doc/closed loop example bandit task

docs/examples/closed-loop-bandit-task/customization.md

@@ -0,0 +1,8 @@
+The code you have created can serve as a template for future projects. The most important files are:
+
+(1) Autora Workflow: research_hub/autora_workflow.py Here, you can customize the AutoRA workflow. Which includes:
+
+Using other experimentalists to change the selection of experiment-conditions.
+Adjusting runners: In this example, we use jsPsych and javascript for the experiments, but depending on the experiment, you can interface with other devices or use other means to collect data.
+Using other theorists. The theorist that is used in the example is custom build for RL tasks. Depending on your experiment, you will use other theorists.
+(2) Website: testng_zone/src/design/main.js Here, you can customzie the website that is shown to the participant. You can find good tutorials on how to build jsPsych experiments on their website Keep in mind, that you should build your website in a way that the conditions are used to customize the experiment. A good way to do so is by creating trial-sequences in autora_workflow.py and using them as timeline_variables.

docs/examples/closed-loop-bandit-task/firebase.md

@@ -0,0 +1,44 @@
+# Set Up The Project On The Firebase Website
+
+Next, we want to set up Firebase for our project. 
+
+![Setup](img/system_firebase.png)
+
+Firebase is a platform developed by Google for creating mobile and web applications. Here, we will leverge firebase as a platform for hosting our web-based behavioral experiment, and firestore for hosting associated experimental data.
+
+To serve a website via Firebase and use the Firestore Database, it is necessary to set up a Firebase project. Follow the steps below to get started:
+
+## Google Account
+You'll need a [Google account](https://www.google.com/account/about/) to use Firebase.
+
+## Firebase Project
+While logged in into your Google account, head over to the [Firebase website](https://firebase.google.com/). Then, create a new project:
+
+1. Click on `Get started`.
+2. Click on the plus sign with `Create a project`.
+3. Name your project (e.g., "closed-loop=study") and click on `Continue`.
+4. As we don't need Google Analytics, we can leave it disabled (you can leave it enabled if you want to use it in the future).
+5. Click `Create project`.
+
+## Adding a Web App to Your Project
+Now, we add a web app to the project, which will correspond to our web experiment. Navigate to the project and follow these steps:
+
+1. Click on ```<\>```.
+![webapp.png](img/webapp.png)
+2. Name the app (can be the same as your project) and check the box `Also set up Firebase Hosting`. Click on `Register app`.
+3. Select `Use npm`. We will use the configuration details later, but for now, click on `Next`.
+3. We will install firebase tools later, for now, click on `Next`.
+4. We will login and deploy our website later, for now, click on `Continue to console`.
+
+## Adding Firestore To Your Project
+For our closed-loop study, we will use a Firestore Database to communicate between the AutoRA workflow and the website conducting the experiment. We will upload experiment conditions to the database and also store experiment data in the database. To build a Firestore Database, follow these steps:
+
+1. In the left-hand menu of your project console, click on `Build` and select `Firestore Database`.
+![build_database.png](img/build_database.png)
+2. Click on `Create database`.
+3. Select a location for the server hosting the database. Click on `Next`. Note that your institution may have restrictions on the location of the server. Click `Next`
+4. Select `Start in production mode` selected and click `Create`.
+
+You have now configured Firebase for your project. Next, we will connect your local project to Firebase and deploy your web-based experiment.
+
+[next](testingzone.md)

docs/examples/closed-loop-bandit-task/index.md

@@ -0,0 +1,44 @@
+# Basic Closed-Loop Two-Armed Bandit Study 
+
+In this example, we will guide you through setting up a closed-loop behavioral study for a two-armed bandit experiment. By leveraging AutoRA, you’ll build a dynamic research workflow that iterates between model discovery, experimental design, and behavioral data collection, all within the context of a two-armed bandit experiment. The ultimate goal is to make AutoRA iteratively uncover an equation that characterizes human participants' learning equations in a two-armed bandit task.
+
+This example provides a hands-on approach to understanding closed-loop behavioral research in the context of the AutoRA framework. 
+
+## What You’ll Learn:
+- **Set up a closed-loop AutoRA workflow**: Learn how to create an automated discovery process, iterating between hypothesis generation and data collection.
+- **Automate experimental design with [SweetPea](https://sites.google.com/view/sweetpea-ai)**: Use SweetPea to generate experimental designs that adapt as the study progresses.
+- **Generate behavioral experiments with [JsPsych](https://www.jspsych.org/v7/)**: Creation of simple behavioral experiment with jsPsych.
+- **Host experiments using [Google Firebase](https://firebase.google.com/)**: Set up a server for hosting your behavioral experiments, making them accessible to participants.
+- **Store experimental data with [Google Firestore](https://firebase.google.com/)**: Efficiently manage and store participant data collected from your experiment.
+- **Collect data from real participants with [Prolific](https://www.prolific.com/)**: Recruit and manage participants through Prolific, ensuring high-quality behavioral data.
+
+## Prerequisites:
+- **Basic Python knowledge**: While most of the workflow is Python-based, only a basic level of understanding is needed to follow along.
+- **Minimal JavaScript knowledge**: Since the behavioral experiments are implemented in JavaScript (via jsPsych), a minimal understanding of JavaScript is required.
+- **A Google account**: You will need a Google account to use Google Firebase and Firestore.
+
+## Overview
+
+Our closed-loop system consists of a bunch of interacting components. Here is a high-level overview of the system:
+![System Overview](../img/system_overview.png)
+
+Our closed-loop system will have two projects talking to each other. The **Firebase project** will host and run the web experiment that participants interact with. Our **local AutoRA project** will host the code that runs the AutoRA workflow, which will generate new experiment conditions, collect data from the web experiment, and update the model based on the collected data. 
+
+### Firebase Project
+To run an online experiment, we need to host it as a **web app**. We will leverage **Google Firebase** to host our web app. Participants from **Prolific** can then interact with the web app to complete the experiment.  
+
+Our experiment is configured by experiment conditions, which are stored in a **Google Firestore** database. In addition, we will use this database to store collected behavioral data from the web experiment. 
+
+### Our Local AutoRA Project
+
+The local project will consist of two folders. The **testing_zone** folder will contain the web app that participants interact with. 
+
+The **researcher_hub** folder will contain the AutoRA workflow. 
+
+## Next Steps 
+
+Each step in the example will lead guide you to set up each component of the closed-loop system. 
+
+By the end of this example, you’ll be able to create a fully automated behavioral closed-loop study that adapts based on the collected participant data.
+
+[next](setup.md)

docs/examples/closed-loop-bandit-task/init_autora.md

@@ -0,0 +1,69 @@
+# Connect AutoRA WorkFlow to Firebase
+
+After setting up a mechanism to deploy your experiments online, you can now connect the AutoRA workflow to Firestore database. This will allow us to update experiment conditions and to download observations collected from the experiment.
+
+![Setup](../img/system_researcherhub.png)
+
+The workflow will manage the entire research process, from generating novel experiments to collecting data and modeling the results. 
+
+Note that the workflow will talk to the Firebase project by uploading and downloading data to the Firestore database. We will upload new experiment conditions to the database and download the results of the experiment from it. 
+
+The AutoRA workflow can be found in the `researcher_hub` folder, which contains a template for an AutoRA workflow for a two-armed bandit task using a model discovery process (AutoRA-theorist) described here: [rnn-sindy-rl](https://github.com/AutoResearch/autora-theorist-rnn-sindy-rl).
+
+
+1. Move into the `researcher_hub` directory, where the template for the workflow is stored.
+
+```shell
+cd researcher_hub
+```
+
+2. Then install the Python packages required for the workflow using `pip`:
+
+```shell
+pip install -r requirements.txt
+```
+
+3. There are three different experiment runners:
+   - `autora-synthetic-runner`: This runner is used to run the AutoRA workflow with simulated data.
+   - `autora-firebase-runner`: This runner is used to run the AutoRA workflow with Firebase. It will upload new experiment conditions to the Firestore database and download the results of the experiment from it. This can be used to test the online experiment.
+   - `autora-prolific-runner`: This runner is used to run the AutoRA workflow with Prolific. It will recruit participants via Prolific and upload the results of the experiment to Prolific.
+
+## Add Firebase Credentials
+
+The AutoRA workflow (specifically the `autora-firebase-runner` and `autora-prolific-runner`) will need access to your firebase project. Therefore, we need the corresponding credentials. 
+
+1. To obtain the credentials, go to the [Firebase console](https://console.firebase.google.com/).
+2. Navigate to the project.
+3. Click on the little gear on the left and then select ``Project settings``. 
+4. Click on ``Service accounts``.
+![service_account.png](../img/service_account.png)
+5. Having ``Node.js`` selected, generate a new private key. This should generate a json file that you can download.
+6. Open the file `autora_workflow.py` in the `research_hub`-folder and navigate to the part of the code that contains a placeholder for the credentials. It should look like this
+```python
+firebase_credentials = {
+    "type": "type",
+    "project_id": "project_id",
+    "private_key_id": "private_key_id",
+    "private_key": "private_key",
+    "client_email": "client_email",
+    "client_id": "client_id",
+    "auth_uri": "auth_uri",
+    "token_uri": "token_uri",
+    "auth_provider_x509_cert_url": "auth_provider_x509_cert_url",
+    "client_x509_cert_url": "client_x509_cert_url"
+}
+```
+7. Replace the placeholders with the credentials from the json file you downloaded.
+
+## Try out the Workflow
+
+First try the synthetic runner to see if everything works correctly. then you can set the experiment to run with firebase and run `autora_workflow.py`. Then head over to your website to test your first online experiment.
+
+This will be the experiment as participants will see it. You can test it and check the output of the loop.
+
+To check the database, go to the Firebase console and select your project. On the left menu, navigate to ``Firestore Database``. If everything worked, you should see database fields called ``autora_in`` and ``autora_out``. The former contains the experiment conditions which are used to configure the experiment. The latter will contain the results of the experiment.
+
+![firestore_data.png](../img/firestore_data.png)
+
+[next](customization.md)
+

docs/examples/closed-loop-bandit-task/setup.md

@@ -0,0 +1,128 @@
+# Set Up Project
+
+First, we need to set up our local AutoRA project.
+
+![Setup](../img/system_project.png)
+
+## Create Repository
+To ease the setup process for this example, we provide a template repository that contains all the necessary files and configurations. 
+
+Simply visit the following repository on GitHub: https://github.com/AutoResearch/autora-closed-loop-firebase-prolific-bandit-task
+
+Next, click on the green "Use this template" button to create a new repository in your account.
+![use_template.png](img/use_template.png)
+
+You may then enter the name of the repository (e.g., "closed-loop-study") and click on the "Create repository" button.
+
+## Clone Repository or Open it in GitHub Codespace
+
+Once you created your own repository from the template, you can clone it to your local machine using ``git clone``. However, we recommend using GitHub Codespaces for this example, as it provides a more streamlined development environment.
+
+To open the repository in GitHub Codespaces, click on the "Code" button and select "Create codespace on main".
+![img.png](img/open_codespace.png)
+
+## Set Up Environment
+
+Once you cloned your repository or opened it in codespaces, it is time to set up your environment. Here, we will use a Python virtual environment to manage dependencies.
+
+!!! success
+    We recommend setting up your development environment using a manager like `venv`, which creates isolated python 
+    environments. Other environment managers, like 
+    [virtualenv](https://virtualenv.pypa.io/en/latest/),
+    [pipenv](https://pipenv.pypa.io/en/latest/),
+    [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/), 
+    [hatch](https://hatch.pypa.io/latest/), 
+    [poetry](https://python-poetry.org), 
+    are available and will likely work, but will have different syntax to the syntax shown here. 
+
+    Our packages are set up using `virtualenv` with `pip`  
+
+In the `<project directory>`, run the following command to create a new virtual environment in the `.venv` directory
+
+```shell
+python3 -m "venv" ".venv" 
+```
+![venv.png](img/venv.png)
+
+!!! hint
+    If you have multiple Python versions installed on your system, it may be necessary to specify the Python version when creating a virtual environment. For example, run the following command to specify Python 3.8 for the virtual environment. 
+    ```shell
+    python3.8 -m "venv" ".venv" 
+    ```
+
+Activate it by running
+```shell
+source ".venv/bin/activate"
+```
+
+## Install Dependencies
+
+First, install the cookiecutter package using pip via
+
+```shell
+pip install cookiecutter
+```
+
+We will use this package to automatically configure our project folder structure. 
+
+Then we install some python dependencies:
+```shell
+pip install -r requirements.txt
+```
+
+We  will also use firebase-tools:
+```shell
+npm install -g firebase-tools
+```
+
+We will use this package to deploy our firebase functions for storing data.
+
+## Set Up Project Folder with Coockiecutter
+
+To establish an online closed-loop for AutoRA, there are two key components that need to be configured:
+
+1. AutoRA Workflow
+    - This workflow can be executed locally, on a server, or using `Cylc`. It must have the ability to communicate with a website, allowing for the writing of new conditions and reading of observation data.
+    - The AutoRA workflow can be customized by adding or removing AutoRA functions, such as AutoRA *experimentalists* or AutoRA *theorists*. It relies on an AutoRA Prolific Firebase *runner* to collect data from an online experiment hosted via Firebase and recruit participants via prolific.
+
+2. Website To Conduct Experiment:
+    - The website serves as a platform for conducting experiments and needs to be compatible with the AutoRA workflow.
+    - In this setup, we use `Firebase` to host on website.
+
+To simplify the setup process, we provide a `cookiecutter` template that generates a project folder containing the following two directories:
+
+1. **Researcher Hub**:
+    - This directory includes a basic example of an AutoRA workflow.
+
+2. **Testing Zone**:
+    - This directory provides a basic example of a website served with Firebase, ensuring compatibility with the AutoRA workflow.
+
+Once you installed the packages above, you can create the project by running the following command in the root directory of your project:
+
+```shell
+cookiecutter https://github.com/AutoResearch/autora-user-cookiecutter
+```
+
+if cookecutter is not recognized, you may need to run the following command:
+
+```shell
+python -m cookiecutterhttps://github.com/AutoResearch/autora-user-cookiecutter
+```
+
+You will be prompted to enter some information about your project. You can select single options by pressing SPACE and confirm your selection by pressing ENTER.
+
+1. You may first enter a project name, e.g., "closed-loop-study".
+2. Select ``yes`` to use advanced features.
+3. We are going to use a theorist from a different package so we don't need to select any theorist here.
+4. Choose at least ``autora[experimentalist-bandit-random]`` and ``autora[experimentalist-model-disagreement]``.
+5. Make sure to select the ``autora[experiment-runner-firebase-prolific]`` option
+6. Select ``yes`` to set up a firebase experiment. When asked to install further packages ([email protected]), select yes (y). This may take some time.
+7. Finally, select ``JsPsych - Bandit`` as project type.
+
+You should now have the following project structure:
+
+![project_structure.png](../img/project_structure.png)
+
+[next](firebase.md)
+
+