feat: set up initial experiment server and cycle on firebase #342

younesStrittmatter · 2023-04-06T15:05:14Z

Description

Add experiment runner to the package

feat: set up initial experiment server and cycle on firebase #236

Type of change

feat: A new feature

Features (Optional)

add features to communicate with a firestore database to send conditions and retrieve dependent variables, added feature to increase the number of participants for a experiment on prolific to collect data

Questions (Optional)

how do we want to store experiment/prolific specific information, that might be sensible (tokens to talk to prolific and credentials to talk to firebase). The solution here just strores it in json files (maybe we can make one single json file for database name, name on prolific, tokens, credentials ... or maybe there are other/better solutions).
This is a first simple solution were the "get_data_function" would just wait until there is data in the database and then start the theorist, but we might want to add an additional "flag database" to make this more elaborate and to catch some errors.
There is a new dependency, "firebase_admin" that is needed to communicate to firebase, how do we handle this?
I am excited to run a first closed loop with this. The experiment is set up on Firebase already, all we would have to do is integrate this in the cycle and set up the experiment on prolific. Then we could already start collecting the first experiment.

…d functionality to increase number of participants in prolific

…ull in cycle

kphan20 · 2023-04-06T21:13:19Z

how do we want to store experiment/prolific specific information, that might be sensible (tokens to talk to prolific and credentials to talk to firebase). The solution here just strores it in json files (maybe we can make one single json file for database name, name on prolific, tokens, credentials ... or maybe there are other/better solutions).

I'm not sure if we already do this in the project, but we could save credentials and such with dotenv. It would allow us to define the credentials as environment variables.

hollandjg · 2023-04-07T14:00:26Z

There is a new dependency, "firebase_admin" that is needed to communicate to firebase, how do we handle this?

You can add it to the pyproject.toml and poetry.lock files by calling poetry add firebase_admin in the root of the project.

hollandjg · 2023-04-07T14:02:14Z

To use this together with the Controller, you'll need a single function which creates the experiment and returns the results as an array. I'm not sure I can see that yet (but let me know if I've not seen it).

hollandjg · 2023-04-07T14:08:11Z

How do we want to store experiment/prolific specific information, that might be sensible (tokens to talk to prolific and credentials to talk to firebase). The solution here just strores it in json files (maybe we can make one single json file for database name, name on prolific, tokens, credentials ... or maybe there are other/better solutions).

For now, I'd just add them as parameters to the functions, which you can pass using a dictionary using **params.
I think the thing which calls the functions should deal with loading the parameters – the functions themselves should be completely agnostic about that.
The calling thing might be the Controller object from #305 (which handles passing parameters to different parts of the cycle), or a command line tool which initiates the runner, or a script which the human-in-the-loop writes.
That calling thing can load the parameters from a JSON/YAML/TOML file, or have them as part of the script, or from a dotenv as Kevin has suggested.

…fic solution

musslick

This looks great! Only a couple of small comments.
One major item is naming: We should use the names for IVs and DVs as determined in the maintenance meeting.

musslick · 2023-04-10T10:08:03Z

autora/runner/data_managment/firebase.py

+    Args:
+        collection_name: the name of the study as given in firebase
+        condition: the condition
+        firebase_credentials: dict with the credentials for firebase


Currently this is typed as string but listed as dict argument in the comment?

musslick · 2023-04-10T10:09:47Z

autora/runner/data_managment/firebase.py

+    doc_ref.set({"condition": condition_json})
+
+
+def get_dependent_variable(collection_name: str, firebase_credentials: dict) -> Any:


Just flagging for discussion: Depending on how we decide about the terminology (independent variables vs. conditions; observations vs. dependent variables), we should rename it. Currently, it's a mix of both (conditions, dependent_variabes).

musslick · 2023-04-10T10:11:28Z

autora/runner/firebase_runner.py

+        condition: the condition
+        **kwargs:
+    """
+    send_condition(kwargs["collection_name"], condition, kwargs["firebase_credentials"])


Is it sending a single condition or an entire set of conditions? If it is the former then we should make this more clear by labeling it accordingly (e.g., send_experiment or send_conditions.

musslick · 2023-04-11T09:22:09Z

autora/runner/firebase_runner.py

+        **kwargs:
+
+    Returns:
+        the dependent variavle


Suggested change

the dependent variavle

the dependent variable

musslick · 2023-04-11T09:22:26Z

autora/runner/firebase_runner.py

+        kwargs["collection_name"], kwargs["firebase_credentials"]
+    )
+    while data is None:
+        sleep(10)


Should the sleep time be a parameter?

…irebase runner

hollandjg · 2023-06-02T15:11:41Z

this is outdated, replaced by the firebase package

younesStrittmatter added 3 commits April 5, 2023 17:09

added features to send conditions to firebase and retrieve data, adde…

e40c9fd

…d functionality to increase number of participants in prolific

added __init__ file

c3fdbcc

add reset database to send condition so download data can check for n…

b3d9bda

…ull in cycle

younesStrittmatter requested review from hollandjg and kphan20 April 6, 2023 15:05

younesStrittmatter requested a review from musslick as a code owner April 6, 2023 15:05

younesStrittmatter linked an issue Apr 6, 2023 that may be closed by this pull request

feat: set up initial experiment server and cycle on firebase #236

Closed

hollandjg changed the title ~~236 feat set up initial experiment server and cycle on firebase~~ feat: set up initial experiment server and cycle on firebase Apr 7, 2023

adding runner generic function and runner for specific firebase proli…

781d458

…fic solution

musslick requested changes Apr 11, 2023

View reviewed changes

younesStrittmatter added 3 commits April 19, 2023 12:47

added checks for pausing and unpausing the experiment. added a pure f…

bbfa3e1

…irebase runner

remove files that are not needed any more

89b3e30

added controler to branch

302e913

hollandjg closed this Jun 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: set up initial experiment server and cycle on firebase #342

feat: set up initial experiment server and cycle on firebase #342

younesStrittmatter commented Apr 6, 2023

kphan20 commented Apr 6, 2023

hollandjg commented Apr 7, 2023

hollandjg commented Apr 7, 2023

hollandjg commented Apr 7, 2023

musslick left a comment

musslick Apr 10, 2023

musslick Apr 10, 2023

musslick Apr 10, 2023

musslick Apr 11, 2023

musslick Apr 11, 2023

hollandjg commented Jun 2, 2023

		doc_ref.set({"condition": condition_json})


		def get_dependent_variable(collection_name: str, firebase_credentials: dict) -> Any:

feat: set up initial experiment server and cycle on firebase #342

feat: set up initial experiment server and cycle on firebase #342

Conversation

younesStrittmatter commented Apr 6, 2023

Description

Type of change

Features (Optional)

Questions (Optional)

kphan20 commented Apr 6, 2023

hollandjg commented Apr 7, 2023

hollandjg commented Apr 7, 2023

hollandjg commented Apr 7, 2023

musslick left a comment

Choose a reason for hiding this comment

musslick Apr 10, 2023

Choose a reason for hiding this comment

musslick Apr 10, 2023

Choose a reason for hiding this comment

musslick Apr 10, 2023

Choose a reason for hiding this comment

musslick Apr 11, 2023

Choose a reason for hiding this comment

musslick Apr 11, 2023

Choose a reason for hiding this comment

hollandjg commented Jun 2, 2023