Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: sampler to experimentalist at the appropriate places #17

Merged
merged 1 commit into from
Sep 2, 2023
Merged

docs: sampler to experimentalist at the appropriate places #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AutoRA Model Disagreement Sampler
# AutoRA Model Disagreement Experimentlist

The model disagreement sampler identifies experimental conditions $\vec{x}' \in X'$ with respect to
The model disagreement experimentlist identifies experimental conditions $\vec{x}' \in X'$ with respect to
a pairwise distance metric between theorist models, $P_{M_{i}}(\hat{y}, \vec{x}')$:

$$
Expand All @@ -26,6 +26,6 @@ darts_theorist = DARTSRegressor()
bms_theorist.fit(X,y)
darts_theorist.fit(X,y)

#Sampler
#experimentlist
X_new = model_disagreement_sample(X, [bms_theorist, darts_theorist], n)
```
111 changes: 20 additions & 91 deletions docs/Basic Usage.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Import Modules\n",
"\n",
Expand All @@ -17,11 +13,7 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"\n",
Expand All @@ -35,11 +27,7 @@
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
Expand All @@ -56,9 +44,6 @@
"metadata": {
"vscode": {
"languageId": "plaintext"
},
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
Expand All @@ -70,11 +55,7 @@
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"#Define meta-parameters\n",
Expand All @@ -90,11 +71,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Plot The Data\n",
"\n",
Expand All @@ -104,11 +81,7 @@
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand All @@ -129,11 +102,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Define And Fit Theorists\n",
"\n",
Expand All @@ -145,11 +114,7 @@
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"name": "stderr",
Expand Down Expand Up @@ -179,11 +144,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Plot Theorists On Data\n",
"\n",
Expand All @@ -193,11 +154,7 @@
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand All @@ -222,25 +179,17 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Plot Model Disagreements\n",
"\n",
"We can also plot the disagreement between each pair of theorists. The grey dashed lines represent the degree of disagreement for each datapoint. Computationally the Model Disagreement Sampler takes the square of these numbers to determine where disagreement is largest. "
"We can also plot the disagreement between each pair of theorists. The grey dashed lines represent the degree of disagreement for each datapoint. Computationally the Model Disagreement Experimentalist takes the square of these numbers to determine where disagreement is largest."
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand Down Expand Up @@ -272,25 +221,17 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Run And Report Model Disagreement Samples\n",
"\n",
"Now we will get a proposal from the sampler as to which datapoints to investigate next. We will retrieve 5 new datapoints in this example."
"Now we will get a proposal from the experimentalist as to which datapoints to investigate next. We will retrieve 5 new datapoints in this example."
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand All @@ -314,11 +255,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Plot New Datapoints With Old\n",
"\n",
Expand All @@ -328,11 +265,7 @@
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand All @@ -353,11 +286,7 @@
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand Down Expand Up @@ -405,4 +334,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
6 changes: 3 additions & 3 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Model Disagreement Sampler
# Model Disagreement Experimentalist

The model disagreement sampler identifies experimental conditions $\vec{x}' \in X'$ with respect to
The model disagreement experimentalist identifies experimental conditions $\vec{x}' \in X'$ with respect to
a pairwise distance metric between theorist models, $P_{M_{i}}(\hat{y}, \vec{x}')$:

$$
Expand All @@ -26,6 +26,6 @@ darts_theorist = DARTSRegressor()
bms_theorist.fit(X,y)
darts_theorist.fit(X,y)

#Sampler
#Experimentalist
X_new = model_disagreement_sample(X, [bms_theorist, darts_theorist], n)
```
2 changes: 1 addition & 1 deletion docs/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ You will need:

- `python` 3.8 or greater: [https://www.python.org/downloads/](https://www.python.org/downloads/)

*Model Disagreement Sampler* is a part of the `autora` package:
*Model Disagreement Experimentalist* is a part of the `autora` package:

```shell
pip install -U autora["experimentalist-model-disagreement"]
Expand Down
11 changes: 6 additions & 5 deletions src/autora/experimentalist/model_disagreement/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ def score_sample(conditions: Union[pd.DataFrame, np.ndarray],
models: List,
num_samples: int = 1):
"""
A sampler that returns selected samples for independent variables
A experimentalist that returns selected samples for independent variables
for which the models disagree the most in terms of their predictions.

Args:
Expand Down Expand Up @@ -82,11 +82,12 @@ def score_sample(conditions: Union[pd.DataFrame, np.ndarray],

return conditions.head(num_samples)


def sample(conditions: Union[pd.DataFrame, np.ndarray],
models: List,
num_samples: int = 1):
"""
A sampler that returns selected samples for independent variables
A experimentalist that returns selected samples for independent variables
for which the models disagree the most in terms of their predictions.

Args:
Expand All @@ -102,8 +103,8 @@ def sample(conditions: Union[pd.DataFrame, np.ndarray],

return selected_conditions


model_disagreement_sample = sample
model_disagreement_sample.__doc__ = """Alias for sample"""
model_disagreement_score_sample = score_sample
model_disagreement_score_sample.__doc__ = """Alias for sample"""
model_disagreement_sampler = deprecated_alias(model_disagreement_sample, "model_disagreement_sampler")
model_disagreement_sampler = deprecated_alias(
model_disagreement_sample, "model_disagreement_sampler")