Skip to content

Commit

Permalink
Merge branch 'main' into 15-chore-consistent-default-argument-for-num…
Browse files Browse the repository at this point in the history
…_samples-in-score_sample
  • Loading branch information
younesStrittmatter authored Sep 3, 2023
2 parents 129ec6e + 6ede48b commit f5e4167
Show file tree
Hide file tree
Showing 5 changed files with 32 additions and 104 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AutoRA Model Disagreement Sampler
# AutoRA Model Disagreement Experimentlist

The model disagreement sampler identifies experimental conditions $\vec{x}' \in X'$ with respect to
The model disagreement experimentlist identifies experimental conditions $\vec{x}' \in X'$ with respect to
a pairwise distance metric between theorist models, $P_{M_{i}}(\hat{y}, \vec{x}')$:

$$
Expand All @@ -26,6 +26,6 @@ darts_theorist = DARTSRegressor()
bms_theorist.fit(X,y)
darts_theorist.fit(X,y)
#Sampler
#experimentlist
X_new = model_disagreement_sample(X, [bms_theorist, darts_theorist], n)
```
111 changes: 20 additions & 91 deletions docs/Basic Usage.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Import Modules\n",
"\n",
Expand All @@ -17,11 +13,7 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"\n",
Expand All @@ -35,11 +27,7 @@
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
Expand All @@ -56,9 +44,6 @@
"metadata": {
"vscode": {
"languageId": "plaintext"
},
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
Expand All @@ -70,11 +55,7 @@
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [],
"source": [
"#Define meta-parameters\n",
Expand All @@ -90,11 +71,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Plot The Data\n",
"\n",
Expand All @@ -104,11 +81,7 @@
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand All @@ -129,11 +102,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Define And Fit Theorists\n",
"\n",
Expand All @@ -145,11 +114,7 @@
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"name": "stderr",
Expand Down Expand Up @@ -179,11 +144,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Plot Theorists On Data\n",
"\n",
Expand All @@ -193,11 +154,7 @@
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand All @@ -222,25 +179,17 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Plot Model Disagreements\n",
"\n",
"We can also plot the disagreement between each pair of theorists. The grey dashed lines represent the degree of disagreement for each datapoint. Computationally the Model Disagreement Sampler takes the square of these numbers to determine where disagreement is largest. "
"We can also plot the disagreement between each pair of theorists. The grey dashed lines represent the degree of disagreement for each datapoint. Computationally the Model Disagreement Experimentalist takes the square of these numbers to determine where disagreement is largest."
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand Down Expand Up @@ -272,25 +221,17 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Run And Report Model Disagreement Samples\n",
"\n",
"Now we will get a proposal from the sampler as to which datapoints to investigate next. We will retrieve 5 new datapoints in this example."
"Now we will get a proposal from the experimentalist as to which datapoints to investigate next. We will retrieve 5 new datapoints in this example."
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"name": "stdout",
Expand All @@ -314,11 +255,7 @@
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"metadata": {},
"source": [
"# Plot New Datapoints With Old\n",
"\n",
Expand All @@ -328,11 +265,7 @@
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand All @@ -353,11 +286,7 @@
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"metadata": {},
"outputs": [
{
"data": {
Expand Down Expand Up @@ -405,4 +334,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
6 changes: 3 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Model Disagreement Sampler
# Model Disagreement Experimentalist

The model disagreement sampler identifies experimental conditions $\vec{x}' \in X'$ with respect to
The model disagreement experimentalist identifies experimental conditions $\vec{x}' \in X'$ with respect to
a pairwise distance metric between theorist models, $P_{M_{i}}(\hat{y}, \vec{x}')$:

$$
Expand All @@ -26,6 +26,6 @@ darts_theorist = DARTSRegressor()
bms_theorist.fit(X,y)
darts_theorist.fit(X,y)
#Sampler
#Experimentalist
X_new = model_disagreement_sample(X, [bms_theorist, darts_theorist], n)
```
2 changes: 1 addition & 1 deletion docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ You will need:

- `python` 3.8 or greater: [https://www.python.org/downloads/](https://www.python.org/downloads/)

*Model Disagreement Sampler* is a part of the `autora` package:
*Model Disagreement Experimentalist* is a part of the `autora` package:

```shell
pip install -U autora["experimentalist-model-disagreement"]
Expand Down
11 changes: 5 additions & 6 deletions src/autora/experimentalist/model_disagreement/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ def score_sample(conditions: Union[pd.DataFrame, np.ndarray],
models: List,
num_samples: Optional[int] = None):
"""
A sampler that returns selected samples for independent variables
A experimentalist that returns selected samples for independent variables
for which the models disagree the most in terms of their predictions.
Args:
Expand Down Expand Up @@ -87,11 +87,12 @@ def score_sample(conditions: Union[pd.DataFrame, np.ndarray],
return conditions.head(num_samples)



def sample(conditions: Union[pd.DataFrame, np.ndarray],
models: List,
num_samples: int = 1):
"""
A sampler that returns selected samples for independent variables
A experimentalist that returns selected samples for independent variables
for which the models disagree the most in terms of their predictions.
Args:
Expand All @@ -109,8 +110,6 @@ def sample(conditions: Union[pd.DataFrame, np.ndarray],


model_disagreement_sample = sample
model_disagreement_sample.__doc__ = """Alias for sample"""
model_disagreement_score_sample = score_sample
model_disagreement_score_sample.__doc__ = """Alias for sample"""
model_disagreement_sampler = deprecated_alias(model_disagreement_sample,
"model_disagreement_sampler")
model_disagreement_sampler = deprecated_alias(
model_disagreement_sample, "model_disagreement_sampler")

0 comments on commit f5e4167

Please sign in to comment.