Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove sampler category #8

Merged
merged 2 commits into from
Sep 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AutoRA Uncertainty Sampler
# AutoRA Uncertainty Experimentalist

The uncertainty sampler identifies experimental conditions $\vec{x}' \in X'$ with respect model uncertainty. Within the uncertainty sampler, there are three methods to determine uncertainty:
The uncertainty experimentalist identifies experimental conditions $\vec{x}' \in X'$ with respect model uncertainty. Within the uncertainty sampler, there are three methods to determine uncertainty:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we need to change 'sampler' to 'experimentalist' in second sentence also, correct? also missing 'to' in 'with respect to model uncertainty'


## Least Confident
$$
Expand All @@ -25,7 +25,7 @@ $$
# Example Code

```
from autora.experimentalist.sampler.uncertainty import uncertainty_sample
from autora.experimentalist.uncertainty import uncertainty_sample
from sklearn.linear_model import LogisticRegression
import numpy as np

Expand Down
4 changes: 2 additions & 2 deletions docs/Basic Usage.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"outputs": [],
"source": [
"# Uncomment the following line when running on Google Colab\n",
"# !pip install \"autora[experimentalist-sampler-uncertainty]\""
"# !pip install \"autora[experimentalist-uncertainty]\""
]
},
{
Expand All @@ -27,7 +27,7 @@
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from sklearn.linear_model import LogisticRegression\n",
"from autora.experimentalist.sampler.uncertainty import uncertainty_sample"
"from autora.experimentalist.uncertainty import uncertainty_sample"
]
},
{
Expand Down
4 changes: 2 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Uncertainty Sampler
# Uncertainty Experimentalist

The uncertainty sampler identifies experimental conditions $\vec{x}' \in X'$ with respect model uncertainty. Within the uncertainty sampler, there are three methods to determine uncertainty:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, need to update 'sampler'


Expand All @@ -25,7 +25,7 @@ $$
# Example Code

```
from autora.experimentalist.sampler.uncertainty import uncertainty_sample
from autora.experimentalist.uncertainty import uncertainty_sample
from sklearn.linear_model import LogisticRegression
import numpy as np

Expand Down
4 changes: 2 additions & 2 deletions docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ You will need:
*Uncertainty-Sampler is a part of the `autora` package:

```shell
pip install -U autora["experimentalist-sampler-uncertainty"]
pip install -U autora["experimentalist-uncertainty"]
```


Check your installation by running:
```shell
python -c "from autora.experimentalist.sampler.uncertainty import uncertainty_sample"
python -c "from autora.experimentalist.uncertainty import uncertainty_sample"
```
4 changes: 2 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# yaml-language-server: $schema=https://squidfunk.github.io/mkdocs-material/schema.json
INHERIT: mkdocs/base.yml # use the shared AutoRA configuration by default

site_name: AutoRA Uncertainty Sampler
repo_url: 'https://github.com/AutoResearch/autora-experimentalist-sampler-uncertainty'
site_name: AutoRA Uncertainty
repo_url: 'https://github.com/AutoResearch/autora-experimentalist-uncertainty'

nav:
- Home: 'index.md'
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
# UPDATE THIS BEFORE PUBLISHING
name = "autora-experimentalist-sampler-uncertainty"
name = "autora-experimentalist-uncertainty"
description = "Sampler based on where the model is least certain."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here?

authors = [{ name = "Built by Sebastian Musslick, restructured by Chad Williams", email = "[email protected]" }]
dynamic = ["version"]
Expand All @@ -22,7 +22,7 @@ dev = [

[project.urls]
homepage = "http://www.empiricalresearch.ai"
repository = "https://github.com/AutoResearch/autora-experimentalist-sampler-uncertainty"
repository = "https://github.com/AutoResearch/autora-experimentalist-uncertainty"
documentation = "https://autoresearch.github.io/autora/"


Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
from typing import Iterable

import numpy as np
from scipy.stats import entropy

from autora.utils.deprecation import deprecated_alias

def uncertainty_sample(X, model, num_samples, measure="least_confident"):

def sample(X, model, num_samples, measure="least_confident"):
"""

Args:
Expand Down Expand Up @@ -60,4 +62,6 @@ class labels under the model, respectively.

return X[idx]


uncertainty_sample = sample
uncertainty_sampler = deprecated_alias(uncertainty_sample, "uncertainty_sampler")
18 changes: 10 additions & 8 deletions tests/test_uncertainty_sampler.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
from autora.experimentalist.sampler.uncertainty import uncertainty_sample
from sklearn.linear_model import LogisticRegression
import numpy as np
from sklearn.linear_model import LogisticRegression

from autora.experimentalist.uncertainty import uncertainty_sample


def test_output_dimensions():
#Meta-Setup
# Meta-Setup
X = np.linspace(start=-3, stop=6, num=10).reshape(-1, 1)
y = (X**2).reshape(-1)
n = 5
#Theorists

# Theorists
lr_theorist = LogisticRegression()

lr_theorist.fit(X,y)

#Sampler
lr_theorist.fit(X, y)

# Sampler
X_new = uncertainty_sample(X, lr_theorist, n)

# Check that the sampler returns n experiment conditions
Expand Down