Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a more precise restore key for GH cache action #6614

Merged
merged 10 commits into from
Sep 10, 2020

Conversation

tczekajlo
Copy link

@tczekajlo tczekajlo commented Sep 9, 2020

Use a more precise restore key for GH cache action - #6503

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2020

Commit: ba98098, The full report is available as an artifact.

Dataset: Carbon Bot

Configuration Intent Classification Micro F1 Entity Recognition Micro F1 Response Selection Micro F1
Sparse + DIET(bow) + ResponseSelector(bow)
test: 23s, train: 4m43s, total: 5m6s
0.7417 0.6260 0.5166

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2020

Commit: ba98098, The full report is available as an artifact.

Dataset: Carbon Bot

Configuration Intent Classification Micro F1 Entity Recognition Micro F1 Response Selection Micro F1
Sparse + DIET(bow) + ResponseSelector(bow)
test: 20s, train: 3m0s, total: 3m20s
0.7262 0.6260 0.5232

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2020

Commit: efd4734, The full report is available as an artifact.

Dataset: Carbon Bot

Configuration Intent Classification Micro F1 Entity Recognition Micro F1 Response Selection Micro F1
Sparse + DIET(bow) + ResponseSelector(bow)
test: 21s, train: 4m36s, total: 4m57s
0.7417 0.6260 0.5166

Copy link
Collaborator

@m-vdb m-vdb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:octocat:

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2020

Hey @tczekajlo! 👋 To run model regression tests, comment with the /modeltest command and a configuration.

Tips 💡: The model regression test will be run on push events. You can re-run the tests by re-add status:model-regression-tests label or use a Re-run jobs button in Github Actions workflow.

Tips 💡: Every time when you want to change a configuration you should edit the comment with the previous configuration.

You can copy this in your comment and customize:

/modeltest

```yml
##########
## Available datasets
##########
# - "Carbon Bot"
# - "Hermit"
# - "Private 1"
# - "Private 2"
# - "Private 3"
# - "Sara"

##########
## Available configurations
##########
# - "BERT + DIET(bow) + ResponseSelector(bow)"
# - "BERT + DIET(seq) + ResponseSelector(t2t)"
# - "ConveRT + DIET(bow) + ResponseSelector(bow)"
# - "ConveRT + DIET(seq) + ResponseSelector(t2t)"
# - "Spacy + DIET(bow) + ResponseSelector(bow)"
# - "Spacy + DIET(seq) + ResponseSelector(t2t)"
# - "Sparse + ConveRT + DIET(bow) + ResponseSelector(bow)"
# - "Sparse + ConveRT + DIET(seq) + ResponseSelector(t2t)"
# - "Sparse + DIET(bow) + ResponseSelector(bow)"
# - "Sparse + DIET(seq) + ResponseSelector(t2t)"
# - "Sparse + Spacy + DIET(bow) + ResponseSelector(bow)"
# - "Sparse + Spacy + DIET(seq) + ResponseSelector(t2t)"

## Example configuration
#################### syntax #################
## include:
##   - dataset: ["<dataset_name>"]
##     config: ["<configuration_name>"]
#
## Example:
## include:
##  - dataset: ["Carbon Bot"]
##    config: ["Sparse + DIET(bow) + ResponseSelector(bow)"]
#
## Shortcut:
## You can use the "all" shortcut to include all available configurations or datasets
#
## Example: Use the "Sparse + EmbeddingIntent + ResponseSelector(bow)" configuration
## for all available datasets
## include:
##  - dataset: ["all"]
##    config: ["Sparse + DIET(bow) + ResponseSelector(bow)"]
#
## Example: Use all available configurations for the "Carbon Bot" and "Sara" datasets
## and for the "Hermit" dataset use the "Sparse + DIET + ResponseSelector(T2T)" and
## "Sparse + ConveRT + DIET + ResponseSelector(T2T)" configurations:
## include:
##  - dataset: ["Carbon Bot", "Sara"]
##    config: ["all"]
##  - dataset: ["Hermit"]
##    config: ["Sparse + DIET(seq) + ResponseSelector(t2t)", "Sparse + ConveRT + DIET(seq) + ResponseSelector(t2t)"]

include:
 - dataset: ["Carbon Bot"]
   config: ["Sparse + DIET(bow) + ResponseSelector(bow)"]

```

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2020

/modeltest

include:
 - dataset: ["Carbon Bot"]
   config: ["Sparse + DIET(bow) + ResponseSelector(bow)"]

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2020

The model regression tests have started. It might take a while, please be patient.
As soon as results are ready you'll see a new comment with the results.

Used configuration can be found in the comment.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2020

Commit: ecf6aa8, The full report is available as an artifact.

Dataset: Carbon Bot

Configuration Intent Classification Micro F1 Entity Recognition Micro F1 Response Selection Micro F1
Sparse + DIET(bow) + ResponseSelector(bow)
test: 22s, train: 4m27s, total: 4m50s
0.7417 0.6260 0.5166

@rasabot rasabot merged commit 4c590b4 into master Sep 10, 2020
@rasabot rasabot deleted the add_cache_restore_key_mrt branch September 10, 2020 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Model regression tests fail because of a dependency issue
3 participants