Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eval metadata Batch 4: Tweet Eval, Tweets Hate Speech Detection, VCTK, Weibo NER, Wisesight Sentiment, XSum, Yahoo Answers Topics, Yelp Polarity, Yelp Review Full #4338

Merged
merged 11 commits into from
May 16, 2022

Conversation

sashavor
Copy link
Contributor

Adding evaluation metadata for:

  • Tweet Eval
  • Tweets Hate Speech Detection
  • VCTK
  • Weibo NER
  • Wisesight Sentiment
  • XSum
  • Yahoo Answers Topics
  • Yelp Polarity
  • Yelp Review Full

…bo NER, Wisesight Sentiment, XSum, Yahoo Answers Topics, Yelp Polarity, Yelp Review Full
@sashavor sashavor requested review from lewtun and lhoestq May 12, 2022 21:02
@sashavor sashavor changed the title Eval metadata for Tweet Eval, Tweets Hate Speech Detection, VCTK, Weibo NER, Wisesight Sentiment, XSum, Yahoo Answers Topics, Yelp Polarity, Yelp Review Full Eval metadata Batch 4: Tweet Eval, Tweets Hate Speech Detection, VCTK, Weibo NER, Wisesight Sentiment, XSum, Yahoo Answers Topics, Yelp Polarity, Yelp Review Full May 12, 2022
Sasha Luccioni added 3 commits May 12, 2022 17:24
oops, duplicate
adding header
adding header
@sashavor
Copy link
Contributor Author

Summary of CircleCI errors:

  • XSum: missing 6 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', and 'source_datasets'
  • Yelp_polarity: missing 8 required positional arguments: 'annotations_creators', 'language_creators', 'licenses', 'multilinguality', 'size_categories', 'source_datasets', 'task_categories', and 'task_ids'

Copy link
Member

@lewtun lewtun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job on powering through these batches of metadata @sashavor - this looks really nice 🔥 !!

Just a few suggestions / nits that are similar to previous PRs :)

@@ -89,6 +89,247 @@ task_ids:
- multi-class-classification
paperswithcode_id: tweeteval
pretty_name: TweetEval
train-eval-index:
- config: emotion
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool example of a multi-config dataset!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😎

datasets/weibo_ner/README.md Show resolved Hide resolved
datasets/xsum/README.md Show resolved Hide resolved
datasets/yelp_polarity/README.md Outdated Show resolved Hide resolved
datasets/xsum/README.md Outdated Show resolved Hide resolved
Sasha Luccioni and others added 7 commits May 16, 2022 11:14
Co-authored-by: Quentin Lhoest <[email protected]>
removing ROUGE args
removing F1 params
updating to binary F1
oops typo
F1 binary
@sashavor sashavor merged commit bce8e65 into master May 16, 2022
@sashavor sashavor deleted the eval-metadata-batch4 branch May 16, 2022 15:43
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented May 16, 2022

The documentation is not available anymore as the PR was closed or merged.

nandwalritik pushed a commit to nandwalritik/datasets that referenced this pull request May 16, 2022
…, Weibo NER, Wisesight Sentiment, XSum, Yahoo Answers Topics, Yelp Polarity, Yelp Review Full (huggingface#4338)

* Eval metadata for Tweet Eval, Tweets Hate Speech Detection, VCTK, Weibo NER, Wisesight Sentiment, XSum, Yahoo Answers Topics, Yelp Polarity, Yelp Review Full

* Update README.md

oops, duplicate

* Update README.md

adding header

* Update README.md

adding header

* Update datasets/xsum/README.md

Co-authored-by: Quentin Lhoest <[email protected]>

* Update README.md

removing ROUGE args

* Update README.md

removing F1 params

* Update README.md

* Update README.md

updating to binary F1

* Update README.md

oops typo

* Update README.md

F1 binary

Co-authored-by: sashavor <[email protected]>
Co-authored-by: Quentin Lhoest <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants