Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

[FEATURE] Add raw attention scores to the AttentionCell #951 #964

Merged
merged 5 commits into from
Oct 25, 2019

Conversation

emilmont
Copy link
Contributor

@emilmont emilmont commented Oct 8, 2019

Description

For implementing a pointer mechanism in sequence to sequence models it is very practical to re-use attention cells. For example see the Attention-Based Copy Mechanism described in Jia, Robin, and Percy Liang. "Data recombination for neural semantic parsing." arXiv preprint arXiv:1606.03622 (2016).
The proposal is to additionally return the raw attention scores in the AttentionCell.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Comments

  • The change is a backward compatible

cc @dmlc/gluon-nlp-team

@emilmont emilmont requested a review from a team as a code owner October 8, 2019 15:41
@codecov
Copy link

codecov bot commented Oct 8, 2019

Codecov Report

Merging #964 into master will increase coverage by 0.03%.
The diff coverage is 87.87%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #964      +/-   ##
=========================================
+ Coverage   89.87%   89.9%   +0.03%     
=========================================
  Files          67      67              
  Lines        6371    6390      +19     
=========================================
+ Hits         5726    5745      +19     
  Misses        645     645
Impacted Files Coverage Δ
src/gluonnlp/model/attention_cell.py 95.06% <87.87%> (+0.65%) ⬆️

@mli
Copy link
Member

mli commented Oct 8, 2019

Job PR-964/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-964/1/index.html

@mli
Copy link
Member

mli commented Oct 8, 2019

Job PR-964/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-964/2/index.html

@mli
Copy link
Member

mli commented Oct 10, 2019

Job PR-964/3 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-964/3/index.html

@szha szha requested review from sxjscience and leezu October 16, 2019 20:18
@mli
Copy link
Member

mli commented Oct 25, 2019

Job PR-964/5 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-964/5/index.html

@leezu leezu merged commit 06ecac8 into dmlc:master Oct 25, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants