Add TF DeiT implementation #17806

amyeroberts · 2022-06-21T19:10:28Z

What does this PR do?

Adds the TF implementations of DeiT

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

…o add-tf-deit

HuggingFaceDocBuilderDev · 2022-06-21T19:37:15Z

The documentation is not available anymore as the PR was closed or merged.

Removing to_2tuple results in many downstream changes needed because of the copies checks

NielsRogge · 2022-06-22T13:24:25Z

Please also incorporate the updates of #17731

amyeroberts · 2022-06-22T17:40:57Z

Please also incorporate the updates of #17731

@NielsRogge Will do! I originally added, but it resulted in lots of changes because of all the Copied From statements, so will wait until your PR is merged and those updates finalised.

tests/models/deit/test_modeling_tf_deit.py

src/transformers/models/deit/modeling_tf_deit.py

tests/models/deit/test_modeling_tf_deit.py

Rocketknight1

I couldn't see any issues with the ported TF code anywhere! (Except my usual nitpicks that people are using tf.multiply and tf.matmul instead of just * and @, but that's purely cosmetic and copied from another file anyway)

Also, tests seem comprehensive, so this PR looks good to me!

patrickvonplaten

Deferring to @NielsRogge and @gante here as it's Vision and TF. Happy to do a final review though if you want :-)

patrickvonplaten · 2022-07-07T12:55:12Z

@sgugger could you maybe give it a quick review as it's vision? (otherwise happy to do it if you're busy)

src/transformers/models/deit/modeling_tf_deit.py

NielsRogge · 2022-07-07T13:32:32Z

src/transformers/models/deit/modeling_tf_deit.py

+        )
+
+
+@dataclass


Usually I add model output classes at the top of the modeling file

Agreed - it's much nicer. I've moved it to the top :)

tests/models/deit/test_modeling_tf_deit.py

NielsRogge

LGTM! Very clean!

sgugger

Very clean, thanks a lot for porting this model to TensorFlow!

tests/models/deit/test_modeling_tf_deit.py

Co-authored-by: Sylvain Gugger <[email protected]> Co-authored-by: NielsRogge <[email protected]> Co-authored-by: NielsRogge <[email protected]> Co-authored-by: Sylvain Gugger <[email protected]>

* Initial TF DeiT implementation * Fix copies naming issues * Fix up + docs * Properly same main layer * Name layers properly * Initial TF DeiT implementation * Fix copies naming issues * Fix up + docs * Properly same main layer * Name layers properly * Fixup * Fix import * Fix import * Fix import * Fix weight loading for tests whilst not on hub * Add doc tests and remove to_2tuple * Add back to_2tuple Removing to_2tuple results in many downstream changes needed because of the copies checks * Incorporate updates in Improve vision models huggingface#17731 PR * Don't hard code num_channels * Copy PyTorch DeiT embeddings and remove pytorch operations with mask * Fix patch embeddings & tidy up * Update PixelShuffle to move logic into class layer * Update doc strings - remove PT references * Use NHWC format in internal layers * Fix up * Use linear activation layer * Remove unused import * Apply suggestions from code review Co-authored-by: Sylvain Gugger <[email protected]> Co-authored-by: NielsRogge <[email protected]> Co-authored-by: NielsRogge <[email protected]> Co-authored-by: Sylvain Gugger <[email protected]> * Move dataclass to top of file * Remove from_pt now weights on hub * Fixup Co-authored-by: NielsRogge <[email protected]> Co-authored-by: Sylvain Gugger <[email protected]> Co-authored-by: Amy Roberts <[email protected]>

amyeroberts added 13 commits June 17, 2022 17:23

Initial TF DeiT implementation

728c600

Fix copies naming issues

565139e

Fix up + docs

6f05fed

Properly same main layer

1601ee8

Name layers properly

271cea4

Initial TF DeiT implementation

078448c

Fix copies naming issues

38a45ad

Fix up + docs

28d197b

Properly same main layer

a6ae3a5

Name layers properly

b4dac42

Merge branch 'add-tf-deit' of github.com:amyeroberts/transformers int…

2df3bad

…o add-tf-deit

Fixup

b7ebb21

Fix import

abc9be3

amyeroberts added 5 commits June 22, 2022 10:32

Fix import

37d9140

Fix import

b247ef3

Fix weight loading for tests whilst not on hub

c0a82eb

Add doc tests and remove to_2tuple

3799e3c

Add back to_2tuple

ea5f126

Removing to_2tuple results in many downstream changes needed because of the copies checks

amyeroberts added 7 commits June 24, 2022 11:09

Merge in main and resolve conflicts

629d552

Incorporate updates in Improve vision models huggingface#17731 PR

79c85dc

Merge branch 'main' into add-tf-deit

61a2541

Merge branch 'main' into add-tf-deit

228bd41

Don't hard code num_channels

139be9f

Copy PyTorch DeiT embeddings and remove pytorch operations with mask

40d5f58

Fix patch embeddings & tidy up

86d91d4

amyeroberts marked this pull request as ready for review June 24, 2022 21:53

amyeroberts changed the title ~~[WIP] Add tf deit~~ Add TF DeiT implementation Jun 24, 2022

amyeroberts added 7 commits June 28, 2022 13:29

Update doc strings - remove PT references

629ad1e

Use NHWC format in internal layers

32d6bc8

Resolve conflict - two models added to auto

890d090

Fix up

cb24ac0

Merge branch 'main' into add-tf-deit

03efdc8

Use linear activation layer

f27d4e7

Remove unused import

5752bf0

amyeroberts commented Jul 5, 2022

View reviewed changes

tests/models/deit/test_modeling_tf_deit.py Outdated Show resolved Hide resolved

amyeroberts commented Jul 5, 2022

View reviewed changes

src/transformers/models/deit/modeling_tf_deit.py Outdated Show resolved Hide resolved

amyeroberts commented Jul 5, 2022

View reviewed changes

src/transformers/models/deit/modeling_tf_deit.py Outdated Show resolved Hide resolved

amyeroberts commented Jul 5, 2022

View reviewed changes

tests/models/deit/test_modeling_tf_deit.py Outdated Show resolved Hide resolved

amyeroberts requested review from NielsRogge, patrickvonplaten and Rocketknight1 July 5, 2022 09:52

Rocketknight1 approved these changes Jul 5, 2022

View reviewed changes

patrickvonplaten reviewed Jul 7, 2022

View reviewed changes

NielsRogge reviewed Jul 7, 2022

View reviewed changes

src/transformers/models/deit/modeling_tf_deit.py Outdated Show resolved Hide resolved

NielsRogge reviewed Jul 7, 2022

View reviewed changes

tests/models/deit/test_modeling_tf_deit.py Outdated Show resolved Hide resolved

NielsRogge approved these changes Jul 7, 2022

View reviewed changes

sgugger approved these changes Jul 7, 2022

View reviewed changes

tests/models/deit/test_modeling_tf_deit.py Outdated Show resolved Hide resolved

amyeroberts and others added 5 commits July 8, 2022 11:32

Apply suggestions from code review

efc22ca

Co-authored-by: Sylvain Gugger <[email protected]> Co-authored-by: NielsRogge <[email protected]> Co-authored-by: NielsRogge <[email protected]> Co-authored-by: Sylvain Gugger <[email protected]>

Move dataclass to top of file

8372cee

Remove from_pt now weights on hub

07c14c9

Merge branch 'main' into add-tf-deit

88db674

Fixup

3936a0c

amyeroberts merged commit 8581a79 into huggingface:main Jul 13, 2022

amyeroberts deleted the add-tf-deit branch July 13, 2022 17:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add TF DeiT implementation #17806

Add TF DeiT implementation #17806

amyeroberts commented Jun 21, 2022

HuggingFaceDocBuilderDev commented Jun 21, 2022 •

edited

Loading

NielsRogge commented Jun 22, 2022

amyeroberts commented Jun 22, 2022

Rocketknight1 left a comment

patrickvonplaten left a comment

patrickvonplaten commented Jul 7, 2022

NielsRogge Jul 7, 2022

amyeroberts Jul 8, 2022

NielsRogge left a comment

sgugger left a comment

		)


		@dataclass

Add TF DeiT implementation #17806

Add TF DeiT implementation #17806

Conversation

amyeroberts commented Jun 21, 2022

What does this PR do?

Before submitting

HuggingFaceDocBuilderDev commented Jun 21, 2022 • edited Loading

NielsRogge commented Jun 22, 2022

amyeroberts commented Jun 22, 2022

Rocketknight1 left a comment

Choose a reason for hiding this comment

patrickvonplaten left a comment

Choose a reason for hiding this comment

patrickvonplaten commented Jul 7, 2022

NielsRogge Jul 7, 2022

Choose a reason for hiding this comment

amyeroberts Jul 8, 2022

Choose a reason for hiding this comment

NielsRogge left a comment

Choose a reason for hiding this comment

sgugger left a comment

Choose a reason for hiding this comment

HuggingFaceDocBuilderDev commented Jun 21, 2022 •

edited

Loading