Add type annotations for Beit, ViT and Deit models - Pytorch #16092

johnnv1 · 2022-03-11T22:30:30Z

What does this PR do?

I added type annotations for Beit, ViT and Deit models at PyTorch as described in #16059

At Beit model, have some missing annotations because the annotation is another class at the file that are declared after. Also, at BeitSelfOutput class, I don't know the correct type for the gamma parameter at forward method -- the gamma and input_tensor seems not to be used, but I prefer not to modify this now.

at BeitEmbeddings - on forward method the return is `PatchEmbeddings` at BeitSelfAttention - on forward method the relative_position_bias is `Optional[BeitRelativePositionBias]` at BeitSelfOutput - on forward method i don't know what is the type of gamma, Optional[float]? Also, don't found where this parameter is used. at BeitAttention - on forward method the relative_position_bias is `Optional[BeitRelativePositionBias]` at BeitLayer - on forward method the relative_position_bias is `Optional[BeitRelativePositionBias]` at BeitRelativePositionBias - on forward method the return is `BeitRelativePositionBias`

HuggingFaceDocBuilderDev · 2022-03-11T22:35:27Z

The documentation is not available anymore as the PR was closed or merged.

* 📝 first draft of audio/vision guides * ✨ make fixup * 🖍 fix typo * 🖍 close parentheses * 🖍 apply feedback * 🖍 apply feedback, make fixup * 🖍 more fixup for perceiver * 🖍 apply feedback * ✨ make fixup * 🖍 fix data collator

* gather z3 params for new_lm_head * Update src/transformers/modeling_utils.py Co-authored-by: Stas Bekman <[email protected]> Co-authored-by: Stas Bekman <[email protected]>

* [WIP] add support for bf16 mode * prep for bf16 * prep for bf16 * fix; zero2/bf16 is ok * check bf16 is available * test fixes * enable zero3_bf16 * config files * docs * split stage_dtype; merge back to non-dtype-specific config file * fix doc * cleanup * cleanup * bfloat16 => bf16 to match the PR changes * s/zero_gather_fp16_weights_on_model_save/zero_gather_16bit_weights_on_model_save/; s/save_fp16_model/save_16bit_model/ * test fixes/skipping * move * fix * Update docs/source/main_classes/deepspeed.mdx Co-authored-by: Sylvain Gugger <[email protected]> * backticks * cleanup * cleanup * cleanup * new version * add note about grad accum in bf16 Co-authored-by: Sylvain Gugger <[email protected]>

Rocketknight1 · 2022-03-12T12:35:06Z

Hi, thank you for this! I won't merge it yet because I'm going to ask the other team members on Monday what we think about the missing annotations - it's possible to use forward references for this, but I want to make sure that plays well with our code analysis tools before we do that.

johnnv1 · 2022-03-12T13:01:51Z

Already have a command that runs something like mypy to check the types?

…16089) * Add missing type hints for all flavors of LayoutLMv2 PyTorch models. * Fixed return types and added type hints for LayoutLM. * Fix removed arguments which breaks tests.

* Make Camembert great again! * Add Camembert to TensorFlow ONNX tests

… Encoder-Decoder Checkpoints (#16056) * Fix Loading of Flax(Speech)EncoderDecoderModel kwargs from PreTrained Encoder-Decoder Checkpoints * change wording

Configuration `tied-embeddings-all` implies `tied-embeddings-src`

* Make TF pt-tf equivalence test more aggressive * Fix for TFConvNextModelTest and TFTransfoXLModelTest * fix kwargs for outputs * clean-up * Add docstring for check_outputs() * remove: need to rename encoder-decoder * clean-up * send PyTorch things to the correct device * Add back the accidentally removed test case in test_pt_tf_model_equivalence() * Fix: change to tuple before calling check_outputs() * Fix: tfo could be a list * use to_tuple() * allow tfo only to be tuple or tensor * allow tfo to be list or tuple for now + style change * minor fix * remove np.copy and update comments * tfo -> tf_output, same for pt * Add more detailed comment * remove the incorrect comment Co-authored-by: ydshieh <[email protected]>

Co-authored-by: ydshieh <[email protected]>

* Change unpacking of TF mobilebert inputs to use decorator * Move unpack_inputs as the top decorator * make fixup Co-authored-by: ChienVM <[email protected]>

Co-authored-by: Niels Rogge <[email protected]>

* Spanish translation of the file training.mdx * Settings - Spanish translation of the file training.mdx * Latest changes to the Spanish translation of the training.mdx file * Delete Hugging.mdx * Last changes to the training fil Espanish version * Latest modifications * Latest changes, document ready for PR * Nits Co-authored-by: Yhary Arias <[email protected]> Co-authored-by: Omar U. Espejel <[email protected]>

Rocketknight1 · 2022-03-14T14:49:56Z

Hi @johnnv1, I just confirmed with the team that forward references are fine for types that aren't defined yet!

…Text2 (#16083) * Fix checkpoint name in docstring example Co-authored-by: ydshieh <[email protected]> Co-authored-by: Patrick von Platen <[email protected]>

* Replace input_processing * move unpack_inputs

* First pass * Fixup * Fix broken tests * Make unpack_inputs the first decorator

* Add type hints for TFDistilBert * Update src/transformers/models/distilbert/modeling_tf_distilbert.py Co-authored-by: Matt <[email protected]>

* Can choose framework for ONNX export * Fix docstring

* Add type hints for LukeModel * Add type hints for entitypairclassification * Remove blank space Co-authored-by: bhavika <bhavika@debian-BULLSEYE-live-builder-AMD64>

* Add type hints for SqueezeBert PyTorch * fixed unused List err * style fixes

* Add missing type hints - ELECTRA TF * bool -> Optional[bool]

* Runtime -> Devel * Torch before DeepSpeed

* clip typhinting #16059 * removed optional type annotations for dataclass in CLIPOutput * type annotation fixes per Rocket - Clip Torch

at BeitEmbeddings - on forward method the return is `PatchEmbeddings` at BeitSelfAttention - on forward method the relative_position_bias is `Optional[BeitRelativePositionBias]` at BeitSelfOutput - on forward method i don't know what is the type of gamma, Optional[float]? Also, don't found where this parameter is used. at BeitAttention - on forward method the relative_position_bias is `Optional[BeitRelativePositionBias]` at BeitLayer - on forward method the relative_position_bias is `Optional[BeitRelativePositionBias]` at BeitRelativePositionBias - on forward method the return is `BeitRelativePositionBias`

…v1/transformers into add_type_annotations/vision_models

johnnv1 · 2022-03-14T18:07:50Z

@Rocketknight1 after rebase upstream/master the make fixup shows that vit_mae need to be updated because of the # Copied from ..., but when I run make fix-copies he has been adding the ViTConfig on the config type annotation, I think these need to be ViTMAEConfig, what is the right way to deal with this?

Rocketknight1 · 2022-03-14T19:01:15Z

Hi @johnnv1, that rebase made this pretty tricky to review! It might be easier to close this and start a new PR and just copy the changes over? If there are classes that are copies then the original source of the copy should be changed, followed by make fix-copies. If other people are working on the classes you need to change, it might be easiest to wait until those PRs are done before starting yours!

johnnv1 added 3 commits March 11, 2022 19:23

add types annotations for ViT (PyTorch)

a2124da

add types annotations for Deit (PyTorch)

7c5dc97

stevhliu and others added 5 commits March 11, 2022 16:43

Audio/vision task guides (#15808)

ae2dd42

* 📝 first draft of audio/vision guides * ✨ make fixup * 🖍 fix typo * 🖍 close parentheses * 🖍 apply feedback * 🖍 apply feedback, make fixup * 🖍 more fixup for perceiver * 🖍 apply feedback * ✨ make fixup * 🖍 fix data collator

[ZeRO] Fixes issue with embedding resize (#16093)

c1f209d

* gather z3 params for new_lm_head * Update src/transformers/modeling_utils.py Co-authored-by: Stas Bekman <[email protected]> Co-authored-by: Stas Bekman <[email protected]>

Change unpacking of TF Bart inputs (#16094)

3e9d0f7

add unpack_inputs decorator to mbart (#16097)

9042dfe

Add type annotations for segformer classes (#16099)

62b05b6

johnnv1 and others added 13 commits March 12, 2022 15:05

apply unpack_input decorator to ViT model (#16102)

8416206

Add type hints to XLM model (PyTorch) (#16108)

65cf33e

Add missing type hints for all flavors of LayoutLMv2 PyTorch models. (#…

20ab158

…16089) * Add missing type hints for all flavors of LayoutLMv2 PyTorch models. * Fixed return types and added type hints for LayoutLM. * Fix removed arguments which breaks tests.

Add TFCamembertForCausalLM and ONNX integration test (#16073)

6e1e88f

* Make Camembert great again! * Add Camembert to TensorFlow ONNX tests

Fix and document Zero Shot Image Classification (#16079)

802984a

Fix Loading of Flax(Speech)EncoderDecoderModel kwargs from PreTrained…

2de99e6

… Encoder-Decoder Checkpoints (#16056) * Fix Loading of Flax(Speech)EncoderDecoderModel kwargs from PreTrained Encoder-Decoder Checkpoints * change wording

Update convert_marian_to_pytorch.py (#16124)

9e9f6b8

Configuration `tied-embeddings-all` implies `tied-embeddings-src`

Fix ProphetNetTokenizer (#16082)

5dbf36b

Co-authored-by: ydshieh <[email protected]>

Change unpacking of TF mobilebert inputs to use decorator (#16110)

e3645fd

* Change unpacking of TF mobilebert inputs to use decorator * Move unpack_inputs as the top decorator * make fixup Co-authored-by: ChienVM <[email protected]>

steps strategy fix for PushtoHubCallback (#16138)

f284aa3

Add copied from statements and fix prefix (#16119)

9fd584e

Co-authored-by: Niels Rogge <[email protected]>

kamalkraj and others added 4 commits March 14, 2022 14:53

Added missing type hints (#16103)

1d43933

Added missing type hints - V1 and V2 (#16105)

72ae06b

[Fix doc example] Fix checkpoint name in docstring example in Speech2…

c8c8c11

…Text2 (#16083) * Fix checkpoint name in docstring example Co-authored-by: ydshieh <[email protected]> Co-authored-by: Patrick von Platen <[email protected]>

Better input variable naming for OpenAI (TF) (#16129)

d02bd4f

* Replace input_processing * move unpack_inputs

bhavika and others added 16 commits March 14, 2022 15:26

Improve model variable naming - CLIP [TF] (#16128)

97e32b7

* First pass * Fixup * Fix broken tests * Make unpack_inputs the first decorator

Add type hints for TFDistilBert (#16107)

3f8360a

* Add type hints for TFDistilBert * Update src/transformers/models/distilbert/modeling_tf_distilbert.py Co-authored-by: Matt <[email protected]>

Choose framework for ONNX export (#16018)

37a9fc4

* Can choose framework for ONNX export * Fix docstring

Add type hints for Luke in PyTorch (#16111)

6c2f3ed

* Add type hints for LukeModel * Add type hints for entitypairclassification * Remove blank space Co-authored-by: bhavika <bhavika@debian-BULLSEYE-live-builder-AMD64>

Add type hints for PoolFormer in Pytorch (#16121)

5493c10

Add type hints for SqueezeBert PyTorch (#16126)

6db8693

* Add type hints for SqueezeBert PyTorch * fixed unused List err * style fixes

Added missing type hints - ELECTRA TF (#16104)

10cf1ff

* Add missing type hints - ELECTRA TF * bool -> Optional[bool]

Dcoker images runtime -> devel (#16141)

c1000e7

* Runtime -> Devel * Torch before DeepSpeed

Add type annotations for CLIP (torch) (#16059) (#16106)

f86235a

* clip typhinting #16059 * removed optional type annotations for dataclass in CLIPOutput * type annotation fixes per Rocket - Clip Torch

Add type hints for FNet PyTorch (#16123)

0dcdfe8

Use HF_ENDPOINT for custom endpoints (#16139)

e109edf

add types annotations for ViT (PyTorch)

545b66e

add types annotations for Deit (PyTorch)

64b289a

complete types annotations for Beit (PyTorch)

0c22daf

Merge branch 'add_type_annotations/vision_models' of github.com:johnn…

39283a2

…v1/transformers into add_type_annotations/vision_models

Rocketknight1 self-assigned this Mar 14, 2022

johnnv1 closed this Mar 14, 2022

johnnv1 mentioned this pull request Mar 14, 2022

Add/type annotations/model vision #16151

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add type annotations for Beit, ViT and Deit models - Pytorch #16092

Add type annotations for Beit, ViT and Deit models - Pytorch #16092

johnnv1 commented Mar 11, 2022 •

edited

Loading

HuggingFaceDocBuilderDev commented Mar 11, 2022 •

edited

Loading

Rocketknight1 commented Mar 12, 2022

johnnv1 commented Mar 12, 2022

Rocketknight1 commented Mar 14, 2022 •

edited

Loading

johnnv1 commented Mar 14, 2022

Rocketknight1 commented Mar 14, 2022

Add type annotations for Beit, ViT and Deit models - Pytorch #16092

Add type annotations for Beit, ViT and Deit models - Pytorch #16092

Conversation

johnnv1 commented Mar 11, 2022 • edited Loading

What does this PR do?

HuggingFaceDocBuilderDev commented Mar 11, 2022 • edited Loading

Rocketknight1 commented Mar 12, 2022

johnnv1 commented Mar 12, 2022

Rocketknight1 commented Mar 14, 2022 • edited Loading

johnnv1 commented Mar 14, 2022

Rocketknight1 commented Mar 14, 2022

johnnv1 commented Mar 11, 2022 •

edited

Loading

HuggingFaceDocBuilderDev commented Mar 11, 2022 •

edited

Loading

Rocketknight1 commented Mar 14, 2022 •

edited

Loading