* Update README.md

* tests: forward ok

* backward test done

* done testing

* removed check. scripts

* Update README.md

* added use_mambapy arg

* fixed typo in warning

* protected imports w/ mambapy package

* delete pscan.py + raise rather than assert

* Update import_utils.py

* fix whitespaces and unused import

* trailing whitespace + import block unformatted

* Update modeling_mamba.py

* transpose before pscan

* shape comment

* ran make style

* use_mambapy=False by default

Co-authored-by: Arthur <[email protected]>

* ran make fix-copies

---------

Co-authored-by: Arthur <[email protected]>

* renamed phi3 rope_scaling type

* fixed trailing whitespaces

* fixed test

* added warning

* fixed format

…e#32148)

Revert "Incorrect Whisper long-form decoding timestamps  (huggingface#32003)"

This reverts commit cd48553.

…ingface#31857)

* feat(cache): StaticCache uses index_copy_ to avoid useless copy

Using index_copy_ allows for explicit in-place change of the tensor.
Some backends (XLA) will otherwise copy the tensor, making the code
slower and using more memory.

Proposed implementation will end up using less memory and on XLA will
result in less compilation, but the change is also quite generic, making
no change whatsoever on CUDA or CPU backend.

* feat(cache): SlidingWindowCache uses index_copy_ to avoid useless copy

Applying the same change done in StaticCache.

* fix(cache): fallback of index_copy_ when not implemented

* fix(cache): in index_copy_ ensure tensors are on same device

* [run slow] llama

* fix(cache): add move of cache_position to same device in SlidingWindowCache

* Revert "[run slow] llama"

This reverts commit 02608dd.

…r search (huggingface#31924)

Update integration_utils.py

Added additional kwarg

…ith Position IDs (huggingface#31629)

* add DataCollatorBatchFlattening

* Update data_collator.py

* change name

* new FA2 flow if position_ids is provided

* add comments

* minor fix

* minor fix data collator

* add test cases for models

* add test case for data collator

* remove extra code

* formating for ruff check and check_repo.py

* ruff format

ruff format tests src utils

* custom_init_isort.py

* Updated ruff version and fixed the required code accorindg to the latest version.

* Updated ruff version and fixed the required code accorindg to the latest version.

* Added noqa directive to ignore 1 error shown by ruff

Co-authored-by: Arthur Zucker <[email protected]>

…face#32160)

Fixed an if condition always evaluating to true.

fix

…eights in the layer (huggingface#32171)

* adds: extra_repr() to MambaRMSNorm to include the hidden size of the layer

* style fix with ruff:

…than the ones present at import time. (huggingface#32153)

* fix: default value reflects the runtime environment variables rather than the ones present at import time.

* Fix: Change `deterministic` to None by default; use env var if None

* Update qwen2.md

outdated description

* Update qwen2.md

amended

* Update qwen2.md

Update

* Update qwen2.md

fix wrong version code, now good to go

Remove conversation pipeline tests

* relaxed rope check

* lets also accept rope_type=None, defaulting to the original implementation

* type and rope_type can coexist

* let's not warn when someone is running a foward without cache + self.training

* more models

* fixup

fix resize when deepspeed

* Fix float8_e4m3fn in modeling_utils

* style

* fix

* comment

* support gguf fp16

* support gguf bf16 with pytorch

* add gguf f16 test

* remove bf16

* No more default chat templates

* Add the template to the GPT-SW3 tests since it's not available by default now

* Fix GPT2 test

* Fix Bloom test

* Fix Bloom test

* Remove default templates again

…ingface#32198)

Replaced deprecated unittest method with the correct one.

* [whisper] fix short-form output type

* add test

* make style

* update long-form tests

* fixes

* last fix

* finalise test

….7.0 (huggingface#32210)

remove unnecessary guard code related with pytorch versions 1.4.2 ~
1.7.0

…gingface#32222)

set _supports_param_buffer_assignment to False

fix E721 warnings

* fix

* [test_all] trigger full CI

---------

Co-authored-by: ydshieh <[email protected]>

* translate philosophy.md to chinese

* add the missing link

…tility functions. Default to using the currently active microphone on Mac (huggingface#31846)

* use currently active microphone on mac for ffmpeg_microphone

* Allow ffmpeg_microphone device to be specified

Co-authored-by: amyeroberts <[email protected]>

---------

Co-authored-by: amyeroberts <[email protected]>

Fix code snippet for grounding-dino

* fix

* move changes to prompt lookup

* add test

* set eos in assistant model

* style

* fix flakiness

* changes for new `main`

* Update tests/generation/test_utils.py

Co-authored-by: amyeroberts <[email protected]>

* Update tests/generation/test_utils.py

Co-authored-by: amyeroberts <[email protected]>

* add comment to explain

---------

Co-authored-by: amyeroberts <[email protected]>

* llava w/o images

* tests

* fix resize when deepspeed

* deepsped uses new embeds

* we needed this

…gingface#32143)

* don't log base model architecture in wandb is log model is false

* Update src/transformers/integrations/integration_utils.py

Co-authored-by: amyeroberts <[email protected]>

* convert log model setting into an enum

* fix formatting

---------

Co-authored-by: amyeroberts <[email protected]>

* Refactored to remove un-necessary object base class.

* small fix.

* adds: extra_repr() to RMSNorm layers in multiple models

* adds: extra_repr for deprecated models as well

* formatting as per style guide

…tection` for owlv2 (huggingface#31934)

* Add check for target_sizes is None in post_process_image_guided_detection

* Make sure Owlvit and Owlv2 in sync

* Fix incorrect indentation; add check for correct size of target_sizes

…n_implementation==flash_attention_2` (huggingface#32039)

* add flash attention check

* fix

* fix

…ngface#32241)

* fix

* fix prev test (half of failures)

* [run-slow] llama, gemma2

* [run-slow] llama, gemma2

update

Co-authored-by: ydshieh <[email protected]>

…uggingface#32244)

* replace for loop by tensor ops

* rm assert; readability

* bloom dynamic cache

* bloom follows standard cache format

* no skips for bloom anymore

* use cache position when possible

* clean up

* codestyle

* Update src/transformers/models/bloom/modeling_bloom.py

Co-authored-by: amyeroberts <[email protected]>

* Update src/transformers/models/bloom/modeling_bloom.py

Co-authored-by: amyeroberts <[email protected]>

* Update src/transformers/models/bloom/modeling_bloom.py

Co-authored-by: amyeroberts <[email protected]>

* pr comments

* isinstance fix

* address comments

* make musicgen test happy

* [run-slow] bloom

---------

Co-authored-by: amyeroberts <[email protected]>

upload

Co-authored-by: ydshieh <[email protected]>

* Optimize t5 tokenize logic to avoid redundant calls

* fix and overwrite copies

…n call (huggingface#32262)

Removed one wrong argument passed to convert_blip_checkpoint function call.

remove exceptions

…uggingface#32076)

* fix

* bug fix

* refine

* fix

Fixed a link in docs.

* mvp

* added test (a few models need fixes)

* fix a few test cases

* test nits

* harder test 😈

* revert changes in stablelm

* test with improved condition

* add todo

* tmp commit

* merged with main

* nits

* add todo

* final corrections

* add docs for generation compilation

* docs nits

* add  tip

* PR suggestions

* add more details to the compilation docs

* fix cache positions

* cache is now init in generate; update docs

* tag test as flaky

* docs

* post rebase make fixup and other nits

* remove unintended changes

* whisper (encoder-decoder) not supported

* move token default updates to ; add tests for token defaults

* push changes

* manual rebase

* chameleon doesn't support this

* fix test_static_cache_mha_mqa_gqa (broken in another PR)

* docs: dynamic is better with end-to-end compilation

* fix _fix_key in PreTrainedModel

* fix _find_longest_common_sequence

* add test

* remove result.json

* nit

* update test

* [pipeline] fix padding for 1-d tensors

* add test

* make style

* Update tests/pipelines/test_pipelines_automatic_speech_recognition.py

Co-authored-by: Kamil Akesbi <[email protected]>

* Update tests/pipelines/test_pipelines_automatic_speech_recognition.py

---------

Co-authored-by: Kamil Akesbi <[email protected]>

)

* Add stream_to_gradio method for running agent in gradio demo

Co-authored-by: ydshieh <[email protected]>

Docs: fix GaLore optimizer example

Fix incorrect usage of GaLore optimizer in Transformers trainer code example.

The GaLore optimizer uses low-rank gradient updates to reduce memory usage. GaLore is quite popular and is implemented by the authors in [https://github.com/jiaweizzhao/GaLore](https://github.com/jiaweizzhao/GaLore). A few months ago GaLore was added to the HuggingFace Transformers library in huggingface#29588.

Documentation of the Trainer module includes a few code examples of how to use GaLore. However, the `optim_targe_modules` argument to the `TrainingArguments` function is incorrect, as discussed in huggingface#29588 (comment). This pull request fixes this issue.

* fix gguf dequantize for gguf==0.9.1

* fix old version

* make style

* fix epochs_trained as int when resuming training

* refactor

---------

Co-authored-by: teddyferdinan <[email protected]>

…ce#31663)

* tentative fix

* do the same for M4T

* doc formatting nits

* ignore non-autodocs

* Apply suggestions from code review

Co-authored-by: amyeroberts <[email protected]>

* Update src/transformers/models/esm/modeling_esm.py

Co-authored-by: amyeroberts <[email protected]>

* Update src/transformers/models/esm/modeling_esm.py

Co-authored-by: amyeroberts <[email protected]>

* make fixup

---------

Co-authored-by: amyeroberts <[email protected]>

* new agent plan

* plan type assertion

* style corrections

* better prompt naming

* make fixup

Fixed raising of few exceptions.

…ading for prequantized 4bit (huggingface#32276)

…ge of 10% o… (huggingface#32335)

fixes huggingface#32329 : The Torch code is correct - to get an average of 10% of the total, we want to take 50% of the remainder after we've already masked 80% with [MASK] in the previous step.

)

* tmp

* skip files not in the diff

* use git.Repo instead of an external subprocess

* add tiny change to confirm that the diff is working on pushed changes

* add make quality task

* more profesh main commit reference

…ss (huggingface#32191)

* Remove user-defined tokens which can be obtained through merges

* Remove debug line

* formatting

* Refactor spm slow -> fast converter

* revert unnecessary refactor

* set comprehension

* remove test files

* Use `vocab_scores`

* Always replace spiece underline with space in decode

* we no longer need token filtering

* Add save fast load slow unit test

* Remove tokenizers version check

* Remove duplicate code

* Make `<start_of_turn>` and `<end_of_turn>` special tokens

* Bias merge priority with length if score is the same

* Add unit test for merge priority

* CI

fix

* enable flash-attn & static cache

* this works, not the prev

* fix for sliding window layers

* not needed anymore

fix 💩

* Fix FA2 call for Perciever layer

* [run_slow] idefics2

* [run_slow] idefics2

* [run_slow] idefics2

* Fix up

* [run_slow] idefics2

* [run_slow] idefics2

* [run_slow] idefics2

…ggingface#32360)

Fix error when streaming agent run to gradio with non-string tool arguments

…oder models (huggingface#32227)

* draft

* apply changes to all relevant archs

* rerun ci - check_docstrings.py failing?

* fix docstring

* move 2D->4D mask creation to modeling file

* repo consistency

* fix the batch size = 1 case - calling contiguous is not enough

* nit

* style

* propagate to gemma/gemma-2

* prepare inputs for gemma generation

* implement test and tiny fix in gemma2

* Update src/transformers/models/bloom/modeling_bloom.py

Co-authored-by: Arthur <[email protected]>

* fix copies

* ci pass

* fix gemma's test_compile_static_cache tests

* flacky

* retrigger ci

---------

Co-authored-by: sanchit-gandhi <[email protected]>
Co-authored-by: Arthur <[email protected]>

* Fixed staticmethods with self as first argument.

* Fixed staticmethods with self as first argument.

* Fixed staticmethods with self as first argument.

* Fixed staticmethods with self as first argument.

cache class flag

…31772)

* [whisper] compile compatibility with long-form decoding

* clarify comment

* fix after rebase

* finalise

* fix bsz

* fix cache split

* remove contiguous

* style

* finish

* update doc

* prevent cuda graph trace

…ngface#32339)

* Remove size check between attn_weights and kv_seq_len

* add unit tests

…uggingface#32359)

Co-authored-by: Guoming Zhang <[email protected]>

…ggingface#31971) (huggingface#32043)

* Remove TPU device map for saving tokenizer config

* Update tokenization_utils_base.py

* Fix error msg when passing non-string device into tokenizer

* Fix error message for non-string tokenizer device

* Print out tokenizer device type in error msg

* Update tokenization_utils_base.py

…sion (huggingface#32342)

empty list in defaults

…gface#31233)

* Fix conflicting key in init kwargs in PreTrainedTokenizerBase

* Update code to check for callable key in save_pretrained

* Apply PR suggestions

* Invoke CI

* Updates based on PR suggestion

* Initial implementation of OffloadedCache

* enable usage via cache_implementation

* Address feedback, add tests, remove legacy methods.

* Remove flash-attn, discover synchronization bugs, fix bugs

* Prevent usage in CPU only mode

* Add a section about offloaded KV cache to the docs

* Fix typos in docs

* Clarifications and better explanation of streams

…#32374)

* fixed hybrid cache init, added test

* Fix Test Typo

---------

Co-authored-by: Aaron Haag <[email protected]>

…e been done (huggingface#32299)

* Test this zach

* Test for improper init w/o zero3

* Move back

* Apply suggestions from code review

Co-authored-by: amyeroberts <[email protected]>

* Get rid of stars in warning

* Make private

* Make clear

---------

Co-authored-by: amyeroberts <[email protected]>

nits

tests! :D

…gingface#32367)

* up

* style

* stopping

…rmers/examples/flax/language-modeling/t5_tokenizer_model.py`. (huggingface#32157)

fix: Exception raised when running .

…ing rotary_seq_len, allowing None position_ids input. (huggingface#31500)

* Mixtral: remove unnecessary plus 1 when calculating rotary_seq_len, allowing position_ids=None (no auto position_ids generation could be unsafe)

* fix typo [:-1] to [:, -1]

* to meet formatting requirement

* to meet formatting requirement

* remove white space

* MixtralFlashAttention2: put "+ 1" inside parentheses when calculating rotary_seq_len, allowing None position_ids input. Fix format/style issue.

* propagate to startcoder2, phi3, mixtral and qwen2

* update qwen2_moe