Skip to content

Commit

Permalink
Cache: revert DynamicCache init for BC (#33861)
Browse files Browse the repository at this point in the history
* tmp commit

* tmp commit

* make fixup

* missing removal

* fix condition

* fix end-to-end compilation

* if -> elif

* BC

* BC

* use @deprecate_kwarg("num_hidden_layers", version="4.47.0")

* wups the import

* 🥴

---------

Co-authored-by: Arthur Zucker <[email protected]>
  • Loading branch information
gante and ArthurZucker committed Oct 7, 2024
1 parent 5df4ca8 commit 2fd49d2
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion tests/generation/test_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -1849,11 +1849,18 @@ def test_new_cache_format(self, num_beams, do_sample):
if config.is_encoder_decoder:
cache_cls = EncoderDecoderCache
past_key_values = cache_cls(DynamicCache(), DynamicCache())
past_key_values = cache_cls(DynamicCache(), DynamicCache())
else:
cache_cls = DynamicCache
past_key_values = cache_cls()

new_results = model.generate(past_key_values=past_key_values, **generation_kwargs, **inputs_dict)
new_results = model.generate(
input_ids,
attention_mask=attention_mask,
past_key_values=past_key_values,
**generation_kwargs,
**inputs_dict,
)

# The two sets of generated sequences must match, despite the cache format between forward passes being
# different
Expand Down

0 comments on commit 2fd49d2

Please sign in to comment.