Target prefix-constrained generation

[gsarti]

Description

This PR introduces a generate_from_target_prefix argument in the model.attribute method and the inseq attribute CLI, allowing the usage of pre-specified generated_text as a prefix for the model to complete, rather than a finished generation.

This functionality exploits the capabilities of the generate function in 🤗 transformers, and is restricted to encoder-decoder models only, since controlling the prefix is handled through the input_text parameter for decoder-only models. The same behavior could already be achieved in inseq v0.4 by providing explicit decoder_input_ids as generation_args.

Current behavior:

>>> import inseq

>>> m = inseq.load_model("Helsinki-NLP/opus-mt-en-it", "saliency")
>>> prefix_ids = m.encode("Hey mondo!", as_targets=True)
>>> out = model.attribute(
...	   "Hello world! My name is Inseq",
...	   generation_args={"decoder_input_ids": prefix_ids.input_ids}
...)
>>> out.info["generated_texts"]
['Hey mondo! mi chiamo Inseq.']

New behavior using generate_from_target_prefix:

>>> import inseq

>>> m = inseq.load_model("Helsinki-NLP/opus-mt-en-it", "saliency")
>>> out = model.attribute(
...    "Hello world! My name is Inseq",
...    "Hey mondo!",
...    generate_from_target_prefix=True
... )
>>> out.info["generated_texts"]
['Hey mondo! mi chiamo Inseq.']

The new argument also works with batched inputs. Setting the target prefix to an empty string will perform the fully unconstrained generation, as if no prefix was provided:

>>> import inseq

>>> model = inseq.load_model("Helsinki-NLP/opus-mt-en-it", "saliency")
>>> src = ["Hello world! My name is Inseq", "I love bugging around code all day"]
>>> prefix = ["Hey mondo!", ""]
>>> out = model.attribute(src, prefix, generate_from_target_prefix=True)
>>> out.info["generated_texts"]
... ['Hey mondo! mi chiamo Inseq', 'Adoro le cimici intorno al codice tutto il giorno']

Type of Change

• 🚀 New feature (non-breaking change which adds functionality)