Can the model google/flan-t5-base be imported despite error?

[kwalcock]

I am completely new to this and the response to my request

djl-import -m google/flan-t5-base

is this error message:

Unsupported model architecture: T5ForConditionalGeneration for google/flan-t5-base. model not found: Namespace(limit=1, output_dir='.', output_format='PyTorch', retry_failed=False, cpu_only=False, optimize=None, device=None, dtype=None, category=None, model_name='google/flan-t5-base', trust_remote_code=False, min_version=None) finished.

Does that just mean it can't be done or is there some trick like first converting to onnx or something else that can help? I'm trying to convert a Python program to Java and it uses this model.

[kwalcock]

On the other hand, the conversion

djl-import -m sentence-transformers/all-MiniLM-L6-v2

had gone fine, so I know that something is working.

[frankfliu]

@kwalcock
flan-t5 is text-generation model which djl-convert doesn't support at the moment, see: https://docs.djl.ai/master/extensions/tokenizers/index.html#use-djl-huggingface-model-converter

If you want to serve flan-t5 model, you can use djl-serving to serve this model, I suggest you use our LMI container to deploy this model: https://docs.djl.ai/master/docs/serving/serving/docs/lmi/index.html

[kwalcock]

@frankfliu, thank you. We have a homemade server that just uses the native Python implementation. I'll watch for updates.