Initial support for attachments for multi-modal models

[simonw]

Refs:

• Solve multi-modal models with a new concept of "attachments" #587

[simonw]

Getting a bunch of mypy failures:

[simonw]

% just mypy
llm/models.py:87: error: Incompatible types in assignment (expression has type "dict[Never, Never]", variable has type "Options")  [assignment]
llm/models.py:118: error: Incompatible default for argument "attachments" (default has type "None", argument has type "Attachment")  [assignment]
llm/models.py:118: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
llm/models.py:118: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
llm/models.py:374: error: Need type annotation for "attachment_types" (hint: "attachment_types: set[<type>] = ...")  [var-annotated]
llm/models.py:400: error: Incompatible default for argument "attachments" (default has type "None", argument has type "Attachment")  [assignment]
llm/models.py:400: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
llm/models.py:400: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
llm/models.py:410: error: Item "Attachment" of "Attachment | list[Never]" has no attribute "__iter__" (not iterable)  [union-attr]
Found 5 errors in 1 file (checked 15 source files)
error: Recipe `mypy` failed on line 28 with exit code 1

[simonw]

Tried this:

files-to-prompt . -c -e py | llm -m claude-3.5-sonnet --system 'llm/models.py:87: error: Incompatible types in assignment (expression has type "dict[Never, Never]", variable has type "Options")  [assignment]
llm/models.py:118: error: Incompatible default for argument "attachments" (default has type "None", argument has type "Attachment")  [assignment]
llm/models.py:118: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
llm/models.py:118: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
llm/models.py:374: error: Need type annotation for "attachment_types" (hint: "attachment_types: set[<type>] = ...")  [var-annotated]
llm/models.py:400: error: Incompatible default for argument "attachments" (default has type "None", argument has type "Attachment")  [assignment]
llm/models.py:400: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
llm/models.py:400: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
llm/models.py:410: error: Item "Attachment" of "Attachment | list[Never]" has no attribute "__iter__" (not iterable)  [union-attr]'

Based on the code, I'll help fix the mypy errors in llm/models.py. Here are the fixes:

1. For the incompatible types in assignment error:

# Change this:
self.options = options or {}

# To this:
if options is None:
    self.options = self.Options()
else:
    self.options = options

2. For the attachment type issues, add type annotations and Optional:

from typing import Optional, List

class Prompt:
    def __init__(
        self,
        prompt: str,
        model: "Model",
        *,
        attachments: Optional[List[Attachment]] = None,
        system: Optional[str] = None,
        prompt_json: Optional[str] = None, 
        options: Optional[Dict[str, Any]] = None
    ):
        self.prompt = prompt
        self.model = model
        self.attachments = list(attachments or [])
        self.system = system
        self.prompt_json = prompt_json
        if options is None:
            self.options = self.Options() 
        else:
            self.options = options

3. For the attachment_types error:

class Model(ABC, _get_key_mixin):
    # Add type annotation
    attachment_types: Set[str] = set()

4. For the union attribute error when checking attachments:

def prompt(
    self,
    prompt: str,
    *,
    attachments: Optional[List[Attachment]] = None,
    system: Optional[str] = None,
    stream: bool = True,
    **options
):
    # Validate attachments
    if attachments and not self.attachment_types:
        raise ValueError(
            "This model does not support attachments, but some were provided"
        ) 
    
    # Use proper list typing
    attachments_list = list(attachments or [])
    
    for attachment in attachments_list:
        attachment_type = attachment.resolve_type()
        if attachment_type not in self.attachment_types:
            raise ValueError(
                f"This model does not support attachments of type '{attachment_type}', only {', '.join(self.attachment_types)}"
            )

Key changes made:

1. Added proper type annotations for Options and attachments arguments
2. Fixed initialization of self.options to use proper Options type
3. Added Set[str] type annotation for attachment_types class variable
4. Fixed attachments handling to properly handle Optional[List[Attachment]]
5. Added proper type hints for method arguments

This should resolve all the mypy type checking errors while maintaining the existing functionality.

Let me know if you would like me to explain any of the changes in more detail!