Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(container): update quay.io/go-skynet/local-ai ( v2.21.1 → v2.22.0 ) #7404

Merged
merged 1 commit into from
Oct 13, 2024
Merged

feat(container): update quay.io/go-skynet/local-ai ( v2.21.1 → v2.22.0 ) #7404

merged 1 commit into from
Oct 13, 2024

Conversation

rosey-the-renovator-bot[bot]
Copy link
Contributor

This PR contains the following updates:

Package Update Change
quay.io/go-skynet/local-ai minor v2.21.1-aio-gpu-nvidia-cuda-12 -> v2.22.0-aio-gpu-nvidia-cuda-12

Release Notes

mudler/LocalAI (quay.io/go-skynet/local-ai)

v2.22.0

Compare Source

LocalAI v2.22.0 is out 🥳

💡 Highlights

  • Image-to-Text and Video-to-Text Support: The VLLM backend now supports both image-to-text and video-to-text processing.
  • Enhanced Multimodal Support: Template placeholders are now available, offering more flexibility in multimodal applications
  • Model Management Made Easy: List all your loaded models directly via the /system endpoint for seamless management.
  • Various bugfixes and improvements: Fixed issues with dangling processes to ensure proper resource management and resolved channel closure issues in the base GRPC server.
🖼️ Multimodal vLLM

To use multimodal models with vLLM simply specify the model in the YAML file. Models however can differ if support multiple images or single images, along how they process internally placeholders for images.

Some models/libraries have different way to express images, videos or audio placeholders. For example, llama.cpp backend expects images within an [img-ID] tag, but other backends/models (e.g. vLLM) use a different notation ( <|image_|>).

For example, to override defaults, now it is possible to set in the model configuration the following:

template:
  video: "<|video_{{.ID}}|> {{.Text}}"
  image: "<|image_{{.ID}}|> {{.Text}}"
  audio: "<|audio_{{.ID}}|> {{.Text}}"
📹 Video and Audio understanding

Some libraries might support both Video and Audio. Currently only vLLM supports Video understanding, and can be used in the API by "extending" the OpenAI API with audio and video type along images:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "What'\''s in this video?"
          },
          {
            "type": "video_url",
            "video_url": {
              "url": "https://video-image-url"
            }
          }
        ]
      }
    ],
    "max_tokens": 300
  }'
🧑‍🏭 Work in progress

What's Changed

Bug fixes 🐛
Exciting New Features 🎉
🧠 Models
📖 Documentation and examples
👒 Dependencies
Other Changes

New Contributors

Full Changelog: mudler/LocalAI@v2.21.1...v2.22.0

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

  • If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

@rosey-the-renovator-bot
feat(container): update quay.io/go-skynet/local-ai ( v2.21.1 → v2.22.0 )
8ddfcce 
| datasource | package                    | from    | to      |
| ---------- | -------------------------- | ------- | ------- |
| docker     | quay.io/go-skynet/local-ai | v2.21.1 | v2.22.0 |
@rosey-the-renovator-bot rosey-the-renovator-bot bot merged commit 51eea05 into main Oct 13, 2024
18 checks passed
@rosey-the-renovator-bot rosey-the-renovator-bot bot deleted the renovate/quay.io-go-skynet-local-ai-2.x branch October 13, 2024 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/kubernetes cluster/main renovate/container type/minor
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
0 participants