Does the app support Chinise?

[weicun581]

The feature

thanks for the product, but as my english is poor, Does it will support Chinise?

Platform

• Server
• Web
• Mobile

[Pheggas]

You can help Immich to translate it to Chinese by contacting @alextran1502 on discord or by mail. He will send you invitation to the translation service and you can start translating

[GJCav]

I admire this project and its smart search function. I am personally comfortable with English but my parents are not good at English and they want to search in Chinese. However, searching in Chinese is not supported correctly. Take the following example:

"昆虫" means insects in English. I searched in Chinese on the left and in English on the right part. As the picture illustrates, Chinese searching gave the wrong answer while English searching is working perfectly.

I would appreciate it if the project would improve the search to fully support Chinese.

[alextran1502]

Hello, you will have to change to CLIP model for the machine learning to understand different language. Available model can be found in a link in machine learning settings from the administration page

[chain710]

Found no chinese language model in that link

I'm no expert in machine learning, but I tried the Chinese-CLIP model, and unfortunately, I had no success. I followed these steps:

1. Converted the ckpt to onnx (both img and txt) as per their deployment documentation.
2. Replaced both the texture.onnx and visual.onnx files in the model cache directory.
3. Restarted the machine learning container.
4. Re-run ENCODE CLIP, but got lots of errors, primarily: ValueError: Required inputs (['image']) are missing from input feed (['pixel_values']).

I suspect there might be a mismatch in the model files? It would be great if someone could identify the errors.

[chain710]

Found this PR
Tried model CN-CLIP/ViT-B-16 in admin page, but got error ValueError: Unknown model name CN-CLIP/ViT-B-16

[Tea-NT]

有解决吗？大佬

[weicun581]

没有，哈哈哈，好久了，我都忘记了。

[nodis]

大佬，你怎么解决的？我同样需要

[Tea-NT]

Found this PR Tried model CN-CLIP/ViT-B-16 in admin page, but got error ValueError: Unknown model name CN-CLIP/ViT-B-16

老哥，有解决么？我后面想的用国内一个叫MT Photo的，感觉框架都是用immich做的，但MT Photo不支持全景照片，无奈了。要支持估计得整个重写后端对接的AI识别服务。
I find an other photo management software which call "MT Photos", but MT Photos need pay subscirption service for ￥3 a month, ￥25 a year ,￥99 for no limit. free try a month, I install and try it. MT Photos full support Chinese clip, and OCR with picture, but not support 360 degree image.

[Tea-NT]

Found no chinese language model in that link

I'm no expert in machine learning, but I tried the Chinese-CLIP model, and unfortunately, I had no success. I followed these steps:

1. Converted the ckpt to onnx (both img and txt) as per their deployment documentation.
2. Replaced both the texture.onnx and visual.onnx files in the model cache directory.
3. Restarted the machine learning container.
4. Re-run ENCODE CLIP, but got lots of errors, primarily: ValueError: Required inputs (['image']) are missing from input feed (['pixel_values']).

I suspect there might be a mismatch in the model files? It would be great if someone could identify the errors.

Found no chinese language model in that link

I'm no expert in machine learning, but I tried the Chinese-CLIP model, and unfortunately, I had no success. I followed these steps:

1. Converted the ckpt to onnx (both img and txt) as per their deployment documentation.
2. Replaced both the texture.onnx and visual.onnx files in the model cache directory.
3. Restarted the machine learning container.
4. Re-run ENCODE CLIP, but got lots of errors, primarily: ValueError: Required inputs (['image']) are missing from input feed (['pixel_values']).

I suspect there might be a mismatch in the model files? It would be great if someone could identify the errors.

我看了下源码，只支持MCLIP 和OpenCLIP 的两类模型，不然加载 就会报错 ValueError: Unknown clip model ViT-B-31__openai，我用Postman直接调用immich_machine_leaning的webapi接口，传入了中文的模型，中文模型改了个名字，出来的结果是这样的。

调用加载模型的源码是：

from typing import Any

from app.schemas import ModelType

from .base import InferenceModel
from .clip import MCLIPEncoder, OpenCLIPEncoder
from .constants import is_insightface, is_mclip, is_openclip
from .facial_recognition import FaceRecognizer


def from_model_type(model_type: ModelType, model_name: str, **model_kwargs: Any) -> InferenceModel:
    match model_type:
        case ModelType.CLIP:
            if is_openclip(model_name):
                return OpenCLIPEncoder(model_name, **model_kwargs)
            elif is_mclip(model_name):
                return MCLIPEncoder(model_name, **model_kwargs)
        case ModelType.FACIAL_RECOGNITION:
            if is_insightface(model_name):
                return FaceRecognizer(model_name, **model_kwargs)
        case _:
            raise ValueError(f"Unknown model type {model_type}")

    raise ValueError(f"Unknown {model_type} model {model_name}")

immich_machine_learning service only support mclip or openclip. so replace model file is not work.

[si9ma]

I tried to change it to support cn clip, you can pull and try if you are interested https://github.com/si9ma/immich
我尝试改了一下，有兴趣的可以pull了试一下，效果还可以 https://github.com/si9ma/immich @Tea-NT @nodis @chain710

[isolierte]

@Tea-NT 而你，和@si9ma ，我的兄弟，是真正的英雄！！
@Tea-NT And you, and @si9ma, are the real heroes! !Thanks.

[Tea-NT]

@isolierte i build cuda image and tested today, it look like work fine.
here is cuda support image ml-cn-clip-cuda

Cuda镜像也builld了一份，今天测试了一下应该是没问题的。

[Tea-NT]

@GJCav @weicun581 @chain710 @nodis thanks to @si9ma 's fork, I build cpu and cuda ML-cn-clip image, test work fine.

[nodis]

@Tea-NT 感谢各位大佬

[KyQiao]

@Tea-NT 在105上测试正常，感谢。

[Tea-NT]

@isolierte It WORK!!!

I clone @si9ma 's fork and build image, this fork add two columns in PostgresSQL database whit table exif , district and address,
and geocode support amap(not try, look like need API key AMAP_GEOCODE_KEYS).
I trying push the image now!

[isolierte]

Thank you and @si9ma for all you do.

[Tea-NT]

@isolierte I pushed the image, database need add two column in exif table, district and address both varchar.

cn-clip-machine-learning

刚刚提交了machine-learning镜像，immich-server镜像也提交了，如果server也用@si9ma的版本，需要在对应immich使用的PostgreSQL数据库中exif 表,，增加 district 和 address两个字段，否则会报错。

[si9ma]

@Tea-NT @isolierte 这个fork中主要根据我的需求做了一些修改，但工作繁忙，暂时没空往官方仓库中提交pr，有需要的可以自行使用，后续也会持续合并主干中的修改进来。具体可以看我的fork中的readme. https://github.com/si9ma/immich

This fork mainly makes some modifications according to my needs, but I'm busy with work and don't have time to submit a PR to the official repository for now. Those in need can use it themselves, and I will continue to merge modifications from the main branch in the future. You can see the details in the readme of my fork. https://github.com/si9ma/immich

[Tea-NT]

@si9ma 非常感谢！！！

[Gxyloj]

N5105的NAS通过Docker compose部署，Job中的任务正常进行但是无法调用核显，是还需要额外设置什么参数吗。用的是immich-machine-learning-cn-clip-cpu @Tea-NT

[Tea-NT]

@Gxyloj sorry. 搞错类型了。>...<

应该是 self.cache_dir =Path( '/cache/cn-clip') 想当然以为是 字符串了。

[Gxyloj]

@Tea-NT 可以正常识别并调用核显了，非常感谢

[Gxyloj]

[Tea-NT]

@Gxyloj 我这边整理了下代码，还有一些peotry里的依赖也更新了，push了最新的1.102.3的版本的镜像。因为我这边没有英特尔核显，没法测试，现在运行正常了 ，不支持核显就直接用cpu计算了。

docker pull vpngates5/immich-machine-learning-cn-clip-openvino:1.102.3

[Tea-NT]

@Gxyloj 非常好，那应该就是那个变量的原因。如果有空可以再测试下刚推的1.102.3，之前的版本是基于1.100的，有些依赖没更新，也应该是没加那行调用cache_dir的代码。

PS:你的NAS界面也蛮好看的也，是群晖吗？

[ThreeAurora]

感谢大佬的付出，这个应用潜力无限

[ThreeAurora]

关于中文模型的问题，我想我必须在这里说一下，给有需要的人。

在官方说明中，这里有一个问答，简单提到了换模型的办法。

而immich支持的多语言模型，是支持100多种语言的，其中就包含了中文，immich支持的模型都在下载这里下载。

而多语言模型中，我最推荐的是性能超强的XLM-Roberta-Large-Vit-B-16Plus。

这个模型不仅支持100多种语言，而在中文甚至包括英文的结果，都超过了原版的CLIP。这是我从该模型的对比数据中看的，应该是有可信度的。

如果有人还在关注这个事情，可以尝试一下，我今天正打算换。

[fenghuan1994]

最终你们都换的什么模型啊？无法访问模型列表网站。看不到信息。。。求告知怎么填写模型名字
十分感谢

[hsu1943]

最终你们都换的什么模型啊？无法访问模型列表网站。看不到信息。。。求告知怎么填写模型名字 十分感谢

就是用的这个模型：XLM-Roberta-Large-Vit-B-16Plus

[fenghuan1994]

谢谢，已经在用了。就是我的配置好像无法调用核显。

[VvLin-61]

谢谢，已经在用了。就是我的配置好像无法调用核显。

我也是，12300T，核显一直是0，不知道为什么

[fenghuan1994]

谢谢，已经在用了。就是我的配置好像无法调用核显。

我也是，12300T，核显一直是0，不知道为什么

看了官方指导，好像模型就是无法单独调用核显的，只有视频转码可以。所以我已经不纠结了

[ThreeAurora]

另外中文方面的对比详情在这里，Validation & Training Curves。

如果可以的话，就不必再用CN-CLIP。

如果我理解有误，或者该模型与CN-CLIP性能差距较大，请忽略。

[nodis]

如果下载不了模型的话，可以去网盘下载，我打包上传到网盘了
immich的CLIP模型model-cache
immich的model-cache.zip
下载链接：
https://www.123pan.com/s/WXqA-EGL6d.html

如果你的网络拉取这几个模型比较费劲，可以去下载，下载之后解压上传到模型目录即可

Smart Search用的模型是：immich-app/XLM-Roberta-Large-Vit-B-16Plus（支持中文，体验很好，默认的ViT-B-32__openai不支持中文）
Facial Recognition用的模型是：buffalo_l，另外在上面压缩包里加入了antelopev2模型（据说这个模型更好，不过我没切换过去）

[RickyGu777]

感谢大佬

[zhpjy]

下载不了模型的解决办法

将 HF_ENDPOINT 添加到您的 env 文件中，
在 env 里添加HF_ENDPOINT=https://hf-mirror.com

来源：#6417 (comment)

[dddd64693]

请教一下，用模型的话，yml文件中这个地方需要修改吗，y要不要指定本地路径

[ThreeAurora]

我刚才看到有人问我，我在这儿提醒一次，网络可以的话不需要去网站里下载模型。

如果要换的话，直接在immich网页的设置页面中，填入 XLM-Roberta-Large-Vit-B-16Plus ，是的，粘贴就可以了！下载等步骤，它会自动进行的。

[fenghuan1994]

嗯嗯。感谢解答。

[Arnie97]

直装成功了，记录下供后人参考：直装走的 CloudFront CDN，在部分地区会被 SNI TCP reset 干扰，需保障immich_machine_learning容器的网络情况。若手动下载，容器内预期的目录结构类似/cache/clip/XLM-Roberta-Large-Vit-B-16Plus/README.md和/cache/facial-recognition/buffalo_l/README.md（其中/cache目录在官方compose.yml中定义为卷model-cache）。内存比较友好，4G 可用 8G 宽裕，但对 CPU 要求比较高，能不能硬件加速未知，还没研究。

[dddd64693]

试了一下XLM-Roberta-Large-Vit-B-16Plus这个模型，中文搜图很多都识别不到，不知道问题出在哪，immich_machine_learning已经正常工作了

[Arnie97]

我偶尔遇到过一次中文搜不到任何内容，改用英文就能找到的情况，当时稳定复现，但是现在忘记复现路径了

[deadash]

https://github.com/deadash/immich_ai

我使用rust写了一个独立的immich的机器识别程序(目前只支持clip)，使用的是chinese_clip来识别，并且支持tensor_rt/cuda/dml

可以使用英特尔的显卡来加速(N100可用)，可以用官方的仓库然后设置服务器地址到这个程序后可以成功识别。

模型下载使用的魔塔，国内也可以高速下载。