+ alt="follow on X(Twitter)">
@@ -196,10 +196,14 @@ If you'd like to configure a highly-available setup, there are community-contrib
#### Using Terraform for Deployment
+Deploy Dify to Cloud Platform with a single click using [terraform](https://www.terraform.io/)
+
##### Azure Global
-Deploy Dify to Azure with a single click using [terraform](https://www.terraform.io/).
- [Azure Terraform by @nikawang](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [Google Cloud Terraform by @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
+
## Contributing
For those who'd like to contribute code, see our [Contribution Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
@@ -219,7 +223,7 @@ At the same time, please consider supporting Dify by sharing it on social media
* [Github Discussion](https://github.com/langgenius/dify/discussions). Best for: sharing feedback and asking questions.
* [GitHub Issues](https://github.com/langgenius/dify/issues). Best for: bugs you encounter using Dify.AI, and feature proposals. See our [Contribution Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
* [Discord](https://discord.gg/FngNHpbcY7). Best for: sharing your applications and hanging out with the community.
-* [Twitter](https://twitter.com/dify_ai). Best for: sharing your applications and hanging out with the community.
+* [X(Twitter)](https://twitter.com/dify_ai). Best for: sharing your applications and hanging out with the community.
## Star history
diff --git a/README_AR.md b/README_AR.md
index 10d572cc49a83..e46ba7373840c 100644
--- a/README_AR.md
+++ b/README_AR.md
@@ -17,7 +17,7 @@
alt="chat on Discord">
+ alt="follow on X(Twitter)">
@@ -179,10 +179,13 @@ docker compose up -d
#### استخدام Terraform للتوزيع
+انشر Dify إلى منصة السحابة بنقرة واحدة باستخدام [terraform](https://www.terraform.io/)
+
##### Azure Global
-استخدم [terraform](https://www.terraform.io/) لنشر Dify على Azure بنقرة واحدة.
- [Azure Terraform بواسطة @nikawang](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [Google Cloud Terraform بواسطة @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
## المساهمة
diff --git a/README_CN.md b/README_CN.md
index 32551fcc31393..4553524ce53b5 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -17,7 +17,7 @@
alt="chat on Discord">
+ alt="follow on X(Twitter)">
@@ -202,10 +202,14 @@ docker compose up -d
#### 使用 Terraform 部署
+使用 [terraform](https://www.terraform.io/) 一键将 Dify 部署到云平台
+
##### Azure Global
-使用 [terraform](https://www.terraform.io/) 一键部署 Dify 到 Azure。
- [Azure Terraform by @nikawang](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [Google Cloud Terraform by @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
+
## Star History
[](https://star-history.com/#langgenius/dify&Date)
@@ -232,7 +236,7 @@ docker compose up -d
- [GitHub Issues](https://github.com/langgenius/dify/issues)。👉：使用 Dify.AI 时遇到的错误和问题，请参阅[贡献指南](CONTRIBUTING.md)。
- [电子邮件支持](mailto:hello@dify.ai?subject=[GitHub]Questions%20About%20Dify)。👉：关于使用 Dify.AI 的问题。
- [Discord](https://discord.gg/FngNHpbcY7)。👉：分享您的应用程序并与社区交流。
-- [Twitter](https://twitter.com/dify_ai)。👉：分享您的应用程序并与社区交流。
+- [X(Twitter)](https://twitter.com/dify_ai)。👉：分享您的应用程序并与社区交流。
- [商业许可](mailto:business@dify.ai?subject=[GitHub]Business%20License%20Inquiry)。👉：有关商业用途许可 Dify.AI 的商业咨询。
- [微信]() 👉：扫描下方二维码，添加微信好友，备注 Dify，我们将邀请您加入 Dify 社区。
diff --git a/README_ES.md b/README_ES.md
index 2ae044b32883b..7da5ac7b61fb1 100644
--- a/README_ES.md
+++ b/README_ES.md
@@ -17,7 +17,7 @@
alt="chat en Discord">
+ alt="seguir en X(Twitter)">
@@ -204,10 +204,13 @@ Si desea configurar una configuración de alta disponibilidad, la comunidad prop
#### Uso de Terraform para el despliegue
+Despliega Dify en una plataforma en la nube con un solo clic utilizando [terraform](https://www.terraform.io/)
+
##### Azure Global
-Utiliza [terraform](https://www.terraform.io/) para desplegar Dify en Azure con un solo clic.
- [Azure Terraform por @nikawang](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [Google Cloud Terraform por @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
## Contribuir
@@ -228,7 +231,7 @@ Al mismo tiempo, considera apoyar a Dify compartiéndolo en redes sociales y en
* [Discusión en GitHub](https://github.com/langgenius/dify/discussions). Lo mejor para: compartir comentarios y hacer preguntas.
* [Reporte de problemas en GitHub](https://github.com/langgenius/dify/issues). Lo mejor para: errores que encuentres usando Dify.AI y propuestas de características. Consulta nuestra [Guía de contribución](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
* [Discord](https://discord.gg/FngNHpbcY7). Lo mejor para: compartir tus aplicaciones y pasar el rato con la comunidad.
-* [Twitter](https://twitter.com/dify_ai). Lo mejor para: compartir tus aplicaciones y pasar el rato con la comunidad.
+* [X(Twitter)](https://twitter.com/dify_ai). Lo mejor para: compartir tus aplicaciones y pasar el rato con la comunidad.
## Historial de Estrellas
diff --git a/README_FR.md b/README_FR.md
index 681d596749c9e..15f6f2650f8f2 100644
--- a/README_FR.md
+++ b/README_FR.md
@@ -17,7 +17,7 @@
alt="chat sur Discord">
+ alt="suivre sur X(Twitter)">
@@ -202,10 +202,13 @@ Si vous souhaitez configurer une configuration haute disponibilité, la communau
#### Utilisation de Terraform pour le déploiement
+Déployez Dify sur une plateforme cloud en un clic en utilisant [terraform](https://www.terraform.io/)
+
##### Azure Global
-Utilisez [terraform](https://www.terraform.io/) pour déployer Dify sur Azure en un clic.
- [Azure Terraform par @nikawang](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [Google Cloud Terraform par @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
## Contribuer
@@ -226,7 +229,7 @@ Dans le même temps, veuillez envisager de soutenir Dify en le partageant sur le
* [Discussion GitHub](https://github.com/langgenius/dify/discussions). Meilleur pour: partager des commentaires et poser des questions.
* [Problèmes GitHub](https://github.com/langgenius/dify/issues). Meilleur pour: les bogues que vous rencontrez en utilisant Dify.AI et les propositions de fonctionnalités. Consultez notre [Guide de contribution](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
* [Discord](https://discord.gg/FngNHpbcY7). Meilleur pour: partager vos applications et passer du temps avec la communauté.
-* [Twitter](https://twitter.com/dify_ai). Meilleur pour: partager vos applications et passer du temps avec la communauté.
+* [X(Twitter)](https://twitter.com/dify_ai). Meilleur pour: partager vos applications et passer du temps avec la communauté.
## Historique des étoiles
diff --git a/README_JA.md b/README_JA.md
index e6a8621e7baae..a2e6b173f5bfc 100644
--- a/README_JA.md
+++ b/README_JA.md
@@ -17,7 +17,7 @@
alt="Discordでチャット">
+ alt="X(Twitter)でフォロー">
@@ -68,7 +68,7 @@ DifyはオープンソースのLLMアプリケーション開発プラットフ
プロンプトの作成、モデルパフォーマンスの比較が行え、チャットベースのアプリに音声合成などの機能も追加できます。
**4. RAGパイプライン**:
- ドキュメントの取り込みから検索までをカバーする広範なRAG機能ができます。ほかにもPDF、PPT、その他の一般的なドキュメントフォーマットからのテキスト抽出のサーポイントも提供します。
+ ドキュメントの取り込みから検索までをカバーする広範なRAG機能ができます。ほかにもPDF、PPT、その他の一般的なドキュメントフォーマットからのテキスト抽出のサポートも提供します。
**5. エージェント機能**:
LLM Function CallingやReActに基づくエージェントの定義が可能で、AIエージェント用のプリビルトまたはカスタムツールを追加できます。Difyには、Google検索、DALL·E、Stable Diffusion、WolframAlphaなどのAIエージェント用の50以上の組み込みツールが提供します。
@@ -201,10 +201,13 @@ docker compose up -d
#### Terraformを使用したデプロイ
+[terraform](https://www.terraform.io/) を使用して、ワンクリックでDifyをクラウドプラットフォームにデプロイします
+
##### Azure Global
-[terraform](https://www.terraform.io/) を使用して、AzureにDifyをワンクリックでデプロイします。
-- [nikawangのAzure Terraform](https://github.com/nikawang/dify-azure-terraform)
+- [@nikawangによるAzure Terraform](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [@sotazumによるGoogle Cloud Terraform](https://github.com/DeNA/dify-google-cloud-terraform)
## 貢献
@@ -225,7 +228,7 @@ docker compose up -d
* [Github Discussion](https://github.com/langgenius/dify/discussions). 主に: フィードバックの共有や質問。
* [GitHub Issues](https://github.com/langgenius/dify/issues). 主に: Dify.AIを使用する際に発生するエラーや問題については、[貢献ガイド](CONTRIBUTING_JA.md)を参照してください
* [Discord](https://discord.gg/FngNHpbcY7). 主に: アプリケーションの共有やコミュニティとの交流。
-* [Twitter](https://twitter.com/dify_ai). 主に: アプリケーションの共有やコミュニティとの交流。
+* [X(Twitter)](https://twitter.com/dify_ai). 主に: アプリケーションの共有やコミュニティとの交流。
diff --git a/README_KL.md b/README_KL.md
index 04620d42bbec8..8f2affdce5ae5 100644
--- a/README_KL.md
+++ b/README_KL.md
@@ -17,7 +17,7 @@
alt="chat on Discord">
+ alt="follow on X(Twitter)">
@@ -202,10 +202,13 @@ If you'd like to configure a highly-available setup, there are community-contrib
#### Terraform atorlugu pilersitsineq
+wa'logh nIqHom neH ghun deployment toy'wI' [terraform](https://www.terraform.io/) lo'laH.
+
##### Azure Global
-Atoruk [terraform](https://www.terraform.io/) Dify-mik Azure-mut ataatsikkut ikkussuilluarlugu.
-- [Azure Terraform atorlugu @nikawang](https://github.com/nikawang/dify-azure-terraform)
+- [Azure Terraform mung @nikawang](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [Google Cloud Terraform qachlot @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
## Contributing
@@ -228,7 +231,7 @@ At the same time, please consider supporting Dify by sharing it on social media
). Best for: sharing feedback and asking questions.
* [GitHub Issues](https://github.com/langgenius/dify/issues). Best for: bugs you encounter using Dify.AI, and feature proposals. See our [Contribution Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md).
* [Discord](https://discord.gg/FngNHpbcY7). Best for: sharing your applications and hanging out with the community.
-* [Twitter](https://twitter.com/dify_ai). Best for: sharing your applications and hanging out with the community.
+* [X(Twitter)](https://twitter.com/dify_ai). Best for: sharing your applications and hanging out with the community.
## Star History
diff --git a/README_KR.md b/README_KR.md
index a5f3bc68d04d7..6c3a9ed7f6ea0 100644
--- a/README_KR.md
+++ b/README_KR.md
@@ -17,7 +17,7 @@
alt="chat on Discord">
+ alt="follow on X(Twitter)">
@@ -39,7 +39,6 @@
-
@@ -195,10 +194,14 @@ Dify를 Kubernetes에 배포하고 프리미엄 스케일링 설정을 구성했
#### Terraform을 사용한 배포
+[terraform](https://www.terraform.io/)을 사용하여 단 한 번의 클릭으로 Dify를 클라우드 플랫폼에 배포하십시오
+
##### Azure Global
-[terraform](https://www.terraform.io/)을 사용하여 Azure에 Dify를 원클릭으로 배포하세요.
- [nikawang의 Azure Terraform](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [sotazum의 Google Cloud Terraform](https://github.com/DeNA/dify-google-cloud-terraform)
+
## 기여
코드에 기여하고 싶은 분들은 [기여 가이드](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md)를 참조하세요.
diff --git a/README_TR.md b/README_TR.md
index 54b6db3f82371..a75889e576063 100644
--- a/README_TR.md
+++ b/README_TR.md
@@ -17,7 +17,7 @@
alt="Discord'da sohbet et">
+ alt="X(Twitter)'da takip et">
@@ -200,9 +200,13 @@ Yüksek kullanılabilirliğe sahip bir kurulum yapılandırmak isterseniz, Dify'
#### Dağıtım için Terraform Kullanımı
+Dify'ı bulut platformuna tek tıklamayla dağıtın [terraform](https://www.terraform.io/) kullanarak
+
##### Azure Global
-[Terraform](https://www.terraform.io/) kullanarak Dify'ı Azure'a tek tıklamayla dağıtın.
-- [@nikawang tarafından Azure Terraform](https://github.com/nikawang/dify-azure-terraform)
+- [Azure Terraform tarafından @nikawang](https://github.com/nikawang/dify-azure-terraform)
+
+##### Google Cloud
+- [Google Cloud Terraform tarafından @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
## Katkıda Bulunma
@@ -222,7 +226,7 @@ Aynı zamanda, lütfen Dify'ı sosyal medyada, etkinliklerde ve konferanslarda p
* [Github Tartışmaları](https://github.com/langgenius/dify/discussions). En uygun: geri bildirim paylaşmak ve soru sormak için.
* [GitHub Sorunları](https://github.com/langgenius/dify/issues). En uygun: Dify.AI kullanırken karşılaştığınız hatalar ve özellik önerileri için. [Katkı Kılavuzumuza](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) bakın.
* [Discord](https://discord.gg/FngNHpbcY7). En uygun: uygulamalarınızı paylaşmak ve toplulukla vakit geçirmek için.
-* [Twitter](https://twitter.com/dify_ai). En uygun: uygulamalarınızı paylaşmak ve toplulukla vakit geçirmek için.
+* [X(Twitter)](https://twitter.com/dify_ai). En uygun: uygulamalarınızı paylaşmak ve toplulukla vakit geçirmek için.
## Star history
diff --git a/README_VI.md b/README_VI.md
index 6d4035eceb06d..8d49e4976626b 100644
--- a/README_VI.md
+++ b/README_VI.md
@@ -17,7 +17,7 @@
alt="chat trên Discord">
+ alt="theo dõi trên X(Twitter)">
@@ -196,10 +196,14 @@ Nếu bạn muốn cấu hình một cài đặt có độ sẵn sàng cao, có
#### Sử dụng Terraform để Triển khai
+Triển khai Dify lên nền tảng đám mây với một cú nhấp chuột bằng cách sử dụng [terraform](https://www.terraform.io/)
+
##### Azure Global
-Triển khai Dify lên Azure chỉ với một cú nhấp chuột bằng cách sử dụng [terraform](https://www.terraform.io/).
- [Azure Terraform bởi @nikawang](https://github.com/nikawang/dify-azure-terraform)
+##### Google Cloud
+- [Google Cloud Terraform bởi @sotazum](https://github.com/DeNA/dify-google-cloud-terraform)
+
## Đóng góp
Đối với những người muốn đóng góp mã, xem [Hướng dẫn Đóng góp](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) của chúng tôi.
@@ -219,7 +223,7 @@ Triển khai Dify lên Azure chỉ với một cú nhấp chuột bằng cách s
* [Thảo luận GitHub](https://github.com/langgenius/dify/discussions). Tốt nhất cho: chia sẻ phản hồi và đặt câu hỏi.
* [Vấn đề GitHub](https://github.com/langgenius/dify/issues). Tốt nhất cho: lỗi bạn gặp phải khi sử dụng Dify.AI và đề xuất tính năng. Xem [Hướng dẫn Đóng góp](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) của chúng tôi.
* [Discord](https://discord.gg/FngNHpbcY7). Tốt nhất cho: chia sẻ ứng dụng của bạn và giao lưu với cộng đồng.
-* [Twitter](https://twitter.com/dify_ai). Tốt nhất cho: chia sẻ ứng dụng của bạn và giao lưu với cộng đồng.
+* [X(Twitter)](https://twitter.com/dify_ai). Tốt nhất cho: chia sẻ ứng dụng của bạn và giao lưu với cộng đồng.
## Lịch sử Yêu thích
diff --git a/api/.env.example b/api/.env.example
index a3db406aeabdc..aa155003abd3f 100644
--- a/api/.env.example
+++ b/api/.env.example
@@ -20,6 +20,9 @@ FILES_URL=http://127.0.0.1:5001
# The time in seconds after the signature is rejected
FILES_ACCESS_TIMEOUT=300
+# Access token expiration time in minutes
+ACCESS_TOKEN_EXPIRE_MINUTES=60
+
# celery configuration
CELERY_BROKER_URL=redis://:difyai123456@localhost:6379/1
@@ -39,7 +42,7 @@ DB_DATABASE=dify
# Storage configuration
# use for store upload files, private keys...
-# storage type: local, s3, azure-blob, google-storage, tencent-cos, huawei-obs, volcengine-tos
+# storage type: local, s3, azure-blob, google-storage, tencent-cos, huawei-obs, volcengine-tos, baidu-obs, supabase
STORAGE_TYPE=local
STORAGE_LOCAL_PATH=storage
S3_USE_AWS_MANAGED_IAM=false
@@ -79,6 +82,12 @@ HUAWEI_OBS_SECRET_KEY=your-secret-key
HUAWEI_OBS_ACCESS_KEY=your-access-key
HUAWEI_OBS_SERVER=your-server-url
+# Baidu OBS Storage Configuration
+BAIDU_OBS_BUCKET_NAME=your-bucket-name
+BAIDU_OBS_SECRET_KEY=your-secret-key
+BAIDU_OBS_ACCESS_KEY=your-access-key
+BAIDU_OBS_ENDPOINT=your-server-url
+
# OCI Storage configuration
OCI_ENDPOINT=your-endpoint
OCI_BUCKET_NAME=your-bucket-name
@@ -93,11 +102,16 @@ VOLCENGINE_TOS_ACCESS_KEY=your-access-key
VOLCENGINE_TOS_SECRET_KEY=your-secret-key
VOLCENGINE_TOS_REGION=your-region
+# Supabase Storage Configuration
+SUPABASE_BUCKET_NAME=your-bucket-name
+SUPABASE_API_KEY=your-access-key
+SUPABASE_URL=your-server-url
+
# CORS configuration
WEB_API_CORS_ALLOW_ORIGINS=http://127.0.0.1:3000,*
CONSOLE_CORS_ALLOW_ORIGINS=http://127.0.0.1:3000,*
-# Vector database configuration, support: weaviate, qdrant, milvus, myscale, relyt, pgvecto_rs, pgvector, pgvector, chroma, opensearch, tidb_vector
+# Vector database configuration, support: weaviate, qdrant, milvus, myscale, relyt, pgvecto_rs, pgvector, pgvector, chroma, opensearch, tidb_vector, vikingdb
VECTOR_STORE=weaviate
# Weaviate configuration
@@ -162,6 +176,8 @@ PGVECTOR_PORT=5433
PGVECTOR_USER=postgres
PGVECTOR_PASSWORD=postgres
PGVECTOR_DATABASE=postgres
+PGVECTOR_MIN_CONNECTION=1
+PGVECTOR_MAX_CONNECTION=5
# Tidb Vector configuration
TIDB_VECTOR_HOST=xxx.eu-central-1.xxx.aws.tidbcloud.com
@@ -195,6 +211,24 @@ OPENSEARCH_USER=admin
OPENSEARCH_PASSWORD=admin
OPENSEARCH_SECURE=true
+# Baidu configuration
+BAIDU_VECTOR_DB_ENDPOINT=http://127.0.0.1:5287
+BAIDU_VECTOR_DB_CONNECTION_TIMEOUT_MS=30000
+BAIDU_VECTOR_DB_ACCOUNT=root
+BAIDU_VECTOR_DB_API_KEY=dify
+BAIDU_VECTOR_DB_DATABASE=dify
+BAIDU_VECTOR_DB_SHARD=1
+BAIDU_VECTOR_DB_REPLICAS=3
+
+# ViKingDB configuration
+VIKINGDB_ACCESS_KEY=your-ak
+VIKINGDB_SECRET_KEY=your-sk
+VIKINGDB_REGION=cn-shanghai
+VIKINGDB_HOST=api-vikingdb.xxx.volces.com
+VIKINGDB_SCHEMA=http
+VIKINGDB_CONNECTION_TIMEOUT=30
+VIKINGDB_SOCKET_TIMEOUT=30
+
# Upload configuration
UPLOAD_FILE_SIZE_LIMIT=15
UPLOAD_FILE_BATCH_LIMIT=5
@@ -263,6 +297,9 @@ HTTP_REQUEST_MAX_WRITE_TIMEOUT=600
HTTP_REQUEST_NODE_MAX_BINARY_SIZE=10485760
HTTP_REQUEST_NODE_MAX_TEXT_SIZE=1048576
+# Respect X-* headers to redirect clients
+RESPECT_XFORWARD_HEADERS_ENABLED=false
+
# Log file path
LOG_FILE=
diff --git a/api/README.md b/api/README.md
index 70ca2e86a85a7..bab33f9293a31 100644
--- a/api/README.md
+++ b/api/README.md
@@ -65,14 +65,12 @@
8. Start Dify [web](../web) service.
9. Setup your application by visiting `http://localhost:3000`...
-10. If you need to debug local async processing, please start the worker service.
+10. If you need to handle and debug the async tasks (e.g. dataset importing and documents indexing), please start the worker service.
```bash
poetry run python -m celery -A app.celery worker -P gevent -c 1 --loglevel INFO -Q dataset,generation,mail,ops_trace,app_deletion
```
- The started celery app handles the async tasks, e.g. dataset importing and documents indexing.
-
## Testing
1. Install dependencies for both the backend and the test environment
diff --git a/api/app.py b/api/app.py
index 91a49337fccbd..52dd492225339 100644
--- a/api/app.py
+++ b/api/app.py
@@ -26,7 +26,7 @@
from configs import dify_config
# DO NOT REMOVE BELOW
-from events import event_handlers
+from events import event_handlers # noqa: F401
from extensions import (
ext_celery,
ext_code_based_extension,
@@ -36,6 +36,7 @@
ext_login,
ext_mail,
ext_migrate,
+ ext_proxy_fix,
ext_redis,
ext_sentry,
ext_storage,
@@ -45,7 +46,7 @@
from libs.passport import PassportService
# TODO: Find a way to avoid importing models here
-from models import account, dataset, model, source, task, tool, tools, web
+from models import account, dataset, model, source, task, tool, tools, web # noqa: F401
from services.account_service import AccountService
# DO NOT REMOVE ABOVE
@@ -53,11 +54,9 @@
warnings.simplefilter("ignore", ResourceWarning)
-# fix windows platform
-if os.name == "nt":
- os.system('tzutil /s "UTC"')
-else:
- os.environ["TZ"] = "UTC"
+os.environ["TZ"] = "UTC"
+# windows platform not support tzset
+if hasattr(time, "tzset"):
time.tzset()
@@ -158,6 +157,7 @@ def initialize_extensions(app):
ext_mail.init_app(app)
ext_hosting_provider.init_app(app)
ext_sentry.init_app(app)
+ ext_proxy_fix.init_app(app)
# Flask-Login configuration
@@ -183,10 +183,10 @@ def load_user_from_request(request_from_flask_login):
decoded = PassportService().verify(auth_token)
user_id = decoded.get("user_id")
- account = AccountService.load_logged_in_account(account_id=user_id, token=auth_token)
- if account:
- contexts.tenant_id.set(account.current_tenant_id)
- return account
+ logged_in_account = AccountService.load_logged_in_account(account_id=user_id)
+ if logged_in_account:
+ contexts.tenant_id.set(logged_in_account.current_tenant_id)
+ return logged_in_account
@login_manager.unauthorized_handler
diff --git a/api/commands.py b/api/commands.py
index 3a6b4963cfa68..dbcd8a744d3a4 100644
--- a/api/commands.py
+++ b/api/commands.py
@@ -28,28 +28,28 @@
@click.command("reset-password", help="Reset the account password.")
-@click.option("--email", prompt=True, help="The email address of the account whose password you need to reset")
-@click.option("--new-password", prompt=True, help="the new password.")
-@click.option("--password-confirm", prompt=True, help="the new password confirm.")
+@click.option("--email", prompt=True, help="Account email to reset password for")
+@click.option("--new-password", prompt=True, help="New password")
+@click.option("--password-confirm", prompt=True, help="Confirm new password")
def reset_password(email, new_password, password_confirm):
"""
Reset password of owner account
Only available in SELF_HOSTED mode
"""
if str(new_password).strip() != str(password_confirm).strip():
- click.echo(click.style("sorry. The two passwords do not match.", fg="red"))
+ click.echo(click.style("Passwords do not match.", fg="red"))
return
account = db.session.query(Account).filter(Account.email == email).one_or_none()
if not account:
- click.echo(click.style("sorry. the account: [{}] not exist .".format(email), fg="red"))
+ click.echo(click.style("Account not found for email: {}".format(email), fg="red"))
return
try:
valid_password(new_password)
except:
- click.echo(click.style("sorry. The passwords must match {} ".format(password_pattern), fg="red"))
+ click.echo(click.style("Invalid password. Must match {}".format(password_pattern), fg="red"))
return
# generate password salt
@@ -62,37 +62,37 @@ def reset_password(email, new_password, password_confirm):
account.password = base64_password_hashed
account.password_salt = base64_salt
db.session.commit()
- click.echo(click.style("Congratulations! Password has been reset.", fg="green"))
+ click.echo(click.style("Password reset successfully.", fg="green"))
@click.command("reset-email", help="Reset the account email.")
-@click.option("--email", prompt=True, help="The old email address of the account whose email you need to reset")
-@click.option("--new-email", prompt=True, help="the new email.")
-@click.option("--email-confirm", prompt=True, help="the new email confirm.")
+@click.option("--email", prompt=True, help="Current account email")
+@click.option("--new-email", prompt=True, help="New email")
+@click.option("--email-confirm", prompt=True, help="Confirm new email")
def reset_email(email, new_email, email_confirm):
"""
Replace account email
:return:
"""
if str(new_email).strip() != str(email_confirm).strip():
- click.echo(click.style("Sorry, new email and confirm email do not match.", fg="red"))
+ click.echo(click.style("New emails do not match.", fg="red"))
return
account = db.session.query(Account).filter(Account.email == email).one_or_none()
if not account:
- click.echo(click.style("sorry. the account: [{}] not exist .".format(email), fg="red"))
+ click.echo(click.style("Account not found for email: {}".format(email), fg="red"))
return
try:
email_validate(new_email)
except:
- click.echo(click.style("sorry. {} is not a valid email. ".format(email), fg="red"))
+ click.echo(click.style("Invalid email: {}".format(new_email), fg="red"))
return
account.email = new_email
db.session.commit()
- click.echo(click.style("Congratulations!, email has been reset.", fg="green"))
+ click.echo(click.style("Email updated successfully.", fg="green"))
@click.command(
@@ -104,7 +104,7 @@ def reset_email(email, new_email, email_confirm):
)
@click.confirmation_option(
prompt=click.style(
- "Are you sure you want to reset encrypt key pair? this operation cannot be rolled back!", fg="red"
+ "Are you sure you want to reset encrypt key pair? This operation cannot be rolled back!", fg="red"
)
)
def reset_encrypt_key_pair():
@@ -114,13 +114,13 @@ def reset_encrypt_key_pair():
Only support SELF_HOSTED mode.
"""
if dify_config.EDITION != "SELF_HOSTED":
- click.echo(click.style("Sorry, only support SELF_HOSTED mode.", fg="red"))
+ click.echo(click.style("This command is only for SELF_HOSTED installations.", fg="red"))
return
tenants = db.session.query(Tenant).all()
for tenant in tenants:
if not tenant:
- click.echo(click.style("Sorry, no workspace found. Please enter /install to initialize.", fg="red"))
+ click.echo(click.style("No workspaces found. Run /install first.", fg="red"))
return
tenant.encrypt_public_key = generate_key_pair(tenant.id)
@@ -137,7 +137,7 @@ def reset_encrypt_key_pair():
)
-@click.command("vdb-migrate", help="migrate vector db.")
+@click.command("vdb-migrate", help="Migrate vector db.")
@click.option("--scope", default="all", prompt=False, help="The scope of vector database to migrate, Default is All.")
def vdb_migrate(scope: str):
if scope in {"knowledge", "all"}:
@@ -150,7 +150,7 @@ def migrate_annotation_vector_database():
"""
Migrate annotation datas to target vector database .
"""
- click.echo(click.style("Start migrate annotation data.", fg="green"))
+ click.echo(click.style("Starting annotation data migration.", fg="green"))
create_count = 0
skipped_count = 0
total_count = 0
@@ -174,14 +174,14 @@ def migrate_annotation_vector_database():
f"Processing the {total_count} app {app.id}. " + f"{create_count} created, {skipped_count} skipped."
)
try:
- click.echo("Create app annotation index: {}".format(app.id))
+ click.echo("Creating app annotation index: {}".format(app.id))
app_annotation_setting = (
db.session.query(AppAnnotationSetting).filter(AppAnnotationSetting.app_id == app.id).first()
)
if not app_annotation_setting:
skipped_count = skipped_count + 1
- click.echo("App annotation setting is disabled: {}".format(app.id))
+ click.echo("App annotation setting disabled: {}".format(app.id))
continue
# get dataset_collection_binding info
dataset_collection_binding = (
@@ -190,7 +190,7 @@ def migrate_annotation_vector_database():
.first()
)
if not dataset_collection_binding:
- click.echo("App annotation collection binding is not exist: {}".format(app.id))
+ click.echo("App annotation collection binding not found: {}".format(app.id))
continue
annotations = db.session.query(MessageAnnotation).filter(MessageAnnotation.app_id == app.id).all()
dataset = Dataset(
@@ -211,11 +211,11 @@ def migrate_annotation_vector_database():
documents.append(document)
vector = Vector(dataset, attributes=["doc_id", "annotation_id", "app_id"])
- click.echo(f"Start to migrate annotation, app_id: {app.id}.")
+ click.echo(f"Migrating annotations for app: {app.id}.")
try:
vector.delete()
- click.echo(click.style(f"Successfully delete vector index for app: {app.id}.", fg="green"))
+ click.echo(click.style(f"Deleted vector index for app {app.id}.", fg="green"))
except Exception as e:
click.echo(click.style(f"Failed to delete vector index for app {app.id}.", fg="red"))
raise e
@@ -223,12 +223,12 @@ def migrate_annotation_vector_database():
try:
click.echo(
click.style(
- f"Start to created vector index with {len(documents)} annotations for app {app.id}.",
+ f"Creating vector index with {len(documents)} annotations for app {app.id}.",
fg="green",
)
)
vector.create(documents)
- click.echo(click.style(f"Successfully created vector index for app {app.id}.", fg="green"))
+ click.echo(click.style(f"Created vector index for app {app.id}.", fg="green"))
except Exception as e:
click.echo(click.style(f"Failed to created vector index for app {app.id}.", fg="red"))
raise e
@@ -237,14 +237,14 @@ def migrate_annotation_vector_database():
except Exception as e:
click.echo(
click.style(
- "Create app annotation index error: {} {}".format(e.__class__.__name__, str(e)), fg="red"
+ "Error creating app annotation index: {} {}".format(e.__class__.__name__, str(e)), fg="red"
)
)
continue
click.echo(
click.style(
- f"Congratulations! Create {create_count} app annotation indexes, and skipped {skipped_count} apps.",
+ f"Migration complete. Created {create_count} app annotation indexes. Skipped {skipped_count} apps.",
fg="green",
)
)
@@ -254,7 +254,7 @@ def migrate_knowledge_vector_database():
"""
Migrate vector database datas to target vector database .
"""
- click.echo(click.style("Start migrate vector db.", fg="green"))
+ click.echo(click.style("Starting vector database migration.", fg="green"))
create_count = 0
skipped_count = 0
total_count = 0
@@ -278,7 +278,7 @@ def migrate_knowledge_vector_database():
f"Processing the {total_count} dataset {dataset.id}. {create_count} created, {skipped_count} skipped."
)
try:
- click.echo("Create dataset vdb index: {}".format(dataset.id))
+ click.echo("Creating dataset vector database index: {}".format(dataset.id))
if dataset.index_struct_dict:
if dataset.index_struct_dict["type"] == vector_type:
skipped_count = skipped_count + 1
@@ -299,7 +299,7 @@ def migrate_knowledge_vector_database():
if dataset_collection_binding:
collection_name = dataset_collection_binding.collection_name
else:
- raise ValueError("Dataset Collection Bindings is not exist!")
+ raise ValueError("Dataset Collection Binding not found")
else:
dataset_id = dataset.id
collection_name = Dataset.gen_collection_name_by_id(dataset_id)
@@ -347,18 +347,24 @@ def migrate_knowledge_vector_database():
index_name = Dataset.gen_collection_name_by_id(dataset_id)
index_struct_dict = {"type": "elasticsearch", "vector_store": {"class_prefix": index_name}}
dataset.index_struct = json.dumps(index_struct_dict)
+ elif vector_type == VectorType.BAIDU:
+ dataset_id = dataset.id
+ collection_name = Dataset.gen_collection_name_by_id(dataset_id)
+ index_struct_dict = {
+ "type": VectorType.BAIDU,
+ "vector_store": {"class_prefix": collection_name},
+ }
+ dataset.index_struct = json.dumps(index_struct_dict)
else:
raise ValueError(f"Vector store {vector_type} is not supported.")
vector = Vector(dataset)
- click.echo(f"Start to migrate dataset {dataset.id}.")
+ click.echo(f"Migrating dataset {dataset.id}.")
try:
vector.delete()
click.echo(
- click.style(
- f"Successfully delete vector index {collection_name} for dataset {dataset.id}.", fg="green"
- )
+ click.style(f"Deleted vector index {collection_name} for dataset {dataset.id}.", fg="green")
)
except Exception as e:
click.echo(
@@ -410,15 +416,13 @@ def migrate_knowledge_vector_database():
try:
click.echo(
click.style(
- f"Start to created vector index with {len(documents)} documents of {segments_count}"
+ f"Creating vector index with {len(documents)} documents of {segments_count}"
f" segments for dataset {dataset.id}.",
fg="green",
)
)
vector.create(documents)
- click.echo(
- click.style(f"Successfully created vector index for dataset {dataset.id}.", fg="green")
- )
+ click.echo(click.style(f"Created vector index for dataset {dataset.id}.", fg="green"))
except Exception as e:
click.echo(click.style(f"Failed to created vector index for dataset {dataset.id}.", fg="red"))
raise e
@@ -429,13 +433,13 @@ def migrate_knowledge_vector_database():
except Exception as e:
db.session.rollback()
click.echo(
- click.style("Create dataset index error: {} {}".format(e.__class__.__name__, str(e)), fg="red")
+ click.style("Error creating dataset index: {} {}".format(e.__class__.__name__, str(e)), fg="red")
)
continue
click.echo(
click.style(
- f"Congratulations! Create {create_count} dataset indexes, and skipped {skipped_count} datasets.", fg="green"
+ f"Migration complete. Created {create_count} dataset indexes. Skipped {skipped_count} datasets.", fg="green"
)
)
@@ -445,7 +449,7 @@ def convert_to_agent_apps():
"""
Convert Agent Assistant to Agent App.
"""
- click.echo(click.style("Start convert to agent apps.", fg="green"))
+ click.echo(click.style("Starting convert to agent apps.", fg="green"))
proceeded_app_ids = []
@@ -496,23 +500,23 @@ def convert_to_agent_apps():
except Exception as e:
click.echo(click.style("Convert app error: {} {}".format(e.__class__.__name__, str(e)), fg="red"))
- click.echo(click.style("Congratulations! Converted {} agent apps.".format(len(proceeded_app_ids)), fg="green"))
+ click.echo(click.style("Conversion complete. Converted {} agent apps.".format(len(proceeded_app_ids)), fg="green"))
-@click.command("add-qdrant-doc-id-index", help="add qdrant doc_id index.")
-@click.option("--field", default="metadata.doc_id", prompt=False, help="index field , default is metadata.doc_id.")
+@click.command("add-qdrant-doc-id-index", help="Add Qdrant doc_id index.")
+@click.option("--field", default="metadata.doc_id", prompt=False, help="Index field , default is metadata.doc_id.")
def add_qdrant_doc_id_index(field: str):
- click.echo(click.style("Start add qdrant doc_id index.", fg="green"))
+ click.echo(click.style("Starting Qdrant doc_id index creation.", fg="green"))
vector_type = dify_config.VECTOR_STORE
if vector_type != "qdrant":
- click.echo(click.style("Sorry, only support qdrant vector store.", fg="red"))
+ click.echo(click.style("This command only supports Qdrant vector store.", fg="red"))
return
create_count = 0
try:
bindings = db.session.query(DatasetCollectionBinding).all()
if not bindings:
- click.echo(click.style("Sorry, no dataset collection bindings found.", fg="red"))
+ click.echo(click.style("No dataset collection bindings found.", fg="red"))
return
import qdrant_client
from qdrant_client.http.exceptions import UnexpectedResponse
@@ -522,7 +526,7 @@ def add_qdrant_doc_id_index(field: str):
for binding in bindings:
if dify_config.QDRANT_URL is None:
- raise ValueError("Qdrant url is required.")
+ raise ValueError("Qdrant URL is required.")
qdrant_config = QdrantConfig(
endpoint=dify_config.QDRANT_URL,
api_key=dify_config.QDRANT_API_KEY,
@@ -539,41 +543,39 @@ def add_qdrant_doc_id_index(field: str):
except UnexpectedResponse as e:
# Collection does not exist, so return
if e.status_code == 404:
- click.echo(
- click.style(f"Collection not found, collection_name:{binding.collection_name}.", fg="red")
- )
+ click.echo(click.style(f"Collection not found: {binding.collection_name}.", fg="red"))
continue
# Some other error occurred, so re-raise the exception
else:
click.echo(
click.style(
- f"Failed to create qdrant index, collection_name:{binding.collection_name}.", fg="red"
+ f"Failed to create Qdrant index for collection: {binding.collection_name}.", fg="red"
)
)
except Exception as e:
- click.echo(click.style("Failed to create qdrant client.", fg="red"))
+ click.echo(click.style("Failed to create Qdrant client.", fg="red"))
- click.echo(click.style(f"Congratulations! Create {create_count} collection indexes.", fg="green"))
+ click.echo(click.style(f"Index creation complete. Created {create_count} collection indexes.", fg="green"))
@click.command("create-tenant", help="Create account and tenant.")
-@click.option("--email", prompt=True, help="The email address of the tenant account.")
-@click.option("--name", prompt=True, help="The workspace name of the tenant account.")
+@click.option("--email", prompt=True, help="Tenant account email.")
+@click.option("--name", prompt=True, help="Workspace name.")
@click.option("--language", prompt=True, help="Account language, default: en-US.")
def create_tenant(email: str, language: Optional[str] = None, name: Optional[str] = None):
"""
Create tenant account
"""
if not email:
- click.echo(click.style("Sorry, email is required.", fg="red"))
+ click.echo(click.style("Email is required.", fg="red"))
return
# Create account
email = email.strip()
if "@" not in email:
- click.echo(click.style("Sorry, invalid email address.", fg="red"))
+ click.echo(click.style("Invalid email address.", fg="red"))
return
account_name = email.split("@")[0]
@@ -593,19 +595,19 @@ def create_tenant(email: str, language: Optional[str] = None, name: Optional[str
click.echo(
click.style(
- "Congratulations! Account and tenant created.\nAccount: {}\nPassword: {}".format(email, new_password),
+ "Account and tenant created.\nAccount: {}\nPassword: {}".format(email, new_password),
fg="green",
)
)
-@click.command("upgrade-db", help="upgrade the database")
+@click.command("upgrade-db", help="Upgrade the database")
def upgrade_db():
click.echo("Preparing database migration...")
lock = redis_client.lock(name="db_upgrade_lock", timeout=60)
if lock.acquire(blocking=False):
try:
- click.echo(click.style("Start database migration.", fg="green"))
+ click.echo(click.style("Starting database migration.", fg="green"))
# run db migration
import flask_migrate
@@ -615,7 +617,7 @@ def upgrade_db():
click.echo(click.style("Database migration successful!", fg="green"))
except Exception as e:
- logging.exception(f"Database migration failed, error: {e}")
+ logging.exception(f"Database migration failed: {e}")
finally:
lock.release()
else:
@@ -627,7 +629,7 @@ def fix_app_site_missing():
"""
Fix app related site missing issue.
"""
- click.echo(click.style("Start fix app related site missing issue.", fg="green"))
+ click.echo(click.style("Starting fix for missing app-related sites.", fg="green"))
failed_app_ids = []
while True:
@@ -650,22 +652,22 @@ def fix_app_site_missing():
if tenant:
accounts = tenant.get_accounts()
if not accounts:
- print("Fix app {} failed.".format(app.id))
+ print("Fix failed for app {}".format(app.id))
continue
account = accounts[0]
- print("Fix app {} related site missing issue.".format(app.id))
+ print("Fixing missing site for app {}".format(app.id))
app_was_created.send(app, account=account)
except Exception as e:
failed_app_ids.append(app_id)
- click.echo(click.style("Fix app {} related site missing issue failed!".format(app_id), fg="red"))
+ click.echo(click.style("Failed to fix missing site for app {}".format(app_id), fg="red"))
logging.exception(f"Fix app related site missing issue failed, error: {e}")
continue
if not processed_count:
break
- click.echo(click.style("Congratulations! Fix app related site missing issue successful!", fg="green"))
+ click.echo(click.style("Fix for missing app-related sites completed successfully!", fg="green"))
def register_commands(app):
diff --git a/api/configs/deploy/__init__.py b/api/configs/deploy/__init__.py
index 10271483c4669..66d6a55b4c7ea 100644
--- a/api/configs/deploy/__init__.py
+++ b/api/configs/deploy/__init__.py
@@ -4,30 +4,30 @@
class DeploymentConfig(BaseSettings):
"""
- Deployment configs
+ Configuration settings for application deployment
"""
APPLICATION_NAME: str = Field(
- description="application name",
+ description="Name of the application, used for identification and logging purposes",
default="langgenius/dify",
)
DEBUG: bool = Field(
- description="whether to enable debug mode.",
+ description="Enable debug mode for additional logging and development features",
default=False,
)
TESTING: bool = Field(
- description="",
+ description="Enable testing mode for running automated tests",
default=False,
)
EDITION: str = Field(
- description="deployment edition",
+ description="Deployment edition of the application (e.g., 'SELF_HOSTED', 'CLOUD')",
default="SELF_HOSTED",
)
DEPLOY_ENV: str = Field(
- description="deployment environment, default to PRODUCTION.",
+ description="Deployment environment (e.g., 'PRODUCTION', 'DEVELOPMENT'), default to PRODUCTION",
default="PRODUCTION",
)
diff --git a/api/configs/enterprise/__init__.py b/api/configs/enterprise/__init__.py
index c661593a44264..eda6345e145a9 100644
--- a/api/configs/enterprise/__init__.py
+++ b/api/configs/enterprise/__init__.py
@@ -4,17 +4,17 @@
class EnterpriseFeatureConfig(BaseSettings):
"""
- Enterprise feature configs.
+ Configuration for enterprise-level features.
**Before using, please contact business@dify.ai by email to inquire about licensing matters.**
"""
ENTERPRISE_ENABLED: bool = Field(
- description="whether to enable enterprise features."
+ description="Enable or disable enterprise-level features."
"Before using, please contact business@dify.ai by email to inquire about licensing matters.",
default=False,
)
CAN_REPLACE_LOGO: bool = Field(
- description="whether to allow replacing enterprise logo.",
+ description="Allow customization of the enterprise logo.",
default=False,
)
diff --git a/api/configs/extra/notion_config.py b/api/configs/extra/notion_config.py
index bd1268fa4565f..f9c4d7346399a 100644
--- a/api/configs/extra/notion_config.py
+++ b/api/configs/extra/notion_config.py
@@ -6,30 +6,31 @@
class NotionConfig(BaseSettings):
"""
- Notion integration configs
+ Configuration settings for Notion integration
"""
NOTION_CLIENT_ID: Optional[str] = Field(
- description="Notion client ID",
+ description="Client ID for Notion API authentication. Required for OAuth 2.0 flow.",
default=None,
)
NOTION_CLIENT_SECRET: Optional[str] = Field(
- description="Notion client secret key",
+ description="Client secret for Notion API authentication. Required for OAuth 2.0 flow.",
default=None,
)
NOTION_INTEGRATION_TYPE: Optional[str] = Field(
- description="Notion integration type, default to None, available values: internal.",
+ description="Type of Notion integration."
+ " Set to 'internal' for internal integrations, or None for public integrations.",
default=None,
)
NOTION_INTERNAL_SECRET: Optional[str] = Field(
- description="Notion internal secret key",
+ description="Secret key for internal Notion integrations. Required when NOTION_INTEGRATION_TYPE is 'internal'.",
default=None,
)
NOTION_INTEGRATION_TOKEN: Optional[str] = Field(
- description="Notion integration token",
+ description="Integration token for Notion API access. Used for direct API calls without OAuth flow.",
default=None,
)
diff --git a/api/configs/extra/sentry_config.py b/api/configs/extra/sentry_config.py
index ea9ea60ffbd6c..f76a6bdb95ca5 100644
--- a/api/configs/extra/sentry_config.py
+++ b/api/configs/extra/sentry_config.py
@@ -6,20 +6,23 @@
class SentryConfig(BaseSettings):
"""
- Sentry configs
+ Configuration settings for Sentry error tracking and performance monitoring
"""
SENTRY_DSN: Optional[str] = Field(
- description="Sentry DSN",
+ description="Sentry Data Source Name (DSN)."
+ " This is the unique identifier of your Sentry project, used to send events to the correct project.",
default=None,
)
SENTRY_TRACES_SAMPLE_RATE: NonNegativeFloat = Field(
- description="Sentry trace sample rate",
+ description="Sample rate for Sentry performance monitoring traces."
+ " Value between 0.0 and 1.0, where 1.0 means 100% of traces are sent to Sentry.",
default=1.0,
)
SENTRY_PROFILES_SAMPLE_RATE: NonNegativeFloat = Field(
- description="Sentry profiles sample rate",
+ description="Sample rate for Sentry profiling."
+ " Value between 0.0 and 1.0, where 1.0 means 100% of profiles are sent to Sentry.",
default=1.0,
)
diff --git a/api/configs/feature/__init__.py b/api/configs/feature/__init__.py
index f794552c36d26..a3334d16345e9 100644
--- a/api/configs/feature/__init__.py
+++ b/api/configs/feature/__init__.py
@@ -8,145 +8,143 @@
class SecurityConfig(BaseSettings):
"""
- Secret Key configs
+ Security-related configurations for the application
"""
SECRET_KEY: Optional[str] = Field(
- description="Your App secret key will be used for securely signing the session cookie"
+ description="Secret key for secure session cookie signing."
"Make sure you are changing this key for your deployment with a strong key."
- "You can generate a strong key using `openssl rand -base64 42`."
- "Alternatively you can set it with `SECRET_KEY` environment variable.",
+ "Generate a strong key using `openssl rand -base64 42` or set via the `SECRET_KEY` environment variable.",
default=None,
)
RESET_PASSWORD_TOKEN_EXPIRY_HOURS: PositiveInt = Field(
- description="Expiry time in hours for reset token",
+ description="Duration in hours for which a password reset token remains valid",
default=24,
)
class AppExecutionConfig(BaseSettings):
"""
- App Execution configs
+ Configuration parameters for application execution
"""
APP_MAX_EXECUTION_TIME: PositiveInt = Field(
- description="execution timeout in seconds for app execution",
+ description="Maximum allowed execution time for the application in seconds",
default=1200,
)
APP_MAX_ACTIVE_REQUESTS: NonNegativeInt = Field(
- description="max active request per app, 0 means unlimited",
+ description="Maximum number of concurrent active requests per app (0 for unlimited)",
default=0,
)
class CodeExecutionSandboxConfig(BaseSettings):
"""
- Code Execution Sandbox configs
+ Configuration for the code execution sandbox environment
"""
CODE_EXECUTION_ENDPOINT: HttpUrl = Field(
- description="endpoint URL of code execution service",
+ description="URL endpoint for the code execution service",
default="http://sandbox:8194",
)
CODE_EXECUTION_API_KEY: str = Field(
- description="API key for code execution service",
+ description="API key for accessing the code execution service",
default="dify-sandbox",
)
CODE_EXECUTION_CONNECT_TIMEOUT: Optional[float] = Field(
- description="connect timeout in seconds for code execution request",
+ description="Connection timeout in seconds for code execution requests",
default=10.0,
)
CODE_EXECUTION_READ_TIMEOUT: Optional[float] = Field(
- description="read timeout in seconds for code execution request",
+ description="Read timeout in seconds for code execution requests",
default=60.0,
)
CODE_EXECUTION_WRITE_TIMEOUT: Optional[float] = Field(
- description="write timeout in seconds for code execution request",
+ description="Write timeout in seconds for code execution request",
default=10.0,
)
CODE_MAX_NUMBER: PositiveInt = Field(
- description="max depth for code execution",
+ description="Maximum allowed numeric value in code execution",
default=9223372036854775807,
)
CODE_MIN_NUMBER: NegativeInt = Field(
- description="",
+ description="Minimum allowed numeric value in code execution",
default=-9223372036854775807,
)
CODE_MAX_DEPTH: PositiveInt = Field(
- description="max depth for code execution",
+ description="Maximum allowed depth for nested structures in code execution",
default=5,
)
CODE_MAX_PRECISION: PositiveInt = Field(
- description="max precision digits for float type in code execution",
+ description="mMaximum number of decimal places for floating-point numbers in code execution",
default=20,
)
CODE_MAX_STRING_LENGTH: PositiveInt = Field(
- description="max string length for code execution",
+ description="Maximum allowed length for strings in code execution",
default=80000,
)
CODE_MAX_STRING_ARRAY_LENGTH: PositiveInt = Field(
- description="",
+ description="Maximum allowed length for string arrays in code execution",
default=30,
)
CODE_MAX_OBJECT_ARRAY_LENGTH: PositiveInt = Field(
- description="",
+ description="Maximum allowed length for object arrays in code execution",
default=30,
)
CODE_MAX_NUMBER_ARRAY_LENGTH: PositiveInt = Field(
- description="",
+ description="Maximum allowed length for numeric arrays in code execution",
default=1000,
)
class EndpointConfig(BaseSettings):
"""
- Module URL configs
+ Configuration for various application endpoints and URLs
"""
CONSOLE_API_URL: str = Field(
- description="The backend URL prefix of the console API."
- "used to concatenate the login authorization callback or notion integration callback.",
+ description="Base URL for the console API,"
+ "used for login authentication callback or notion integration callbacks",
default="",
)
CONSOLE_WEB_URL: str = Field(
- description="The front-end URL prefix of the console web."
- "used to concatenate some front-end addresses and for CORS configuration use.",
+ description="Base URL for the console web interface," "used for frontend references and CORS configuration",
default="",
)
SERVICE_API_URL: str = Field(
- description="Service API Url prefix. used to display Service API Base Url to the front-end.",
+ description="Base URL for the service API, displayed to users for API access",
default="",
)
APP_WEB_URL: str = Field(
- description="WebApp Url prefix. used to display WebAPP API Base Url to the front-end.",
+ description="Base URL for the web application, used for frontend references",
default="",
)
class FileAccessConfig(BaseSettings):
"""
- File Access configs
+ Configuration for file access and handling
"""
FILES_URL: str = Field(
- description="File preview or download Url prefix."
- " used to display File preview or download Url to the front-end or as Multi-model inputs;"
+ description="Base URL for file preview or download,"
+ " used for frontend display and multi-model inputs"
"Url is signed and has expiration time.",
validation_alias=AliasChoices("FILES_URL", "CONSOLE_API_URL"),
alias_priority=1,
@@ -154,49 +152,49 @@ class FileAccessConfig(BaseSettings):
)
FILES_ACCESS_TIMEOUT: int = Field(
- description="timeout in seconds for file accessing",
+ description="Expiration time in seconds for file access URLs",
default=300,
)
class FileUploadConfig(BaseSettings):
"""
- File Uploading configs
+ Configuration for file upload limitations
"""
UPLOAD_FILE_SIZE_LIMIT: NonNegativeInt = Field(
- description="size limit in Megabytes for uploading files",
+ description="Maximum allowed file size for uploads in megabytes",
default=15,
)
UPLOAD_FILE_BATCH_LIMIT: NonNegativeInt = Field(
- description="batch size limit for uploading files",
+ description="Maximum number of files allowed in a single upload batch",
default=5,
)
UPLOAD_IMAGE_FILE_SIZE_LIMIT: NonNegativeInt = Field(
- description="image file size limit in Megabytes for uploading files",
+ description="Maximum allowed image file size for uploads in megabytes",
default=10,
)
BATCH_UPLOAD_LIMIT: NonNegativeInt = Field(
- description="", # todo: to be clarified
+ description="Maximum number of files allowed in a batch upload operation",
default=20,
)
class HttpConfig(BaseSettings):
"""
- HTTP configs
+ HTTP-related configurations for the application
"""
API_COMPRESSION_ENABLED: bool = Field(
- description="whether to enable HTTP response compression of gzip",
+ description="Enable or disable gzip compression for HTTP responses",
default=False,
)
inner_CONSOLE_CORS_ALLOW_ORIGINS: str = Field(
- description="",
+ description="Comma-separated list of allowed origins for CORS in the console",
validation_alias=AliasChoices("CONSOLE_CORS_ALLOW_ORIGINS", "CONSOLE_WEB_URL"),
default="",
)
@@ -218,359 +216,371 @@ def WEB_API_CORS_ALLOW_ORIGINS(self) -> list[str]:
return self.inner_WEB_API_CORS_ALLOW_ORIGINS.split(",")
HTTP_REQUEST_MAX_CONNECT_TIMEOUT: Annotated[
- PositiveInt, Field(ge=10, description="connect timeout in seconds for HTTP request")
+ PositiveInt, Field(ge=10, description="Maximum connection timeout in seconds for HTTP requests")
] = 10
HTTP_REQUEST_MAX_READ_TIMEOUT: Annotated[
- PositiveInt, Field(ge=60, description="read timeout in seconds for HTTP request")
+ PositiveInt, Field(ge=60, description="Maximum read timeout in seconds for HTTP requests")
] = 60
HTTP_REQUEST_MAX_WRITE_TIMEOUT: Annotated[
- PositiveInt, Field(ge=10, description="read timeout in seconds for HTTP request")
+ PositiveInt, Field(ge=10, description="Maximum write timeout in seconds for HTTP requests")
] = 20
HTTP_REQUEST_NODE_MAX_BINARY_SIZE: PositiveInt = Field(
- description="",
+ description="Maximum allowed size in bytes for binary data in HTTP requests",
default=10 * 1024 * 1024,
)
HTTP_REQUEST_NODE_MAX_TEXT_SIZE: PositiveInt = Field(
- description="",
+ description="Maximum allowed size in bytes for text data in HTTP requests",
default=1 * 1024 * 1024,
)
SSRF_PROXY_HTTP_URL: Optional[str] = Field(
- description="HTTP URL for SSRF proxy",
+ description="Proxy URL for HTTP requests to prevent Server-Side Request Forgery (SSRF)",
default=None,
)
SSRF_PROXY_HTTPS_URL: Optional[str] = Field(
- description="HTTPS URL for SSRF proxy",
+ description="Proxy URL for HTTPS requests to prevent Server-Side Request Forgery (SSRF)",
default=None,
)
+ RESPECT_XFORWARD_HEADERS_ENABLED: bool = Field(
+ description="Enable or disable the X-Forwarded-For Proxy Fix middleware from Werkzeug"
+ " to respect X-* headers to redirect clients",
+ default=False,
+ )
+
class InnerAPIConfig(BaseSettings):
"""
- Inner API configs
+ Configuration for internal API functionality
"""
INNER_API: bool = Field(
- description="whether to enable the inner API",
+ description="Enable or disable the internal API",
default=False,
)
INNER_API_KEY: Optional[str] = Field(
- description="The inner API key is used to authenticate the inner API",
+ description="API key for accessing the internal API",
default=None,
)
class LoggingConfig(BaseSettings):
"""
- Logging configs
+ Configuration for application logging
"""
LOG_LEVEL: str = Field(
- description="Log output level, default to INFO. It is recommended to set it to ERROR for production.",
+ description="Logging level, default to INFO. Set to ERROR for production environments.",
default="INFO",
)
LOG_FILE: Optional[str] = Field(
- description="logging output file path",
+ description="File path for log output.",
default=None,
)
LOG_FORMAT: str = Field(
- description="log format",
+ description="Format string for log messages",
default="%(asctime)s.%(msecs)03d %(levelname)s [%(threadName)s] [%(filename)s:%(lineno)d] - %(message)s",
)
LOG_DATEFORMAT: Optional[str] = Field(
- description="log date format",
+ description="Date format string for log timestamps",
default=None,
)
LOG_TZ: Optional[str] = Field(
- description="specify log timezone, eg: America/New_York",
+ description="Timezone for log timestamps (e.g., 'America/New_York')",
default=None,
)
class ModelLoadBalanceConfig(BaseSettings):
"""
- Model load balance configs
+ Configuration for model load balancing
"""
MODEL_LB_ENABLED: bool = Field(
- description="whether to enable model load balancing",
+ description="Enable or disable load balancing for models",
default=False,
)
class BillingConfig(BaseSettings):
"""
- Platform Billing Configurations
+ Configuration for platform billing features
"""
BILLING_ENABLED: bool = Field(
- description="whether to enable billing",
+ description="Enable or disable billing functionality",
default=False,
)
class UpdateConfig(BaseSettings):
"""
- Update configs
+ Configuration for application update checks
"""
CHECK_UPDATE_URL: str = Field(
- description="url for checking updates",
+ description="URL to check for application updates",
default="https://updates.dify.ai",
)
class WorkflowConfig(BaseSettings):
"""
- Workflow feature configs
+ Configuration for workflow execution
"""
WORKFLOW_MAX_EXECUTION_STEPS: PositiveInt = Field(
- description="max execution steps in single workflow execution",
+ description="Maximum number of steps allowed in a single workflow execution",
default=500,
)
WORKFLOW_MAX_EXECUTION_TIME: PositiveInt = Field(
- description="max execution time in seconds in single workflow execution",
+ description="Maximum execution time in seconds for a single workflow",
default=1200,
)
WORKFLOW_CALL_MAX_DEPTH: PositiveInt = Field(
- description="max depth of calling in single workflow execution",
+ description="Maximum allowed depth for nested workflow calls",
default=5,
)
MAX_VARIABLE_SIZE: PositiveInt = Field(
- description="The maximum size in bytes of a variable. default to 5KB.",
+ description="Maximum size in bytes for a single variable in workflows. Default to 5KB.",
default=5 * 1024,
)
-class OAuthConfig(BaseSettings):
+class AuthConfig(BaseSettings):
"""
- oauth configs
+ Configuration for authentication and OAuth
"""
OAUTH_REDIRECT_PATH: str = Field(
- description="redirect path for OAuth",
+ description="Redirect path for OAuth authentication callbacks",
default="/console/api/oauth/authorize",
)
GITHUB_CLIENT_ID: Optional[str] = Field(
- description="GitHub client id for OAuth",
+ description="GitHub OAuth client ID",
default=None,
)
GITHUB_CLIENT_SECRET: Optional[str] = Field(
- description="GitHub client secret key for OAuth",
+ description="GitHub OAuth client secret",
default=None,
)
GOOGLE_CLIENT_ID: Optional[str] = Field(
- description="Google client id for OAuth",
+ description="Google OAuth client ID",
default=None,
)
GOOGLE_CLIENT_SECRET: Optional[str] = Field(
- description="Google client secret key for OAuth",
+ description="Google OAuth client secret",
default=None,
)
+ ACCESS_TOKEN_EXPIRE_MINUTES: PositiveInt = Field(
+ description="Expiration time for access tokens in minutes",
+ default=60,
+ )
+
class ModerationConfig(BaseSettings):
"""
- Moderation in app configs.
+ Configuration for content moderation
"""
MODERATION_BUFFER_SIZE: PositiveInt = Field(
- description="buffer size for moderation",
+ description="Size of the buffer for content moderation processing",
default=300,
)
class ToolConfig(BaseSettings):
"""
- Tool configs
+ Configuration for tool management
"""
TOOL_ICON_CACHE_MAX_AGE: PositiveInt = Field(
- description="max age in seconds for tool icon caching",
+ description="Maximum age in seconds for caching tool icons",
default=3600,
)
class MailConfig(BaseSettings):
"""
- Mail Configurations
+ Configuration for email services
"""
MAIL_TYPE: Optional[str] = Field(
- description="Mail provider type name, default to None, available values are `smtp` and `resend`.",
+ description="Email service provider type ('smtp' or 'resend'), default to None.",
default=None,
)
MAIL_DEFAULT_SEND_FROM: Optional[str] = Field(
- description="default email address for sending from ",
+ description="Default email address to use as the sender",
default=None,
)
RESEND_API_KEY: Optional[str] = Field(
- description="API key for Resend",
+ description="API key for Resend email service",
default=None,
)
RESEND_API_URL: Optional[str] = Field(
- description="API URL for Resend",
+ description="API URL for Resend email service",
default=None,
)
SMTP_SERVER: Optional[str] = Field(
- description="smtp server host",
+ description="SMTP server hostname",
default=None,
)
SMTP_PORT: Optional[int] = Field(
- description="smtp server port",
+ description="SMTP server port number",
default=465,
)
SMTP_USERNAME: Optional[str] = Field(
- description="smtp server username",
+ description="Username for SMTP authentication",
default=None,
)
SMTP_PASSWORD: Optional[str] = Field(
- description="smtp server password",
+ description="Password for SMTP authentication",
default=None,
)
SMTP_USE_TLS: bool = Field(
- description="whether to use TLS connection to smtp server",
+ description="Enable TLS encryption for SMTP connections",
default=False,
)
SMTP_OPPORTUNISTIC_TLS: bool = Field(
- description="whether to use opportunistic TLS connection to smtp server",
+ description="Enable opportunistic TLS for SMTP connections",
default=False,
)
class RagEtlConfig(BaseSettings):
"""
- RAG ETL Configurations.
+ Configuration for RAG ETL processes
"""
ETL_TYPE: str = Field(
- description="RAG ETL type name, default to `dify`, available values are `dify` and `Unstructured`. ",
+ description="RAG ETL type ('dify' or 'Unstructured'), default to 'dify'",
default="dify",
)
KEYWORD_DATA_SOURCE_TYPE: str = Field(
- description="source type for keyword data, default to `database`, available values are `database` .",
+ description="Data source type for keyword extraction"
+ " ('database' or other supported types), default to 'database'",
default="database",
)
UNSTRUCTURED_API_URL: Optional[str] = Field(
- description="API URL for Unstructured",
+ description="API URL for Unstructured.io service",
default=None,
)
UNSTRUCTURED_API_KEY: Optional[str] = Field(
- description="API key for Unstructured",
+ description="API key for Unstructured.io service",
default=None,
)
class DataSetConfig(BaseSettings):
"""
- Dataset configs
+ Configuration for dataset management
"""
CLEAN_DAY_SETTING: PositiveInt = Field(
- description="interval in days for cleaning up dataset",
+ description="Interval in days for dataset cleanup operations",
default=30,
)
DATASET_OPERATOR_ENABLED: bool = Field(
- description="whether to enable dataset operator",
+ description="Enable or disable dataset operator functionality",
default=False,
)
class WorkspaceConfig(BaseSettings):
"""
- Workspace configs
+ Configuration for workspace management
"""
INVITE_EXPIRY_HOURS: PositiveInt = Field(
- description="workspaces invitation expiration in hours",
+ description="Expiration time in hours for workspace invitation links",
default=72,
)
class IndexingConfig(BaseSettings):
"""
- Indexing configs.
+ Configuration for indexing operations
"""
INDEXING_MAX_SEGMENTATION_TOKENS_LENGTH: PositiveInt = Field(
- description="max segmentation token length for indexing",
+ description="Maximum token length for text segmentation during indexing",
default=1000,
)
class ImageFormatConfig(BaseSettings):
MULTIMODAL_SEND_IMAGE_FORMAT: str = Field(
- description="multi model send image format, support base64, url, default is base64",
+ description="Format for sending images in multimodal contexts ('base64' or 'url'), default is base64",
default="base64",
)
class CeleryBeatConfig(BaseSettings):
CELERY_BEAT_SCHEDULER_TIME: int = Field(
- description="the time of the celery scheduler, default to 1 day",
+ description="Interval in days for Celery Beat scheduler execution, default to 1 day",
default=1,
)
class PositionConfig(BaseSettings):
POSITION_PROVIDER_PINS: str = Field(
- description="The heads of model providers",
+ description="Comma-separated list of pinned model providers",
default="",
)
POSITION_PROVIDER_INCLUDES: str = Field(
- description="The included model providers",
+ description="Comma-separated list of included model providers",
default="",
)
POSITION_PROVIDER_EXCLUDES: str = Field(
- description="The excluded model providers",
+ description="Comma-separated list of excluded model providers",
default="",
)
POSITION_TOOL_PINS: str = Field(
- description="The heads of tools",
+ description="Comma-separated list of pinned tools",
default="",
)
POSITION_TOOL_INCLUDES: str = Field(
- description="The included tools",
+ description="Comma-separated list of included tools",
default="",
)
POSITION_TOOL_EXCLUDES: str = Field(
- description="The excluded tools",
+ description="Comma-separated list of excluded tools",
default="",
)
@@ -602,6 +612,7 @@ def POSITION_TOOL_EXCLUDES_SET(self) -> set[str]:
class FeatureConfig(
# place the configs in alphabet order
AppExecutionConfig,
+ AuthConfig, # Changed from OAuthConfig to AuthConfig
BillingConfig,
CodeExecutionSandboxConfig,
DataSetConfig,
@@ -616,14 +627,13 @@ class FeatureConfig(
MailConfig,
ModelLoadBalanceConfig,
ModerationConfig,
- OAuthConfig,
+ PositionConfig,
RagEtlConfig,
SecurityConfig,
ToolConfig,
UpdateConfig,
WorkflowConfig,
WorkspaceConfig,
- PositionConfig,
# hosted services config
HostedServiceConfig,
CeleryBeatConfig,
diff --git a/api/configs/feature/hosted_service/__init__.py b/api/configs/feature/hosted_service/__init__.py
index f269d0ab9c89c..7f103be8f4f90 100644
--- a/api/configs/feature/hosted_service/__init__.py
+++ b/api/configs/feature/hosted_service/__init__.py
@@ -6,31 +6,31 @@
class HostedOpenAiConfig(BaseSettings):
"""
- Hosted OpenAI service config
+ Configuration for hosted OpenAI service
"""
HOSTED_OPENAI_API_KEY: Optional[str] = Field(
- description="",
+ description="API key for hosted OpenAI service",
default=None,
)
HOSTED_OPENAI_API_BASE: Optional[str] = Field(
- description="",
+ description="Base URL for hosted OpenAI API",
default=None,
)
HOSTED_OPENAI_API_ORGANIZATION: Optional[str] = Field(
- description="",
+ description="Organization ID for hosted OpenAI service",
default=None,
)
HOSTED_OPENAI_TRIAL_ENABLED: bool = Field(
- description="",
+ description="Enable trial access to hosted OpenAI service",
default=False,
)
HOSTED_OPENAI_TRIAL_MODELS: str = Field(
- description="",
+ description="Comma-separated list of available models for trial access",
default="gpt-3.5-turbo,"
"gpt-3.5-turbo-1106,"
"gpt-3.5-turbo-instruct,"
@@ -42,17 +42,17 @@ class HostedOpenAiConfig(BaseSettings):
)
HOSTED_OPENAI_QUOTA_LIMIT: NonNegativeInt = Field(
- description="",
+ description="Quota limit for hosted OpenAI service usage",
default=200,
)
HOSTED_OPENAI_PAID_ENABLED: bool = Field(
- description="",
+ description="Enable paid access to hosted OpenAI service",
default=False,
)
HOSTED_OPENAI_PAID_MODELS: str = Field(
- description="",
+ description="Comma-separated list of available models for paid access",
default="gpt-4,"
"gpt-4-turbo-preview,"
"gpt-4-turbo-2024-04-09,"
@@ -71,124 +71,122 @@ class HostedOpenAiConfig(BaseSettings):
class HostedAzureOpenAiConfig(BaseSettings):
"""
- Hosted OpenAI service config
+ Configuration for hosted Azure OpenAI service
"""
HOSTED_AZURE_OPENAI_ENABLED: bool = Field(
- description="",
+ description="Enable hosted Azure OpenAI service",
default=False,
)
HOSTED_AZURE_OPENAI_API_KEY: Optional[str] = Field(
- description="",
+ description="API key for hosted Azure OpenAI service",
default=None,
)
HOSTED_AZURE_OPENAI_API_BASE: Optional[str] = Field(
- description="",
+ description="Base URL for hosted Azure OpenAI API",
default=None,
)
HOSTED_AZURE_OPENAI_QUOTA_LIMIT: NonNegativeInt = Field(
- description="",
+ description="Quota limit for hosted Azure OpenAI service usage",
default=200,
)
class HostedAnthropicConfig(BaseSettings):
"""
- Hosted Azure OpenAI service config
+ Configuration for hosted Anthropic service
"""
HOSTED_ANTHROPIC_API_BASE: Optional[str] = Field(
- description="",
+ description="Base URL for hosted Anthropic API",
default=None,
)
HOSTED_ANTHROPIC_API_KEY: Optional[str] = Field(
- description="",
+ description="API key for hosted Anthropic service",
default=None,
)
HOSTED_ANTHROPIC_TRIAL_ENABLED: bool = Field(
- description="",
+ description="Enable trial access to hosted Anthropic service",
default=False,
)
HOSTED_ANTHROPIC_QUOTA_LIMIT: NonNegativeInt = Field(
- description="",
+ description="Quota limit for hosted Anthropic service usage",
default=600000,
)
HOSTED_ANTHROPIC_PAID_ENABLED: bool = Field(
- description="",
+ description="Enable paid access to hosted Anthropic service",
default=False,
)
class HostedMinmaxConfig(BaseSettings):
"""
- Hosted Minmax service config
+ Configuration for hosted Minmax service
"""
HOSTED_MINIMAX_ENABLED: bool = Field(
- description="",
+ description="Enable hosted Minmax service",
default=False,
)
class HostedSparkConfig(BaseSettings):
"""
- Hosted Spark service config
+ Configuration for hosted Spark service
"""
HOSTED_SPARK_ENABLED: bool = Field(
- description="",
+ description="Enable hosted Spark service",
default=False,
)
class HostedZhipuAIConfig(BaseSettings):
"""
- Hosted Minmax service config
+ Configuration for hosted ZhipuAI service
"""
HOSTED_ZHIPUAI_ENABLED: bool = Field(
- description="",
+ description="Enable hosted ZhipuAI service",
default=False,
)
class HostedModerationConfig(BaseSettings):
"""
- Hosted Moderation service config
+ Configuration for hosted Moderation service
"""
HOSTED_MODERATION_ENABLED: bool = Field(
- description="",
+ description="Enable hosted Moderation service",
default=False,
)
HOSTED_MODERATION_PROVIDERS: str = Field(
- description="",
+ description="Comma-separated list of moderation providers",
default="",
)
class HostedFetchAppTemplateConfig(BaseSettings):
"""
- Hosted Moderation service config
+ Configuration for fetching app templates
"""
HOSTED_FETCH_APP_TEMPLATES_MODE: str = Field(
- description="the mode for fetching app templates,"
- " default to remote,"
- " available values: remote, db, builtin",
+ description="Mode for fetching app templates: remote, db, or builtin" " default to remote,",
default="remote",
)
HOSTED_FETCH_APP_TEMPLATES_REMOTE_DOMAIN: str = Field(
- description="the domain for fetching remote app templates",
+ description="Domain for fetching remote app templates",
default="https://tmpl.dify.ai",
)
diff --git a/api/configs/middleware/__init__.py b/api/configs/middleware/__init__.py
index e017c2c5b828e..fa7f41d630965 100644
--- a/api/configs/middleware/__init__.py
+++ b/api/configs/middleware/__init__.py
@@ -8,9 +8,11 @@
from configs.middleware.storage.aliyun_oss_storage_config import AliyunOSSStorageConfig
from configs.middleware.storage.amazon_s3_storage_config import S3StorageConfig
from configs.middleware.storage.azure_blob_storage_config import AzureBlobStorageConfig
+from configs.middleware.storage.baidu_obs_storage_config import BaiduOBSStorageConfig
from configs.middleware.storage.google_cloud_storage_config import GoogleCloudStorageConfig
from configs.middleware.storage.huawei_obs_storage_config import HuaweiCloudOBSStorageConfig
from configs.middleware.storage.oci_storage_config import OCIStorageConfig
+from configs.middleware.storage.supabase_storage_config import SupabaseStorageConfig
from configs.middleware.storage.tencent_cos_storage_config import TencentCloudCOSStorageConfig
from configs.middleware.storage.volcengine_tos_storage_config import VolcengineTOSStorageConfig
from configs.middleware.vdb.analyticdb_config import AnalyticdbConfig
@@ -26,75 +28,77 @@
from configs.middleware.vdb.relyt_config import RelytConfig
from configs.middleware.vdb.tencent_vector_config import TencentVectorDBConfig
from configs.middleware.vdb.tidb_vector_config import TiDBVectorConfig
+from configs.middleware.vdb.vikingdb_config import VikingDBConfig
from configs.middleware.vdb.weaviate_config import WeaviateConfig
class StorageConfig(BaseSettings):
STORAGE_TYPE: str = Field(
- description="storage type,"
- " default to `local`,"
- " available values are `local`, `s3`, `azure-blob`, `aliyun-oss`, `google-storage`.",
+ description="Type of storage to use."
+ " Options: 'local', 's3', 'azure-blob', 'aliyun-oss', 'google-storage'. Default is 'local'.",
default="local",
)
STORAGE_LOCAL_PATH: str = Field(
- description="local storage path",
+ description="Path for local storage when STORAGE_TYPE is set to 'local'.",
default="storage",
)
class VectorStoreConfig(BaseSettings):
VECTOR_STORE: Optional[str] = Field(
- description="vector store type",
+ description="Type of vector store to use for efficient similarity search."
+ " Set to None if not using a vector store.",
default=None,
)
class KeywordStoreConfig(BaseSettings):
KEYWORD_STORE: str = Field(
- description="keyword store type",
+ description="Method for keyword extraction and storage."
+ " Default is 'jieba', a Chinese text segmentation library.",
default="jieba",
)
class DatabaseConfig:
DB_HOST: str = Field(
- description="db host",
+ description="Hostname or IP address of the database server.",
default="localhost",
)
DB_PORT: PositiveInt = Field(
- description="db port",
+ description="Port number for database connection.",
default=5432,
)
DB_USERNAME: str = Field(
- description="db username",
+ description="Username for database authentication.",
default="postgres",
)
DB_PASSWORD: str = Field(
- description="db password",
+ description="Password for database authentication.",
default="",
)
DB_DATABASE: str = Field(
- description="db database",
+ description="Name of the database to connect to.",
default="dify",
)
DB_CHARSET: str = Field(
- description="db charset",
+ description="Character set for database connection.",
default="",
)
DB_EXTRAS: str = Field(
- description="db extras options. Example: keepalives_idle=60&keepalives=1",
+ description="Additional database connection parameters. Example: 'keepalives_idle=60&keepalives=1'",
default="",
)
SQLALCHEMY_DATABASE_URI_SCHEME: str = Field(
- description="db uri scheme",
+ description="Database URI scheme for SQLAlchemy connection.",
default="postgresql",
)
@@ -112,27 +116,27 @@ def SQLALCHEMY_DATABASE_URI(self) -> str:
)
SQLALCHEMY_POOL_SIZE: NonNegativeInt = Field(
- description="pool size of SqlAlchemy",
+ description="Maximum number of database connections in the pool.",
default=30,
)
SQLALCHEMY_MAX_OVERFLOW: NonNegativeInt = Field(
- description="max overflows for SqlAlchemy",
+ description="Maximum number of connections that can be created beyond the pool_size.",
default=10,
)
SQLALCHEMY_POOL_RECYCLE: NonNegativeInt = Field(
- description="SqlAlchemy pool recycle",
+ description="Number of seconds after which a connection is automatically recycled.",
default=3600,
)
SQLALCHEMY_POOL_PRE_PING: bool = Field(
- description="whether to enable pool pre-ping in SqlAlchemy",
+ description="If True, enables connection pool pre-ping feature to check connections.",
default=False,
)
SQLALCHEMY_ECHO: bool | str = Field(
- description="whether to enable SqlAlchemy echo",
+ description="If True, SQLAlchemy will log all SQL statements.",
default=False,
)
@@ -150,27 +154,27 @@ def SQLALCHEMY_ENGINE_OPTIONS(self) -> dict[str, Any]:
class CeleryConfig(DatabaseConfig):
CELERY_BACKEND: str = Field(
- description="Celery backend, available values are `database`, `redis`",
+ description="Backend for Celery task results. Options: 'database', 'redis'.",
default="database",
)
CELERY_BROKER_URL: Optional[str] = Field(
- description="CELERY_BROKER_URL",
+ description="URL of the message broker for Celery tasks.",
default=None,
)
CELERY_USE_SENTINEL: Optional[bool] = Field(
- description="Whether to use Redis Sentinel mode",
+ description="Whether to use Redis Sentinel for high availability.",
default=False,
)
CELERY_SENTINEL_MASTER_NAME: Optional[str] = Field(
- description="Redis Sentinel master name",
+ description="Name of the Redis Sentinel master.",
default=None,
)
CELERY_SENTINEL_SOCKET_TIMEOUT: Optional[PositiveFloat] = Field(
- description="Redis Sentinel socket timeout",
+ description="Timeout for Redis Sentinel socket operations in seconds.",
default=0.1,
)
@@ -189,6 +193,22 @@ def BROKER_USE_SSL(self) -> bool:
return self.CELERY_BROKER_URL.startswith("rediss://") if self.CELERY_BROKER_URL else False
+class InternalTestConfig(BaseSettings):
+ """
+ Configuration settings for Internal Test
+ """
+
+ AWS_SECRET_ACCESS_KEY: Optional[str] = Field(
+ description="Internal test AWS secret access key",
+ default=None,
+ )
+
+ AWS_ACCESS_KEY_ID: Optional[str] = Field(
+ description="Internal test AWS access key ID",
+ default=None,
+ )
+
+
class MiddlewareConfig(
# place the configs in alphabet order
CeleryConfig,
@@ -199,12 +219,14 @@ class MiddlewareConfig(
StorageConfig,
AliyunOSSStorageConfig,
AzureBlobStorageConfig,
+ BaiduOBSStorageConfig,
GoogleCloudStorageConfig,
- TencentCloudCOSStorageConfig,
HuaweiCloudOBSStorageConfig,
- VolcengineTOSStorageConfig,
- S3StorageConfig,
OCIStorageConfig,
+ S3StorageConfig,
+ SupabaseStorageConfig,
+ TencentCloudCOSStorageConfig,
+ VolcengineTOSStorageConfig,
# configs of vdb and vdb providers
VectorStoreConfig,
AnalyticdbConfig,
@@ -221,5 +243,7 @@ class MiddlewareConfig(
TiDBVectorConfig,
WeaviateConfig,
ElasticsearchConfig,
+ InternalTestConfig,
+ VikingDBConfig,
):
pass
diff --git a/api/configs/middleware/cache/redis_config.py b/api/configs/middleware/cache/redis_config.py
index 4fcd52ddc9cbc..26b9b1347c61b 100644
--- a/api/configs/middleware/cache/redis_config.py
+++ b/api/configs/middleware/cache/redis_config.py
@@ -6,65 +6,65 @@
class RedisConfig(BaseSettings):
"""
- Redis configs
+ Configuration settings for Redis connection
"""
REDIS_HOST: str = Field(
- description="Redis host",
+ description="Hostname or IP address of the Redis server",
default="localhost",
)
REDIS_PORT: PositiveInt = Field(
- description="Redis port",
+ description="Port number on which the Redis server is listening",
default=6379,
)
REDIS_USERNAME: Optional[str] = Field(
- description="Redis username",
+ description="Username for Redis authentication (if required)",
default=None,
)
REDIS_PASSWORD: Optional[str] = Field(
- description="Redis password",
+ description="Password for Redis authentication (if required)",
default=None,
)
REDIS_DB: NonNegativeInt = Field(
- description="Redis database id, default to 0",
+ description="Redis database number to use (0-15)",
default=0,
)
REDIS_USE_SSL: bool = Field(
- description="whether to use SSL for Redis connection",
+ description="Enable SSL/TLS for the Redis connection",
default=False,
)
REDIS_USE_SENTINEL: Optional[bool] = Field(
- description="Whether to use Redis Sentinel mode",
+ description="Enable Redis Sentinel mode for high availability",
default=False,
)
REDIS_SENTINELS: Optional[str] = Field(
- description="Redis Sentinel nodes",
+ description="Comma-separated list of Redis Sentinel nodes (host:port)",
default=None,
)
REDIS_SENTINEL_SERVICE_NAME: Optional[str] = Field(
- description="Redis Sentinel service name",
+ description="Name of the Redis Sentinel service to monitor",
default=None,
)
REDIS_SENTINEL_USERNAME: Optional[str] = Field(
- description="Redis Sentinel username",
+ description="Username for Redis Sentinel authentication (if required)",
default=None,
)
REDIS_SENTINEL_PASSWORD: Optional[str] = Field(
- description="Redis Sentinel password",
+ description="Password for Redis Sentinel authentication (if required)",
default=None,
)
REDIS_SENTINEL_SOCKET_TIMEOUT: Optional[PositiveFloat] = Field(
- description="Redis Sentinel socket timeout",
+ description="Socket timeout in seconds for Redis Sentinel connections",
default=0.1,
)
diff --git a/api/configs/middleware/storage/aliyun_oss_storage_config.py b/api/configs/middleware/storage/aliyun_oss_storage_config.py
index c1843dc26cee0..07eb527170b2e 100644
--- a/api/configs/middleware/storage/aliyun_oss_storage_config.py
+++ b/api/configs/middleware/storage/aliyun_oss_storage_config.py
@@ -6,40 +6,40 @@
class AliyunOSSStorageConfig(BaseSettings):
"""
- Aliyun storage configs
+ Configuration settings for Aliyun Object Storage Service (OSS)
"""
ALIYUN_OSS_BUCKET_NAME: Optional[str] = Field(
- description="Aliyun OSS bucket name",
+ description="Name of the Aliyun OSS bucket to store and retrieve objects",
default=None,
)
ALIYUN_OSS_ACCESS_KEY: Optional[str] = Field(
- description="Aliyun OSS access key",
+ description="Access key ID for authenticating with Aliyun OSS",
default=None,
)
ALIYUN_OSS_SECRET_KEY: Optional[str] = Field(
- description="Aliyun OSS secret key",
+ description="Secret access key for authenticating with Aliyun OSS",
default=None,
)
ALIYUN_OSS_ENDPOINT: Optional[str] = Field(
- description="Aliyun OSS endpoint URL",
+ description="URL of the Aliyun OSS endpoint for your chosen region",
default=None,
)
ALIYUN_OSS_REGION: Optional[str] = Field(
- description="Aliyun OSS region",
+ description="Aliyun OSS region where your bucket is located (e.g., 'oss-cn-hangzhou')",
default=None,
)
ALIYUN_OSS_AUTH_VERSION: Optional[str] = Field(
- description="Aliyun OSS authentication version",
+ description="Version of the authentication protocol to use with Aliyun OSS (e.g., 'v4')",
default=None,
)
ALIYUN_OSS_PATH: Optional[str] = Field(
- description="Aliyun OSS path",
+ description="Base path within the bucket to store objects (e.g., 'my-app-data/')",
default=None,
)
diff --git a/api/configs/middleware/storage/amazon_s3_storage_config.py b/api/configs/middleware/storage/amazon_s3_storage_config.py
index bef9326108709..f2d94b12ffa97 100644
--- a/api/configs/middleware/storage/amazon_s3_storage_config.py
+++ b/api/configs/middleware/storage/amazon_s3_storage_config.py
@@ -6,40 +6,40 @@
class S3StorageConfig(BaseSettings):
"""
- S3 storage configs
+ Configuration settings for S3-compatible object storage
"""
S3_ENDPOINT: Optional[str] = Field(
- description="S3 storage endpoint",
+ description="URL of the S3-compatible storage endpoint (e.g., 'https://s3.amazonaws.com')",
default=None,
)
S3_REGION: Optional[str] = Field(
- description="S3 storage region",
+ description="Region where the S3 bucket is located (e.g., 'us-east-1')",
default=None,
)
S3_BUCKET_NAME: Optional[str] = Field(
- description="S3 storage bucket name",
+ description="Name of the S3 bucket to store and retrieve objects",
default=None,
)
S3_ACCESS_KEY: Optional[str] = Field(
- description="S3 storage access key",
+ description="Access key ID for authenticating with the S3 service",
default=None,
)
S3_SECRET_KEY: Optional[str] = Field(
- description="S3 storage secret key",
+ description="Secret access key for authenticating with the S3 service",
default=None,
)
S3_ADDRESS_STYLE: str = Field(
- description="S3 storage address style",
+ description="S3 addressing style: 'auto', 'path', or 'virtual'",
default="auto",
)
S3_USE_AWS_MANAGED_IAM: bool = Field(
- description="whether to use aws managed IAM for S3",
+ description="Use AWS managed IAM roles for authentication instead of access/secret keys",
default=False,
)
diff --git a/api/configs/middleware/storage/azure_blob_storage_config.py b/api/configs/middleware/storage/azure_blob_storage_config.py
index 10944b58eddc6..b7ab5247a9d4d 100644
--- a/api/configs/middleware/storage/azure_blob_storage_config.py
+++ b/api/configs/middleware/storage/azure_blob_storage_config.py
@@ -6,25 +6,25 @@
class AzureBlobStorageConfig(BaseSettings):
"""
- Azure Blob storage configs
+ Configuration settings for Azure Blob Storage
"""
AZURE_BLOB_ACCOUNT_NAME: Optional[str] = Field(
- description="Azure Blob account name",
+ description="Name of the Azure Storage account (e.g., 'mystorageaccount')",
default=None,
)
AZURE_BLOB_ACCOUNT_KEY: Optional[str] = Field(
- description="Azure Blob account key",
+ description="Access key for authenticating with the Azure Storage account",
default=None,
)
AZURE_BLOB_CONTAINER_NAME: Optional[str] = Field(
- description="Azure Blob container name",
+ description="Name of the Azure Blob container to store and retrieve objects",
default=None,
)
AZURE_BLOB_ACCOUNT_URL: Optional[str] = Field(
- description="Azure Blob account URL",
+ description="URL of the Azure Blob storage endpoint (e.g., 'https://mystorageaccount.blob.core.windows.net')",
default=None,
)
diff --git a/api/configs/middleware/storage/baidu_obs_storage_config.py b/api/configs/middleware/storage/baidu_obs_storage_config.py
new file mode 100644
index 0000000000000..c511628a1514a
--- /dev/null
+++ b/api/configs/middleware/storage/baidu_obs_storage_config.py
@@ -0,0 +1,29 @@
+from typing import Optional
+
+from pydantic import BaseModel, Field
+
+
+class BaiduOBSStorageConfig(BaseModel):
+ """
+ Configuration settings for Baidu Object Storage Service (OBS)
+ """
+
+ BAIDU_OBS_BUCKET_NAME: Optional[str] = Field(
+ description="Name of the Baidu OBS bucket to store and retrieve objects (e.g., 'my-obs-bucket')",
+ default=None,
+ )
+
+ BAIDU_OBS_ACCESS_KEY: Optional[str] = Field(
+ description="Access Key ID for authenticating with Baidu OBS",
+ default=None,
+ )
+
+ BAIDU_OBS_SECRET_KEY: Optional[str] = Field(
+ description="Secret Access Key for authenticating with Baidu OBS",
+ default=None,
+ )
+
+ BAIDU_OBS_ENDPOINT: Optional[str] = Field(
+ description="URL of the Baidu OSS endpoint for your chosen region (e.g., 'https://.bj.bcebos.com')",
+ default=None,
+ )
diff --git a/api/configs/middleware/storage/google_cloud_storage_config.py b/api/configs/middleware/storage/google_cloud_storage_config.py
index 10a2d97e8dbcd..e5d763d7f5c61 100644
--- a/api/configs/middleware/storage/google_cloud_storage_config.py
+++ b/api/configs/middleware/storage/google_cloud_storage_config.py
@@ -6,15 +6,15 @@
class GoogleCloudStorageConfig(BaseSettings):
"""
- Google Cloud storage configs
+ Configuration settings for Google Cloud Storage
"""
GOOGLE_STORAGE_BUCKET_NAME: Optional[str] = Field(
- description="Google Cloud storage bucket name",
+ description="Name of the Google Cloud Storage bucket to store and retrieve objects (e.g., 'my-gcs-bucket')",
default=None,
)
GOOGLE_STORAGE_SERVICE_ACCOUNT_JSON_BASE64: Optional[str] = Field(
- description="Google Cloud storage service account json base64",
+ description="Base64-encoded JSON key file for Google Cloud service account authentication",
default=None,
)
diff --git a/api/configs/middleware/storage/huawei_obs_storage_config.py b/api/configs/middleware/storage/huawei_obs_storage_config.py
index c5cb379caee3c..3e9e7543ab2ba 100644
--- a/api/configs/middleware/storage/huawei_obs_storage_config.py
+++ b/api/configs/middleware/storage/huawei_obs_storage_config.py
@@ -5,25 +5,25 @@
class HuaweiCloudOBSStorageConfig(BaseModel):
"""
- Huawei Cloud OBS storage configs
+ Configuration settings for Huawei Cloud Object Storage Service (OBS)
"""
HUAWEI_OBS_BUCKET_NAME: Optional[str] = Field(
- description="Huawei Cloud OBS bucket name",
+ description="Name of the Huawei Cloud OBS bucket to store and retrieve objects (e.g., 'my-obs-bucket')",
default=None,
)
HUAWEI_OBS_ACCESS_KEY: Optional[str] = Field(
- description="Huawei Cloud OBS Access key",
+ description="Access Key ID for authenticating with Huawei Cloud OBS",
default=None,
)
HUAWEI_OBS_SECRET_KEY: Optional[str] = Field(
- description="Huawei Cloud OBS Secret key",
+ description="Secret Access Key for authenticating with Huawei Cloud OBS",
default=None,
)
HUAWEI_OBS_SERVER: Optional[str] = Field(
- description="Huawei Cloud OBS server URL",
+ description="Endpoint URL for Huawei Cloud OBS (e.g., 'https://obs.cn-north-4.myhuaweicloud.com')",
default=None,
)
diff --git a/api/configs/middleware/storage/oci_storage_config.py b/api/configs/middleware/storage/oci_storage_config.py
index f8993496c952b..edc245bcac59b 100644
--- a/api/configs/middleware/storage/oci_storage_config.py
+++ b/api/configs/middleware/storage/oci_storage_config.py
@@ -6,30 +6,30 @@
class OCIStorageConfig(BaseSettings):
"""
- OCI storage configs
+ Configuration settings for Oracle Cloud Infrastructure (OCI) Object Storage
"""
OCI_ENDPOINT: Optional[str] = Field(
- description="OCI storage endpoint",
+ description="URL of the OCI Object Storage endpoint (e.g., 'https://objectstorage.us-phoenix-1.oraclecloud.com')",
default=None,
)
OCI_REGION: Optional[str] = Field(
- description="OCI storage region",
+ description="OCI region where the bucket is located (e.g., 'us-phoenix-1')",
default=None,
)
OCI_BUCKET_NAME: Optional[str] = Field(
- description="OCI storage bucket name",
+ description="Name of the OCI Object Storage bucket to store and retrieve objects (e.g., 'my-oci-bucket')",
default=None,
)
OCI_ACCESS_KEY: Optional[str] = Field(
- description="OCI storage access key",
+ description="Access key (also known as API key) for authenticating with OCI Object Storage",
default=None,
)
OCI_SECRET_KEY: Optional[str] = Field(
- description="OCI storage secret key",
+ description="Secret key associated with the access key for authenticating with OCI Object Storage",
default=None,
)
diff --git a/api/configs/middleware/storage/supabase_storage_config.py b/api/configs/middleware/storage/supabase_storage_config.py
new file mode 100644
index 0000000000000..a3e905b21c63e
--- /dev/null
+++ b/api/configs/middleware/storage/supabase_storage_config.py
@@ -0,0 +1,24 @@
+from typing import Optional
+
+from pydantic import BaseModel, Field
+
+
+class SupabaseStorageConfig(BaseModel):
+ """
+ Configuration settings for Supabase Object Storage Service
+ """
+
+ SUPABASE_BUCKET_NAME: Optional[str] = Field(
+ description="Name of the Supabase bucket to store and retrieve objects (e.g., 'dify-bucket')",
+ default=None,
+ )
+
+ SUPABASE_API_KEY: Optional[str] = Field(
+ description="API KEY for authenticating with Supabase",
+ default=None,
+ )
+
+ SUPABASE_URL: Optional[str] = Field(
+ description="URL of the Supabase",
+ default=None,
+ )
diff --git a/api/configs/middleware/storage/tencent_cos_storage_config.py b/api/configs/middleware/storage/tencent_cos_storage_config.py
index 765ac08f3e404..255c4e8938e0f 100644
--- a/api/configs/middleware/storage/tencent_cos_storage_config.py
+++ b/api/configs/middleware/storage/tencent_cos_storage_config.py
@@ -6,30 +6,30 @@
class TencentCloudCOSStorageConfig(BaseSettings):
"""
- Tencent Cloud COS storage configs
+ Configuration settings for Tencent Cloud Object Storage (COS)
"""
TENCENT_COS_BUCKET_NAME: Optional[str] = Field(
- description="Tencent Cloud COS bucket name",
+ description="Name of the Tencent Cloud COS bucket to store and retrieve objects",
default=None,
)
TENCENT_COS_REGION: Optional[str] = Field(
- description="Tencent Cloud COS region",
+ description="Tencent Cloud region where the COS bucket is located (e.g., 'ap-guangzhou')",
default=None,
)
TENCENT_COS_SECRET_ID: Optional[str] = Field(
- description="Tencent Cloud COS secret id",
+ description="SecretId for authenticating with Tencent Cloud COS (part of API credentials)",
default=None,
)
TENCENT_COS_SECRET_KEY: Optional[str] = Field(
- description="Tencent Cloud COS secret key",
+ description="SecretKey for authenticating with Tencent Cloud COS (part of API credentials)",
default=None,
)
TENCENT_COS_SCHEME: Optional[str] = Field(
- description="Tencent Cloud COS scheme",
+ description="Protocol scheme for COS requests: 'https' (recommended) or 'http'",
default=None,
)
diff --git a/api/configs/middleware/storage/volcengine_tos_storage_config.py b/api/configs/middleware/storage/volcengine_tos_storage_config.py
index a0e09a3cc7671..89ea885002300 100644
--- a/api/configs/middleware/storage/volcengine_tos_storage_config.py
+++ b/api/configs/middleware/storage/volcengine_tos_storage_config.py
@@ -5,30 +5,30 @@
class VolcengineTOSStorageConfig(BaseModel):
"""
- Volcengine tos storage configs
+ Configuration settings for Volcengine Tinder Object Storage (TOS)
"""
VOLCENGINE_TOS_BUCKET_NAME: Optional[str] = Field(
- description="Volcengine TOS Bucket Name",
+ description="Name of the Volcengine TOS bucket to store and retrieve objects (e.g., 'my-tos-bucket')",
default=None,
)
VOLCENGINE_TOS_ACCESS_KEY: Optional[str] = Field(
- description="Volcengine TOS Access Key",
+ description="Access Key ID for authenticating with Volcengine TOS",
default=None,
)
VOLCENGINE_TOS_SECRET_KEY: Optional[str] = Field(
- description="Volcengine TOS Secret Key",
+ description="Secret Access Key for authenticating with Volcengine TOS",
default=None,
)
VOLCENGINE_TOS_ENDPOINT: Optional[str] = Field(
- description="Volcengine TOS Endpoint URL",
+ description="URL of the Volcengine TOS endpoint (e.g., 'https://tos-cn-beijing.volces.com')",
default=None,
)
VOLCENGINE_TOS_REGION: Optional[str] = Field(
- description="Volcengine TOS Region",
+ description="Volcengine region where the TOS bucket is located (e.g., 'cn-beijing')",
default=None,
)
diff --git a/api/configs/middleware/vdb/analyticdb_config.py b/api/configs/middleware/vdb/analyticdb_config.py
index 04f5b0e5bf769..247a8ea555948 100644
--- a/api/configs/middleware/vdb/analyticdb_config.py
+++ b/api/configs/middleware/vdb/analyticdb_config.py
@@ -5,33 +5,38 @@
class AnalyticdbConfig(BaseModel):
"""
- Configuration for connecting to AnalyticDB.
+ Configuration for connecting to Alibaba Cloud AnalyticDB for PostgreSQL.
Refer to the following documentation for details on obtaining credentials:
https://www.alibabacloud.com/help/en/analyticdb-for-postgresql/getting-started/create-an-instance-instances-with-vector-engine-optimization-enabled
"""
ANALYTICDB_KEY_ID: Optional[str] = Field(
- default=None, description="The Access Key ID provided by Alibaba Cloud for authentication."
+ default=None, description="The Access Key ID provided by Alibaba Cloud for API authentication."
)
ANALYTICDB_KEY_SECRET: Optional[str] = Field(
- default=None, description="The Secret Access Key corresponding to the Access Key ID for secure access."
+ default=None, description="The Secret Access Key corresponding to the Access Key ID for secure API access."
)
ANALYTICDB_REGION_ID: Optional[str] = Field(
- default=None, description="The region where the AnalyticDB instance is deployed (e.g., 'cn-hangzhou')."
+ default=None,
+ description="The region where the AnalyticDB instance is deployed (e.g., 'cn-hangzhou', 'ap-southeast-1').",
)
ANALYTICDB_INSTANCE_ID: Optional[str] = Field(
default=None,
- description="The unique identifier of the AnalyticDB instance you want to connect to (e.g., 'gp-ab123456')..",
+ description="The unique identifier of the AnalyticDB instance you want to connect to.",
)
ANALYTICDB_ACCOUNT: Optional[str] = Field(
- default=None, description="The account name used to log in to the AnalyticDB instance."
+ default=None,
+ description="The account name used to log in to the AnalyticDB instance"
+ " (usually the initial account created with the instance).",
)
ANALYTICDB_PASSWORD: Optional[str] = Field(
- default=None, description="The password associated with the AnalyticDB account for authentication."
+ default=None, description="The password associated with the AnalyticDB account for database authentication."
)
ANALYTICDB_NAMESPACE: Optional[str] = Field(
- default=None, description="The namespace within AnalyticDB for schema isolation."
+ default=None, description="The namespace within AnalyticDB for schema isolation (if using namespace feature)."
)
ANALYTICDB_NAMESPACE_PASSWORD: Optional[str] = Field(
- default=None, description="The password for accessing the specified namespace within the AnalyticDB instance."
+ default=None,
+ description="The password for accessing the specified namespace within the AnalyticDB instance"
+ " (if namespace feature is enabled).",
)
diff --git a/api/configs/middleware/vdb/baidu_vector_config.py b/api/configs/middleware/vdb/baidu_vector_config.py
new file mode 100644
index 0000000000000..44742c2e2f434
--- /dev/null
+++ b/api/configs/middleware/vdb/baidu_vector_config.py
@@ -0,0 +1,45 @@
+from typing import Optional
+
+from pydantic import Field, NonNegativeInt, PositiveInt
+from pydantic_settings import BaseSettings
+
+
+class BaiduVectorDBConfig(BaseSettings):
+ """
+ Configuration settings for Baidu Vector Database
+ """
+
+ BAIDU_VECTOR_DB_ENDPOINT: Optional[str] = Field(
+ description="URL of the Baidu Vector Database service (e.g., 'http://vdb.bj.baidubce.com')",
+ default=None,
+ )
+
+ BAIDU_VECTOR_DB_CONNECTION_TIMEOUT_MS: PositiveInt = Field(
+ description="Timeout in milliseconds for Baidu Vector Database operations (default is 30000 milliseconds)",
+ default=30000,
+ )
+
+ BAIDU_VECTOR_DB_ACCOUNT: Optional[str] = Field(
+ description="Account for authenticating with the Baidu Vector Database",
+ default=None,
+ )
+
+ BAIDU_VECTOR_DB_API_KEY: Optional[str] = Field(
+ description="API key for authenticating with the Baidu Vector Database service",
+ default=None,
+ )
+
+ BAIDU_VECTOR_DB_DATABASE: Optional[str] = Field(
+ description="Name of the specific Baidu Vector Database to connect to",
+ default=None,
+ )
+
+ BAIDU_VECTOR_DB_SHARD: PositiveInt = Field(
+ description="Number of shards for the Baidu Vector Database (default is 1)",
+ default=1,
+ )
+
+ BAIDU_VECTOR_DB_REPLICAS: NonNegativeInt = Field(
+ description="Number of replicas for the Baidu Vector Database (default is 3)",
+ default=3,
+ )
diff --git a/api/configs/middleware/vdb/chroma_config.py b/api/configs/middleware/vdb/chroma_config.py
index d386623a569d3..e83a9902dee90 100644
--- a/api/configs/middleware/vdb/chroma_config.py
+++ b/api/configs/middleware/vdb/chroma_config.py
@@ -6,35 +6,35 @@
class ChromaConfig(BaseSettings):
"""
- Chroma configs
+ Configuration settings for Chroma vector database
"""
CHROMA_HOST: Optional[str] = Field(
- description="Chroma host",
+ description="Hostname or IP address of the Chroma server (e.g., 'localhost' or '192.168.1.100')",
default=None,
)
CHROMA_PORT: PositiveInt = Field(
- description="Chroma port",
+ description="Port number on which the Chroma server is listening (default is 8000)",
default=8000,
)
CHROMA_TENANT: Optional[str] = Field(
- description="Chroma database",
+ description="Tenant identifier for multi-tenancy support in Chroma",
default=None,
)
CHROMA_DATABASE: Optional[str] = Field(
- description="Chroma database",
+ description="Name of the Chroma database to connect to",
default=None,
)
CHROMA_AUTH_PROVIDER: Optional[str] = Field(
- description="Chroma authentication provider",
+ description="Authentication provider for Chroma (e.g., 'basic', 'token', or a custom provider)",
default=None,
)
CHROMA_AUTH_CREDENTIALS: Optional[str] = Field(
- description="Chroma authentication credentials",
+ description="Authentication credentials for Chroma (format depends on the auth provider)",
default=None,
)
diff --git a/api/configs/middleware/vdb/elasticsearch_config.py b/api/configs/middleware/vdb/elasticsearch_config.py
index 5b6a8fd939c29..df8182985dc19 100644
--- a/api/configs/middleware/vdb/elasticsearch_config.py
+++ b/api/configs/middleware/vdb/elasticsearch_config.py
@@ -6,25 +6,25 @@
class ElasticsearchConfig(BaseSettings):
"""
- Elasticsearch configs
+ Configuration settings for Elasticsearch
"""
ELASTICSEARCH_HOST: Optional[str] = Field(
- description="Elasticsearch host",
+ description="Hostname or IP address of the Elasticsearch server (e.g., 'localhost' or '192.168.1.100')",
default="127.0.0.1",
)
ELASTICSEARCH_PORT: PositiveInt = Field(
- description="Elasticsearch port",
+ description="Port number on which the Elasticsearch server is listening (default is 9200)",
default=9200,
)
ELASTICSEARCH_USERNAME: Optional[str] = Field(
- description="Elasticsearch username",
+ description="Username for authenticating with Elasticsearch (default is 'elastic')",
default="elastic",
)
ELASTICSEARCH_PASSWORD: Optional[str] = Field(
- description="Elasticsearch password",
+ description="Password for authenticating with Elasticsearch (default is 'elastic')",
default="elastic",
)
diff --git a/api/configs/middleware/vdb/milvus_config.py b/api/configs/middleware/vdb/milvus_config.py
index 98d375966a194..231cbbbe8ffc9 100644
--- a/api/configs/middleware/vdb/milvus_config.py
+++ b/api/configs/middleware/vdb/milvus_config.py
@@ -6,30 +6,30 @@
class MilvusConfig(BaseSettings):
"""
- Milvus configs
+ Configuration settings for Milvus vector database
"""
MILVUS_URI: Optional[str] = Field(
- description="Milvus uri",
+ description="URI for connecting to the Milvus server (e.g., 'http://localhost:19530' or 'https://milvus-instance.example.com:19530')",
default="http://127.0.0.1:19530",
)
MILVUS_TOKEN: Optional[str] = Field(
- description="Milvus token",
+ description="Authentication token for Milvus, if token-based authentication is enabled",
default=None,
)
MILVUS_USER: Optional[str] = Field(
- description="Milvus user",
+ description="Username for authenticating with Milvus, if username/password authentication is enabled",
default=None,
)
MILVUS_PASSWORD: Optional[str] = Field(
- description="Milvus password",
+ description="Password for authenticating with Milvus, if username/password authentication is enabled",
default=None,
)
MILVUS_DATABASE: str = Field(
- description="Milvus database, default to `default`",
+ description="Name of the Milvus database to connect to (default is 'default')",
default="default",
)
diff --git a/api/configs/middleware/vdb/myscale_config.py b/api/configs/middleware/vdb/myscale_config.py
index 6451d26e1c65d..5896c19d27d11 100644
--- a/api/configs/middleware/vdb/myscale_config.py
+++ b/api/configs/middleware/vdb/myscale_config.py
@@ -3,35 +3,35 @@
class MyScaleConfig(BaseModel):
"""
- MyScale configs
+ Configuration settings for MyScale vector database
"""
MYSCALE_HOST: str = Field(
- description="MyScale host",
+ description="Hostname or IP address of the MyScale server (e.g., 'localhost' or 'myscale.example.com')",
default="localhost",
)
MYSCALE_PORT: PositiveInt = Field(
- description="MyScale port",
+ description="Port number on which the MyScale server is listening (default is 8123)",
default=8123,
)
MYSCALE_USER: str = Field(
- description="MyScale user",
+ description="Username for authenticating with MyScale (default is 'default')",
default="default",
)
MYSCALE_PASSWORD: str = Field(
- description="MyScale password",
+ description="Password for authenticating with MyScale (default is an empty string)",
default="",
)
MYSCALE_DATABASE: str = Field(
- description="MyScale database name",
+ description="Name of the MyScale database to connect to (default is 'default')",
default="default",
)
MYSCALE_FTS_PARAMS: str = Field(
- description="MyScale fts index parameters",
+ description="Additional parameters for MyScale Full Text Search index)",
default="",
)
diff --git a/api/configs/middleware/vdb/opensearch_config.py b/api/configs/middleware/vdb/opensearch_config.py
index 5823dc1433d83..81dde4c04d472 100644
--- a/api/configs/middleware/vdb/opensearch_config.py
+++ b/api/configs/middleware/vdb/opensearch_config.py
@@ -6,30 +6,30 @@
class OpenSearchConfig(BaseSettings):
"""
- OpenSearch configs
+ Configuration settings for OpenSearch
"""
OPENSEARCH_HOST: Optional[str] = Field(
- description="OpenSearch host",
+ description="Hostname or IP address of the OpenSearch server (e.g., 'localhost' or 'opensearch.example.com')",
default=None,
)
OPENSEARCH_PORT: PositiveInt = Field(
- description="OpenSearch port",
+ description="Port number on which the OpenSearch server is listening (default is 9200)",
default=9200,
)
OPENSEARCH_USER: Optional[str] = Field(
- description="OpenSearch user",
+ description="Username for authenticating with OpenSearch",
default=None,
)
OPENSEARCH_PASSWORD: Optional[str] = Field(
- description="OpenSearch password",
+ description="Password for authenticating with OpenSearch",
default=None,
)
OPENSEARCH_SECURE: bool = Field(
- description="whether to use SSL connection for OpenSearch",
+ description="Whether to use SSL/TLS encrypted connection for OpenSearch (True for HTTPS, False for HTTP)",
default=False,
)
diff --git a/api/configs/middleware/vdb/oracle_config.py b/api/configs/middleware/vdb/oracle_config.py
index 62614ae870cc8..44e2f133458f1 100644
--- a/api/configs/middleware/vdb/oracle_config.py
+++ b/api/configs/middleware/vdb/oracle_config.py
@@ -6,30 +6,30 @@
class OracleConfig(BaseSettings):
"""
- ORACLE configs
+ Configuration settings for Oracle database
"""
ORACLE_HOST: Optional[str] = Field(
- description="ORACLE host",
+ description="Hostname or IP address of the Oracle database server (e.g., 'localhost' or 'oracle.example.com')",
default=None,
)
ORACLE_PORT: Optional[PositiveInt] = Field(
- description="ORACLE port",
+ description="Port number on which the Oracle database server is listening (default is 1521)",
default=1521,
)
ORACLE_USER: Optional[str] = Field(
- description="ORACLE user",
+ description="Username for authenticating with the Oracle database",
default=None,
)
ORACLE_PASSWORD: Optional[str] = Field(
- description="ORACLE password",
+ description="Password for authenticating with the Oracle database",
default=None,
)
ORACLE_DATABASE: Optional[str] = Field(
- description="ORACLE database",
+ description="Name of the Oracle database or service to connect to (e.g., 'ORCL' or 'pdborcl')",
default=None,
)
diff --git a/api/configs/middleware/vdb/pgvector_config.py b/api/configs/middleware/vdb/pgvector_config.py
index 39a7c1d8d5e4a..85f5dca7e2322 100644
--- a/api/configs/middleware/vdb/pgvector_config.py
+++ b/api/configs/middleware/vdb/pgvector_config.py
@@ -6,30 +6,40 @@
class PGVectorConfig(BaseSettings):
"""
- PGVector configs
+ Configuration settings for PGVector (PostgreSQL with vector extension)
"""
PGVECTOR_HOST: Optional[str] = Field(
- description="PGVector host",
+ description="Hostname or IP address of the PostgreSQL server with PGVector extension (e.g., 'localhost')",
default=None,
)
PGVECTOR_PORT: Optional[PositiveInt] = Field(
- description="PGVector port",
+ description="Port number on which the PostgreSQL server is listening (default is 5433)",
default=5433,
)
PGVECTOR_USER: Optional[str] = Field(
- description="PGVector user",
+ description="Username for authenticating with the PostgreSQL database",
default=None,
)
PGVECTOR_PASSWORD: Optional[str] = Field(
- description="PGVector password",
+ description="Password for authenticating with the PostgreSQL database",
default=None,
)
PGVECTOR_DATABASE: Optional[str] = Field(
- description="PGVector database",
+ description="Name of the PostgreSQL database to connect to",
default=None,
)
+
+ PGVECTOR_MIN_CONNECTION: PositiveInt = Field(
+ description="Min connection of the PostgreSQL database",
+ default=1,
+ )
+
+ PGVECTOR_MAX_CONNECTION: PositiveInt = Field(
+ description="Max connection of the PostgreSQL database",
+ default=5,
+ )
diff --git a/api/configs/middleware/vdb/pgvectors_config.py b/api/configs/middleware/vdb/pgvectors_config.py
index c40e5ff92103f..8d7a4b8d25511 100644
--- a/api/configs/middleware/vdb/pgvectors_config.py
+++ b/api/configs/middleware/vdb/pgvectors_config.py
@@ -6,30 +6,30 @@
class PGVectoRSConfig(BaseSettings):
"""
- PGVectoRS configs
+ Configuration settings for PGVecto.RS (Rust-based vector extension for PostgreSQL)
"""
PGVECTO_RS_HOST: Optional[str] = Field(
- description="PGVectoRS host",
+ description="Hostname or IP address of the PostgreSQL server with PGVecto.RS extension (e.g., 'localhost')",
default=None,
)
PGVECTO_RS_PORT: Optional[PositiveInt] = Field(
- description="PGVectoRS port",
+ description="Port number on which the PostgreSQL server with PGVecto.RS is listening (default is 5431)",
default=5431,
)
PGVECTO_RS_USER: Optional[str] = Field(
- description="PGVectoRS user",
+ description="Username for authenticating with the PostgreSQL database using PGVecto.RS",
default=None,
)
PGVECTO_RS_PASSWORD: Optional[str] = Field(
- description="PGVectoRS password",
+ description="Password for authenticating with the PostgreSQL database using PGVecto.RS",
default=None,
)
PGVECTO_RS_DATABASE: Optional[str] = Field(
- description="PGVectoRS database",
+ description="Name of the PostgreSQL database with PGVecto.RS extension to connect to",
default=None,
)
diff --git a/api/configs/middleware/vdb/qdrant_config.py b/api/configs/middleware/vdb/qdrant_config.py
index 27f75491c9876..b70f6246523c5 100644
--- a/api/configs/middleware/vdb/qdrant_config.py
+++ b/api/configs/middleware/vdb/qdrant_config.py
@@ -6,30 +6,30 @@
class QdrantConfig(BaseSettings):
"""
- Qdrant configs
+ Configuration settings for Qdrant vector database
"""
QDRANT_URL: Optional[str] = Field(
- description="Qdrant url",
+ description="URL of the Qdrant server (e.g., 'http://localhost:6333' or 'https://qdrant.example.com')",
default=None,
)
QDRANT_API_KEY: Optional[str] = Field(
- description="Qdrant api key",
+ description="API key for authenticating with the Qdrant server",
default=None,
)
QDRANT_CLIENT_TIMEOUT: NonNegativeInt = Field(
- description="Qdrant client timeout in seconds",
+ description="Timeout in seconds for Qdrant client operations (default is 20 seconds)",
default=20,
)
QDRANT_GRPC_ENABLED: bool = Field(
- description="whether enable grpc support for Qdrant connection",
+ description="Whether to enable gRPC support for Qdrant connection (True for gRPC, False for HTTP)",
default=False,
)
QDRANT_GRPC_PORT: PositiveInt = Field(
- description="Qdrant grpc port",
+ description="Port number for gRPC connection to Qdrant server (default is 6334)",
default=6334,
)
diff --git a/api/configs/middleware/vdb/relyt_config.py b/api/configs/middleware/vdb/relyt_config.py
index 66b9ecc03f6aa..5ffbea7b19bb8 100644
--- a/api/configs/middleware/vdb/relyt_config.py
+++ b/api/configs/middleware/vdb/relyt_config.py
@@ -6,30 +6,30 @@
class RelytConfig(BaseSettings):
"""
- Relyt configs
+ Configuration settings for Relyt database
"""
RELYT_HOST: Optional[str] = Field(
- description="Relyt host",
+ description="Hostname or IP address of the Relyt server (e.g., 'localhost' or 'relyt.example.com')",
default=None,
)
RELYT_PORT: PositiveInt = Field(
- description="Relyt port",
+ description="Port number on which the Relyt server is listening (default is 9200)",
default=9200,
)
RELYT_USER: Optional[str] = Field(
- description="Relyt user",
+ description="Username for authenticating with the Relyt database",
default=None,
)
RELYT_PASSWORD: Optional[str] = Field(
- description="Relyt password",
+ description="Password for authenticating with the Relyt database",
default=None,
)
RELYT_DATABASE: Optional[str] = Field(
- description="Relyt database",
+ description="Name of the Relyt database to connect to (default is 'default')",
default="default",
)
diff --git a/api/configs/middleware/vdb/tencent_vector_config.py b/api/configs/middleware/vdb/tencent_vector_config.py
index 46b4cb6a24ff8..9cf4d07f6fe66 100644
--- a/api/configs/middleware/vdb/tencent_vector_config.py
+++ b/api/configs/middleware/vdb/tencent_vector_config.py
@@ -6,45 +6,45 @@
class TencentVectorDBConfig(BaseSettings):
"""
- Tencent Vector configs
+ Configuration settings for Tencent Vector Database
"""
TENCENT_VECTOR_DB_URL: Optional[str] = Field(
- description="Tencent Vector URL",
+ description="URL of the Tencent Vector Database service (e.g., 'https://vectordb.tencentcloudapi.com')",
default=None,
)
TENCENT_VECTOR_DB_API_KEY: Optional[str] = Field(
- description="Tencent Vector API key",
+ description="API key for authenticating with the Tencent Vector Database service",
default=None,
)
TENCENT_VECTOR_DB_TIMEOUT: PositiveInt = Field(
- description="Tencent Vector timeout in seconds",
+ description="Timeout in seconds for Tencent Vector Database operations (default is 30 seconds)",
default=30,
)
TENCENT_VECTOR_DB_USERNAME: Optional[str] = Field(
- description="Tencent Vector username",
+ description="Username for authenticating with the Tencent Vector Database (if required)",
default=None,
)
TENCENT_VECTOR_DB_PASSWORD: Optional[str] = Field(
- description="Tencent Vector password",
+ description="Password for authenticating with the Tencent Vector Database (if required)",
default=None,
)
TENCENT_VECTOR_DB_SHARD: PositiveInt = Field(
- description="Tencent Vector sharding number",
+ description="Number of shards for the Tencent Vector Database (default is 1)",
default=1,
)
TENCENT_VECTOR_DB_REPLICAS: NonNegativeInt = Field(
- description="Tencent Vector replicas",
+ description="Number of replicas for the Tencent Vector Database (default is 2)",
default=2,
)
TENCENT_VECTOR_DB_DATABASE: Optional[str] = Field(
- description="Tencent Vector Database",
+ description="Name of the specific Tencent Vector Database to connect to",
default=None,
)
diff --git a/api/configs/middleware/vdb/tidb_vector_config.py b/api/configs/middleware/vdb/tidb_vector_config.py
index dbcb276c01a1f..bc68be69d86ad 100644
--- a/api/configs/middleware/vdb/tidb_vector_config.py
+++ b/api/configs/middleware/vdb/tidb_vector_config.py
@@ -6,30 +6,30 @@
class TiDBVectorConfig(BaseSettings):
"""
- TiDB Vector configs
+ Configuration settings for TiDB Vector database
"""
TIDB_VECTOR_HOST: Optional[str] = Field(
- description="TiDB Vector host",
+ description="Hostname or IP address of the TiDB Vector server (e.g., 'localhost' or 'tidb.example.com')",
default=None,
)
TIDB_VECTOR_PORT: Optional[PositiveInt] = Field(
- description="TiDB Vector port",
+ description="Port number on which the TiDB Vector server is listening (default is 4000)",
default=4000,
)
TIDB_VECTOR_USER: Optional[str] = Field(
- description="TiDB Vector user",
+ description="Username for authenticating with the TiDB Vector database",
default=None,
)
TIDB_VECTOR_PASSWORD: Optional[str] = Field(
- description="TiDB Vector password",
+ description="Password for authenticating with the TiDB Vector database",
default=None,
)
TIDB_VECTOR_DATABASE: Optional[str] = Field(
- description="TiDB Vector database",
+ description="Name of the TiDB Vector database to connect to",
default=None,
)
diff --git a/api/configs/middleware/vdb/vikingdb_config.py b/api/configs/middleware/vdb/vikingdb_config.py
new file mode 100644
index 0000000000000..5ad98d898a16e
--- /dev/null
+++ b/api/configs/middleware/vdb/vikingdb_config.py
@@ -0,0 +1,37 @@
+from typing import Optional
+
+from pydantic import BaseModel, Field
+
+
+class VikingDBConfig(BaseModel):
+ """
+ Configuration for connecting to Volcengine VikingDB.
+ Refer to the following documentation for details on obtaining credentials:
+ https://www.volcengine.com/docs/6291/65568
+ """
+
+ VIKINGDB_ACCESS_KEY: Optional[str] = Field(
+ default=None, description="The Access Key provided by Volcengine VikingDB for API authentication."
+ )
+ VIKINGDB_SECRET_KEY: Optional[str] = Field(
+ default=None, description="The Secret Key provided by Volcengine VikingDB for API authentication."
+ )
+ VIKINGDB_REGION: Optional[str] = Field(
+ default="cn-shanghai",
+ description="The region of the Volcengine VikingDB service.(e.g., 'cn-shanghai', 'cn-beijing').",
+ )
+ VIKINGDB_HOST: Optional[str] = Field(
+ default="api-vikingdb.mlp.cn-shanghai.volces.com",
+ description="The host of the Volcengine VikingDB service.(e.g., 'api-vikingdb.volces.com', \
+ 'api-vikingdb.mlp.cn-shanghai.volces.com')",
+ )
+ VIKINGDB_SCHEME: Optional[str] = Field(
+ default="http",
+ description="The scheme of the Volcengine VikingDB service.(e.g., 'http', 'https').",
+ )
+ VIKINGDB_CONNECTION_TIMEOUT: Optional[int] = Field(
+ default=30, description="The connection timeout of the Volcengine VikingDB service."
+ )
+ VIKINGDB_SOCKET_TIMEOUT: Optional[int] = Field(
+ default=30, description="The socket timeout of the Volcengine VikingDB service."
+ )
diff --git a/api/configs/middleware/vdb/weaviate_config.py b/api/configs/middleware/vdb/weaviate_config.py
index 63d1022f6a451..25000e8bde290 100644
--- a/api/configs/middleware/vdb/weaviate_config.py
+++ b/api/configs/middleware/vdb/weaviate_config.py
@@ -6,25 +6,25 @@
class WeaviateConfig(BaseSettings):
"""
- Weaviate configs
+ Configuration settings for Weaviate vector database
"""
WEAVIATE_ENDPOINT: Optional[str] = Field(
- description="Weaviate endpoint URL",
+ description="URL of the Weaviate server (e.g., 'http://localhost:8080' or 'https://weaviate.example.com')",
default=None,
)
WEAVIATE_API_KEY: Optional[str] = Field(
- description="Weaviate API key",
+ description="API key for authenticating with the Weaviate server",
default=None,
)
WEAVIATE_GRPC_ENABLED: bool = Field(
- description="whether to enable gRPC for Weaviate connection",
+ description="Whether to enable gRPC for Weaviate connection (True for gRPC, False for HTTP)",
default=True,
)
WEAVIATE_BATCH_SIZE: PositiveInt = Field(
- description="Weaviate batch size",
+ description="Number of objects to be processed in a single batch operation (default is 100)",
default=100,
)
diff --git a/api/configs/packaging/__init__.py b/api/configs/packaging/__init__.py
index c7526601225d8..c832fb671e5af 100644
--- a/api/configs/packaging/__init__.py
+++ b/api/configs/packaging/__init__.py
@@ -9,7 +9,7 @@ class PackagingInfo(BaseSettings):
CURRENT_VERSION: str = Field(
description="Dify version",
- default="0.8.3",
+ default="0.9.2",
)
COMMIT_SHA: str = Field(
diff --git a/api/constants/__init__.py b/api/constants/__init__.py
index e22c3268ef428..75eaf81638cdf 100644
--- a/api/constants/__init__.py
+++ b/api/constants/__init__.py
@@ -1 +1,2 @@
HIDDEN_VALUE = "[__HIDDEN__]"
+UUID_NIL = "00000000-0000-0000-0000-000000000000"
diff --git a/api/controllers/console/__init__.py b/api/controllers/console/__init__.py
index eb7c1464d3972..c7282fcf14531 100644
--- a/api/controllers/console/__init__.py
+++ b/api/controllers/console/__init__.py
@@ -37,7 +37,16 @@
from .billing import billing
# Import datasets controllers
-from .datasets import data_source, datasets, datasets_document, datasets_segments, file, hit_testing, website
+from .datasets import (
+ data_source,
+ datasets,
+ datasets_document,
+ datasets_segments,
+ external,
+ file,
+ hit_testing,
+ website,
+)
# Import explore controllers
from .explore import (
diff --git a/api/controllers/console/app/completion.py b/api/controllers/console/app/completion.py
index 53de51c24d798..d3296d3dff44a 100644
--- a/api/controllers/console/app/completion.py
+++ b/api/controllers/console/app/completion.py
@@ -109,6 +109,7 @@ def post(self, app_model):
parser.add_argument("files", type=list, required=False, location="json")
parser.add_argument("model_config", type=dict, required=True, location="json")
parser.add_argument("conversation_id", type=uuid_value, location="json")
+ parser.add_argument("parent_message_id", type=uuid_value, required=False, location="json")
parser.add_argument("response_mode", type=str, choices=["blocking", "streaming"], location="json")
parser.add_argument("retriever_from", type=str, required=False, default="dev", location="json")
args = parser.parse_args()
diff --git a/api/controllers/console/app/conversation.py b/api/controllers/console/app/conversation.py
index df7bd352af296..c1e16b3b9b65a 100644
--- a/api/controllers/console/app/conversation.py
+++ b/api/controllers/console/app/conversation.py
@@ -188,6 +188,7 @@ def get(self, app_model):
subquery.c.from_end_user_session_id.ilike(keyword_filter),
),
)
+ .group_by(Conversation.id)
)
account = current_user
diff --git a/api/controllers/console/app/message.py b/api/controllers/console/app/message.py
index fe06201982374..2fba3e0af02be 100644
--- a/api/controllers/console/app/message.py
+++ b/api/controllers/console/app/message.py
@@ -105,8 +105,6 @@ def get(self, app_model):
if rest_count > 0:
has_more = True
- history_messages = list(reversed(history_messages))
-
return InfiniteScrollPagination(data=history_messages, limit=args["limit"], has_more=has_more)
diff --git a/api/controllers/console/app/workflow.py b/api/controllers/console/app/workflow.py
index b488deb89d90c..0a693b84e27ec 100644
--- a/api/controllers/console/app/workflow.py
+++ b/api/controllers/console/app/workflow.py
@@ -166,6 +166,8 @@ def post(self, app_model: App):
parser.add_argument("query", type=str, required=True, location="json", default="")
parser.add_argument("files", type=list, location="json")
parser.add_argument("conversation_id", type=uuid_value, location="json")
+ parser.add_argument("parent_message_id", type=uuid_value, required=False, location="json")
+
args = parser.parse_args()
try:
diff --git a/api/controllers/console/auth/login.py b/api/controllers/console/auth/login.py
index 62837af2b9b0e..18a7b2316660c 100644
--- a/api/controllers/console/auth/login.py
+++ b/api/controllers/console/auth/login.py
@@ -7,7 +7,7 @@
import services
from controllers.console import api
from controllers.console.setup import setup_required
-from libs.helper import email, get_remote_ip
+from libs.helper import email, extract_remote_ip
from libs.password import valid_password
from models.account import Account
from services.account_service import AccountService, TenantService
@@ -40,17 +40,16 @@ def post(self):
"data": "workspace not found, please contact system admin to invite you to join in a workspace",
}
- token = AccountService.login(account, ip_address=get_remote_ip(request))
+ token_pair = AccountService.login(account=account, ip_address=extract_remote_ip(request))
- return {"result": "success", "data": token}
+ return {"result": "success", "data": token_pair.model_dump()}
class LogoutApi(Resource):
@setup_required
def get(self):
account = cast(Account, flask_login.current_user)
- token = request.headers.get("Authorization", "").split(" ")[1]
- AccountService.logout(account=account, token=token)
+ AccountService.logout(account=account)
flask_login.logout_user()
return {"result": "success"}
@@ -106,5 +105,19 @@ def get(self):
return {"result": "success"}
+class RefreshTokenApi(Resource):
+ def post(self):
+ parser = reqparse.RequestParser()
+ parser.add_argument("refresh_token", type=str, required=True, location="json")
+ args = parser.parse_args()
+
+ try:
+ new_token_pair = AccountService.refresh_token(args["refresh_token"])
+ return {"result": "success", "data": new_token_pair.model_dump()}
+ except Exception as e:
+ return {"result": "fail", "data": str(e)}, 401
+
+
api.add_resource(LoginApi, "/login")
api.add_resource(LogoutApi, "/logout")
+api.add_resource(RefreshTokenApi, "/refresh-token")
diff --git a/api/controllers/console/auth/oauth.py b/api/controllers/console/auth/oauth.py
index ad0c0580aeaba..c5909b8c1092e 100644
--- a/api/controllers/console/auth/oauth.py
+++ b/api/controllers/console/auth/oauth.py
@@ -9,7 +9,7 @@
from configs import dify_config
from constants.languages import languages
from extensions.ext_database import db
-from libs.helper import get_remote_ip
+from libs.helper import extract_remote_ip
from libs.oauth import GitHubOAuth, GoogleOAuth, OAuthUserInfo
from models.account import Account, AccountStatus
from services.account_service import AccountService, RegisterService, TenantService
@@ -81,9 +81,14 @@ def get(self, provider: str):
TenantService.create_owner_tenant_if_not_exist(account)
- token = AccountService.login(account, ip_address=get_remote_ip(request))
+ token_pair = AccountService.login(
+ account=account,
+ ip_address=extract_remote_ip(request),
+ )
- return redirect(f"{dify_config.CONSOLE_WEB_URL}?console_token={token}")
+ return redirect(
+ f"{dify_config.CONSOLE_WEB_URL}?access_token={token_pair.access_token}&refresh_token={token_pair.refresh_token}"
+ )
def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) -> Optional[Account]:
diff --git a/api/controllers/console/datasets/datasets.py b/api/controllers/console/datasets/datasets.py
index 2c4e5ac60789e..6583356d23c9e 100644
--- a/api/controllers/console/datasets/datasets.py
+++ b/api/controllers/console/datasets/datasets.py
@@ -49,7 +49,7 @@ def get(self):
page = request.args.get("page", default=1, type=int)
limit = request.args.get("limit", default=20, type=int)
ids = request.args.getlist("ids")
- provider = request.args.get("provider", default="vendor")
+ # provider = request.args.get("provider", default="vendor")
search = request.args.get("keyword", default=None, type=str)
tag_ids = request.args.getlist("tag_ids")
@@ -57,7 +57,7 @@ def get(self):
datasets, total = DatasetService.get_datasets_by_ids(ids, current_user.current_tenant_id)
else:
datasets, total = DatasetService.get_datasets(
- page, limit, provider, current_user.current_tenant_id, current_user, search, tag_ids
+ page, limit, current_user.current_tenant_id, current_user, search, tag_ids
)
# check embedding setting
@@ -110,6 +110,26 @@ def post(self):
nullable=True,
help="Invalid indexing technique.",
)
+ parser.add_argument(
+ "external_knowledge_api_id",
+ type=str,
+ nullable=True,
+ required=False,
+ )
+ parser.add_argument(
+ "provider",
+ type=str,
+ nullable=True,
+ choices=Dataset.PROVIDER_LIST,
+ required=False,
+ default="vendor",
+ )
+ parser.add_argument(
+ "external_knowledge_id",
+ type=str,
+ nullable=True,
+ required=False,
+ )
args = parser.parse_args()
# The role of the current user in the ta table must be admin, owner, or editor, or dataset_operator
@@ -123,6 +143,9 @@ def post(self):
indexing_technique=args["indexing_technique"],
account=current_user,
permission=DatasetPermissionEnum.ONLY_ME,
+ provider=args["provider"],
+ external_knowledge_api_id=args["external_knowledge_api_id"],
+ external_knowledge_id=args["external_knowledge_id"],
)
except services.errors.dataset.DatasetNameDuplicateError:
raise DatasetNameDuplicateError()
@@ -211,6 +234,33 @@ def patch(self, dataset_id):
)
parser.add_argument("retrieval_model", type=dict, location="json", help="Invalid retrieval model.")
parser.add_argument("partial_member_list", type=list, location="json", help="Invalid parent user list.")
+
+ parser.add_argument(
+ "external_retrieval_model",
+ type=dict,
+ required=False,
+ nullable=True,
+ location="json",
+ help="Invalid external retrieval model.",
+ )
+
+ parser.add_argument(
+ "external_knowledge_id",
+ type=str,
+ required=False,
+ nullable=True,
+ location="json",
+ help="Invalid external knowledge id.",
+ )
+
+ parser.add_argument(
+ "external_knowledge_api_id",
+ type=str,
+ required=False,
+ nullable=True,
+ location="json",
+ help="Invalid external knowledge api id.",
+ )
args = parser.parse_args()
data = request.get_json()
@@ -563,10 +613,12 @@ def get(self):
case (
VectorType.MILVUS
| VectorType.RELYT
- | VectorType.PGVECTOR
| VectorType.TIDB_VECTOR
| VectorType.CHROMA
| VectorType.TENCENT
+ | VectorType.PGVECTO_RS
+ | VectorType.BAIDU
+ | VectorType.VIKINGDB
):
return {"retrieval_method": [RetrievalMethod.SEMANTIC_SEARCH.value]}
case (
@@ -577,6 +629,7 @@ def get(self):
| VectorType.MYSCALE
| VectorType.ORACLE
| VectorType.ELASTICSEARCH
+ | VectorType.PGVECTOR
):
return {
"retrieval_method": [
@@ -602,6 +655,8 @@ def get(self, vector_type):
| VectorType.CHROMA
| VectorType.TENCENT
| VectorType.PGVECTO_RS
+ | VectorType.BAIDU
+ | VectorType.VIKINGDB
):
return {"retrieval_method": [RetrievalMethod.SEMANTIC_SEARCH.value]}
case (
diff --git a/api/controllers/console/datasets/external.py b/api/controllers/console/datasets/external.py
new file mode 100644
index 0000000000000..2dc054cfbdf4a
--- /dev/null
+++ b/api/controllers/console/datasets/external.py
@@ -0,0 +1,263 @@
+from flask import request
+from flask_login import current_user
+from flask_restful import Resource, marshal, reqparse
+from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
+
+import services
+from controllers.console import api
+from controllers.console.datasets.error import DatasetNameDuplicateError
+from controllers.console.setup import setup_required
+from controllers.console.wraps import account_initialization_required
+from fields.dataset_fields import dataset_detail_fields
+from libs.login import login_required
+from services.dataset_service import DatasetService
+from services.external_knowledge_service import ExternalDatasetService
+from services.hit_testing_service import HitTestingService
+from services.knowledge_service import ExternalDatasetTestService
+
+
+def _validate_name(name):
+ if not name or len(name) < 1 or len(name) > 100:
+ raise ValueError("Name must be between 1 to 100 characters.")
+ return name
+
+
+def _validate_description_length(description):
+ if description and len(description) > 400:
+ raise ValueError("Description cannot exceed 400 characters.")
+ return description
+
+
+class ExternalApiTemplateListApi(Resource):
+ @setup_required
+ @login_required
+ @account_initialization_required
+ def get(self):
+ page = request.args.get("page", default=1, type=int)
+ limit = request.args.get("limit", default=20, type=int)
+ search = request.args.get("keyword", default=None, type=str)
+
+ external_knowledge_apis, total = ExternalDatasetService.get_external_knowledge_apis(
+ page, limit, current_user.current_tenant_id, search
+ )
+ response = {
+ "data": [item.to_dict() for item in external_knowledge_apis],
+ "has_more": len(external_knowledge_apis) == limit,
+ "limit": limit,
+ "total": total,
+ "page": page,
+ }
+ return response, 200
+
+ @setup_required
+ @login_required
+ @account_initialization_required
+ def post(self):
+ parser = reqparse.RequestParser()
+ parser.add_argument(
+ "name",
+ nullable=False,
+ required=True,
+ help="Name is required. Name must be between 1 to 100 characters.",
+ type=_validate_name,
+ )
+ parser.add_argument(
+ "settings",
+ type=dict,
+ location="json",
+ nullable=False,
+ required=True,
+ )
+ args = parser.parse_args()
+
+ ExternalDatasetService.validate_api_list(args["settings"])
+
+ # The role of the current user in the ta table must be admin, owner, or editor, or dataset_operator
+ if not current_user.is_dataset_editor:
+ raise Forbidden()
+
+ try:
+ external_knowledge_api = ExternalDatasetService.create_external_knowledge_api(
+ tenant_id=current_user.current_tenant_id, user_id=current_user.id, args=args
+ )
+ except services.errors.dataset.DatasetNameDuplicateError:
+ raise DatasetNameDuplicateError()
+
+ return external_knowledge_api.to_dict(), 201
+
+
+class ExternalApiTemplateApi(Resource):
+ @setup_required
+ @login_required
+ @account_initialization_required
+ def get(self, external_knowledge_api_id):
+ external_knowledge_api_id = str(external_knowledge_api_id)
+ external_knowledge_api = ExternalDatasetService.get_external_knowledge_api(external_knowledge_api_id)
+ if external_knowledge_api is None:
+ raise NotFound("API template not found.")
+
+ return external_knowledge_api.to_dict(), 200
+
+ @setup_required
+ @login_required
+ @account_initialization_required
+ def patch(self, external_knowledge_api_id):
+ external_knowledge_api_id = str(external_knowledge_api_id)
+
+ parser = reqparse.RequestParser()
+ parser.add_argument(
+ "name",
+ nullable=False,
+ required=True,
+ help="type is required. Name must be between 1 to 100 characters.",
+ type=_validate_name,
+ )
+ parser.add_argument(
+ "settings",
+ type=dict,
+ location="json",
+ nullable=False,
+ required=True,
+ )
+ args = parser.parse_args()
+ ExternalDatasetService.validate_api_list(args["settings"])
+
+ external_knowledge_api = ExternalDatasetService.update_external_knowledge_api(
+ tenant_id=current_user.current_tenant_id,
+ user_id=current_user.id,
+ external_knowledge_api_id=external_knowledge_api_id,
+ args=args,
+ )
+
+ return external_knowledge_api.to_dict(), 200
+
+ @setup_required
+ @login_required
+ @account_initialization_required
+ def delete(self, external_knowledge_api_id):
+ external_knowledge_api_id = str(external_knowledge_api_id)
+
+ # The role of the current user in the ta table must be admin, owner, or editor
+ if not current_user.is_editor or current_user.is_dataset_operator:
+ raise Forbidden()
+
+ ExternalDatasetService.delete_external_knowledge_api(current_user.current_tenant_id, external_knowledge_api_id)
+ return {"result": "success"}, 200
+
+
+class ExternalApiUseCheckApi(Resource):
+ @setup_required
+ @login_required
+ @account_initialization_required
+ def get(self, external_knowledge_api_id):
+ external_knowledge_api_id = str(external_knowledge_api_id)
+
+ external_knowledge_api_is_using, count = ExternalDatasetService.external_knowledge_api_use_check(
+ external_knowledge_api_id
+ )
+ return {"is_using": external_knowledge_api_is_using, "count": count}, 200
+
+
+class ExternalDatasetCreateApi(Resource):
+ @setup_required
+ @login_required
+ @account_initialization_required
+ def post(self):
+ # The role of the current user in the ta table must be admin, owner, or editor
+ if not current_user.is_editor:
+ raise Forbidden()
+
+ parser = reqparse.RequestParser()
+ parser.add_argument("external_knowledge_api_id", type=str, required=True, nullable=False, location="json")
+ parser.add_argument("external_knowledge_id", type=str, required=True, nullable=False, location="json")
+ parser.add_argument(
+ "name",
+ nullable=False,
+ required=True,
+ help="name is required. Name must be between 1 to 100 characters.",
+ type=_validate_name,
+ )
+ parser.add_argument("description", type=str, required=False, nullable=True, location="json")
+ parser.add_argument("external_retrieval_model", type=dict, required=False, location="json")
+
+ args = parser.parse_args()
+
+ # The role of the current user in the ta table must be admin, owner, or editor, or dataset_operator
+ if not current_user.is_dataset_editor:
+ raise Forbidden()
+
+ try:
+ dataset = ExternalDatasetService.create_external_dataset(
+ tenant_id=current_user.current_tenant_id,
+ user_id=current_user.id,
+ args=args,
+ )
+ except services.errors.dataset.DatasetNameDuplicateError:
+ raise DatasetNameDuplicateError()
+
+ return marshal(dataset, dataset_detail_fields), 201
+
+
+class ExternalKnowledgeHitTestingApi(Resource):
+ @setup_required
+ @login_required
+ @account_initialization_required
+ def post(self, dataset_id):
+ dataset_id_str = str(dataset_id)
+ dataset = DatasetService.get_dataset(dataset_id_str)
+ if dataset is None:
+ raise NotFound("Dataset not found.")
+
+ try:
+ DatasetService.check_dataset_permission(dataset, current_user)
+ except services.errors.account.NoPermissionError as e:
+ raise Forbidden(str(e))
+
+ parser = reqparse.RequestParser()
+ parser.add_argument("query", type=str, location="json")
+ parser.add_argument("external_retrieval_model", type=dict, required=False, location="json")
+ args = parser.parse_args()
+
+ HitTestingService.hit_testing_args_check(args)
+
+ try:
+ response = HitTestingService.external_retrieve(
+ dataset=dataset,
+ query=args["query"],
+ account=current_user,
+ external_retrieval_model=args["external_retrieval_model"],
+ )
+
+ return response
+ except Exception as e:
+ raise InternalServerError(str(e))
+
+
+class BedrockRetrievalApi(Resource):
+ # this api is only for internal testing
+ def post(self):
+ parser = reqparse.RequestParser()
+ parser.add_argument("retrieval_setting", nullable=False, required=True, type=dict, location="json")
+ parser.add_argument(
+ "query",
+ nullable=False,
+ required=True,
+ type=str,
+ )
+ parser.add_argument("knowledge_id", nullable=False, required=True, type=str)
+ args = parser.parse_args()
+
+ # Call the knowledge retrieval service
+ result = ExternalDatasetTestService.knowledge_retrieval(
+ args["retrieval_setting"], args["query"], args["knowledge_id"]
+ )
+ return result, 200
+
+
+api.add_resource(ExternalKnowledgeHitTestingApi, "/datasets/{userProfile.name}
+{userProfile.email}
+{t('common.account.account')}
+