-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #1 from om-ai-lab/add_doc/v0.2.1
Add doc/v0.2.1
- Loading branch information
Showing
92 changed files
with
1,617 additions
and
79 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
# Outfit Recommendation with Loop Example | ||
|
||
This example demonstrates how to use the framework for outfit recommendation tasks with loop functionality. The example code can be found in the `examples/step3_outfit_with_loop` directory. | ||
```bash | ||
cd examples/step3_outfit_with_loop | ||
``` | ||
|
||
## Overview | ||
|
||
This example implements an interactive outfit recommendation workflow that uses a loop-based approach to refine recommendations based on user feedback. The workflow consists of the following key components: | ||
|
||
1. **Initial Image Input** | ||
- OutfitImageInput: Handles the upload and processing of the initial clothing item image | ||
- Serves as the starting point for the recommendation process | ||
|
||
2. **Interactive QA Loop with Weather Integration** | ||
- OutfitQA: Conducts an interactive Q&A session to gather context and preferences | ||
- Uses web search tool to fetch real-time weather data for the specified location | ||
- OutfitDecider: Evaluates if sufficient information has been collected based on: | ||
- User preferences | ||
- Current weather conditions | ||
- Uses DoWhileTask to continue the loop until adequate information is gathered | ||
- Loop terminates when OutfitDecider returns decision=true | ||
|
||
3. **Final Recommendation** | ||
- OutfitRecommendation: Generates the final outfit suggestions based on: | ||
- The initial uploaded image | ||
- Information collected during the Q&A loop | ||
- Current weather conditions from web search | ||
- Other context (occasion, preferences, etc.) | ||
|
||
4. **Workflow Flow** | ||
``` | ||
Start -> Image Input -> OutfitQA Loop (QA + Weather Search + Decision) -> Final Recommendation -> End | ||
``` | ||
|
||
The workflow leverages Redis for state management and the Conductor server for workflow orchestration. This architecture enables: | ||
- Image-based outfit recommendations | ||
- Weather-aware outfit suggestions using real-time data | ||
- Interactive refinement through structured Q&A | ||
- Context-aware suggestions incorporating multiple factors | ||
- Persistent state management across the workflow | ||
|
||
|
||
## Prerequisites | ||
|
||
- Python 3.10+ | ||
- Required packages installed (see requirements.txt) | ||
- Access to OpenAI API or compatible endpoint | ||
- Access to Bing API key for web search functionality to search real-time weather information for outfit recommendations (see configs/tools/websearch.yml) | ||
- Redis server running locally or remotely | ||
- Conductor server running locally or remotely | ||
|
||
## Configuration | ||
|
||
The container.yaml file is a configuration file that manages dependencies and settings for different components of the system, including Conductor connections, Redis connections, and other service configurations. To set up your configuration: | ||
|
||
1. Generate the container.yaml file: | ||
```bash | ||
python compile_container.py | ||
``` | ||
This will create a container.yaml file with default settings under `examples/step3_outfit_with_loop`. | ||
|
||
2. Configure your LLM settings in `configs/llms/gpt.yml` and `configs/llms/text_res.yml`: | ||
- Set your OpenAI API key or compatible endpoint through environment variable or by directly modifying the yml file | ||
```bash | ||
export custom_openai_key="your_openai_api_key" | ||
export custom_openai_endpoint="your_openai_endpoint" | ||
``` | ||
- Configure other model settings like temperature as needed through environment variable or by directly modifying the yml file | ||
|
||
3. Configure your Bing Search API key in `configs/tools/websearch.yml`: | ||
- Set your Bing API key through environment variable or by directly modifying the yml file | ||
```bash | ||
export bing_api_key="your_bing_api_key" | ||
``` | ||
|
||
4. Update settings in the generated `container.yaml`: | ||
- Modify Redis connection settings: | ||
- Set the host, port and credentials for your Redis instance | ||
- Configure both `redis_stream_client` and `redis_stm_client` sections | ||
- Update the Conductor server URL under conductor_config section | ||
- Adjust any other component settings as needed | ||
|
||
## Running the Example | ||
|
||
1. Run the outfit recommendation workflow: | ||
|
||
For terminal/CLI usage: | ||
```bash | ||
python run_cli.py | ||
``` | ||
|
||
For app/GUI usage: | ||
```bash | ||
python run_app.py | ||
``` | ||
|
||
|
||
## Troubleshooting | ||
|
||
If you encounter issues: | ||
- Verify Redis is running and accessible | ||
- Check your OpenAI API key and Bing API key are valid | ||
- Ensure all dependencies are installed correctly | ||
- Review logs for any error messages | ||
- Confirm Conductor server is running and accessible | ||
- Check Redis Stream client and Redis STM client configuration | ||
|
||
## Building the Example | ||
|
||
Coming soon! This section will provide detailed instructions for building the step3_outfit_with_loop example step by step. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,149 @@ | ||
# Outfit Recommendation with Long-Term Memory Example | ||
|
||
This example demonstrates how to use the framework for outfit recommendation tasks with long-term memory functionality. The example code can be found in the `examples/step4_outfit_with_ltm` directory. | ||
```bash | ||
cd examples/step4_outfit_with_ltm | ||
``` | ||
|
||
|
||
## Overview | ||
|
||
This example implements an outfit recommendation system with long-term memory capabilities through two main workflows: | ||
|
||
1. **Image Storage Workflow** | ||
- ImageIndexListener: Monitors and captures new clothing images | ||
- OutfitImagePreprocessor: Processes and prepares images for storage | ||
- Stores processed images in Milvus long-term memory (LTM) for future retrieval | ||
- Workflow sequence: Image Listening -> Preprocessing -> LTM Storage | ||
|
||
2. **Outfit Recommendation Workflow** | ||
- OutfitQA: Conducts interactive Q&A to understand user preferences | ||
- OutfitDecider: Determines if sufficient information is collected | ||
- Uses DoWhileTask for iterative refinement until decision is positive | ||
- OutfitGeneration: Generates outfit recommendations using stored image data | ||
- OutfitConclusion: Presents final recommendations with explanations | ||
|
||
The system leverages both short-term memory (Redis STM) and long-term memory (Milvus LTM) for: | ||
- Efficient image storage and retrieval | ||
- Persistent clothing item database | ||
- Context-aware outfit recommendations | ||
- Interactive preference refinement | ||
- Stateful conversation management | ||
|
||
3. **Workflow Architecture** | ||
``` | ||
Image Storage: Listen -> Preprocess -> Store in LTM | ||
Recommendation: QA Loop (QA + Decision) -> Generation -> Conclusion | ||
``` | ||
|
||
The system uses Redis for state management, Milvus for long-term image storage, and Conductor for workflow orchestration. This architecture enables: | ||
- Scalable image database management | ||
- Intelligent outfit recommendations based on stored items | ||
- Interactive preference gathering | ||
- Persistent clothing knowledge base | ||
- Efficient retrieval of relevant items | ||
|
||
## Prerequisites | ||
|
||
- Python 3.10+ | ||
- Required packages installed (see requirements.txt) | ||
- Access to OpenAI API or compatible endpoint (see configs/llms/gpt.yml) | ||
- Access to Bing API key for web search functionality to search real-time weather information for outfit recommendations (see configs/tools/websearch.yml) | ||
- Redis server running locally or remotely | ||
- Conductor server running locally or remotely | ||
- Milvus vector database (will be started automatically when workflow runs) | ||
- Sufficient storage space for image database | ||
- Install Git LFS by `git lfs intall`, then pull sample images by `git lfs pull` | ||
|
||
## Configuration | ||
|
||
The container.yaml file is a configuration file that manages dependencies and settings for different components of the system, including Conductor connections, Redis connections, Milvus connections and other service configurations. To set up your configuration: | ||
|
||
1. Generate the container.yaml files: | ||
```bash | ||
# For image storage workflow | ||
python image_storage/compile_container.py | ||
|
||
# For outfit recommendation workflow | ||
python outfit_from_storage/compile_container.py | ||
``` | ||
This will create two container.yaml files with default settings under `image_storage` and `outfit_from_storage` directories: | ||
- `image_storage/container.yaml`: Configuration for the image storage workflow | ||
- `outfit_from_storage/container.yaml`: Configuration for the outfit recommendation workflow | ||
|
||
2. Configure your LLM settings in `configs/llms/gpt.yml` and `configs/llms/text_res.yml` in the two workflow directories: | ||
- Set your OpenAI API key or compatible endpoint through environment variable or by directly modifying the yml file | ||
```bash | ||
export custom_openai_key="your_openai_api_key" | ||
export custom_openai_endpoint="your_openai_endpoint" | ||
``` | ||
- Configure other model settings like temperature as needed through environment variable or by directly modifying the yml file | ||
|
||
3. Configure your Bing Search API key in `configs/tools/websearch.yml` in the two workflow directories: | ||
- Set your Bing API key through environment variable or by directly modifying the yml file | ||
```bash | ||
export bing_api_key="your_bing_api_key" | ||
``` | ||
4. Configure your text encoder settings in `configs/llms/text_encoder.yml` in the two workflow directories: | ||
- Set your OpenAI text encoder endpoint and API key through environment variable or by directly modifying the yml file | ||
```bash | ||
export custom_openai_text_encoder_key="openai_text_encoder_key" | ||
export custom_openai_text_encoder_endpoint="your_openai_endpoint" | ||
``` | ||
- The default text encoder configuration uses OpenAI text embedding v3 with 3072 dimensions, make sure you change the dim value of `MilvusLTM` in `container.yaml` | ||
- Adjust the embedding dimension and other settings as needed through environment variable or by directly modifying the yml file | ||
|
||
4. Update settings in the generated `container.yaml`: | ||
- Modify Redis connection settings: | ||
- Set the host, port and credentials for your Redis instance | ||
- Configure both `redis_stream_client` and `redis_stm_client` sections | ||
- Update the Conductor server URL under conductor_config section | ||
- Configure MilvusLTM in `components` section: | ||
- Set the `storage_name` and `dim` for MilvusLTM | ||
- Adjust other settings as needed | ||
- Adjust any other component settings as needed | ||
|
||
## Running the Example | ||
|
||
1. Run the image storage workflow first: | ||
|
||
For terminal/CLI usage: | ||
```bash | ||
python image_storage/run_image_storage_cli.py | ||
``` | ||
For app usage: | ||
```bash | ||
python image_storage/run_image_storage_app.py | ||
``` | ||
|
||
This workflow will store outfit images in the Milvus database. | ||
|
||
2. Run the outfit recommendation workflow in a separate terminal: | ||
|
||
For terminal/CLI usage: | ||
```bash | ||
python outfit_from_storage/run_outfit_recommendation_cli.py | ||
``` | ||
|
||
For app/GUI usage: | ||
```bash | ||
python outfit_from_storage/run_outfit_recommendation_app.py | ||
``` | ||
|
||
This workflow will retrieve outfit recommendations from the stored images. | ||
|
||
|
||
## Troubleshooting | ||
|
||
If you encounter issues: | ||
- Verify Redis is running and accessible | ||
- Check your OpenAI API key and Bing API key are valid | ||
- Ensure all dependencies are installed correctly | ||
- Review logs for any error messages | ||
- Confirm Conductor server is running and accessible | ||
- Check Redis Stream client and Redis STM client configuration | ||
|
||
## Building the Example | ||
|
||
Coming soon! This section will provide detailed instructions for building the step4_outfit_with_ltm example step by step. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
# Outfit Recommendation with Switch Example | ||
|
||
This example demonstrates how to use the framework for outfit recommendation tasks with switch_case functionality. The example code can be found in the `examples/step2_outfit_with_switch` directory. | ||
|
||
```bash | ||
cd examples/step2_outfit_with_switch | ||
``` | ||
|
||
## Overview | ||
|
||
This example implements an outfit recommendation workflow that uses switch-case functionality to conditionally include weather information in the recommendation process. The workflow consists of the following key components: | ||
|
||
1. **Input Interface** | ||
- Handles user input containing clothing requests and image data | ||
- Processes and caches any uploaded images | ||
- Extracts the user's outfit request instructions | ||
|
||
2. **Weather Decision Logic** | ||
- WeatherDecider: Analyzes the user's request to determine if weather information is needed | ||
- Makes a binary decision (0 or 1) based on context in the user's request | ||
- Controls whether weather data should be fetched | ||
|
||
3. **Conditional Weather Search** | ||
- WeatherSearcher: Only executes if WeatherDecider returns 0 (weather info needed) | ||
- Uses web search functionality to fetch current weather conditions | ||
- Integrates weather data into the recommendation context | ||
|
||
4. **Outfit Recommendation** | ||
- Generates final clothing suggestions based on: | ||
- User's original request | ||
- Weather information (if available) | ||
- Any provided image context | ||
- Provides complete outfit recommendations | ||
|
||
The workflow follows this sequence: | ||
|
||
|
||
## Prerequisites | ||
|
||
- Python 3.10+ | ||
- Required packages installed (see requirements.txt) | ||
- Access to OpenAI API or compatible endpoint (see configs/llms/gpt.yml) | ||
- Access to Bing API key for web search functionality to search real-time weather information for outfit recommendations (see configs/tools/websearch.yml) | ||
- Redis server running locally or remotely | ||
- Conductor server running locally or remotely | ||
|
||
## Configuration | ||
|
||
The container.yaml file is a configuration file that manages dependencies and settings for different components of the system, including Conductor connections, Redis connections, and other service configurations. To set up your configuration: | ||
|
||
1. Generate the container.yaml file: | ||
```bash | ||
python compile_container.py | ||
``` | ||
This will create a container.yaml file with default settings under `examples/step2_outfit_with_switch`. | ||
|
||
|
||
|
||
2. Configure your LLM settings in `configs/llms/gpt.yml` and `configs/llms/text_res.yml`: | ||
|
||
- Set your OpenAI API key or compatible endpoint through environment variable or by directly modifying the yml file | ||
```bash | ||
export custom_openai_key="your_openai_api_key" | ||
export custom_openai_endpoint="your_openai_endpoint" | ||
``` | ||
- Configure other model settings like temperature as needed through environment variable or by directly modifying the yml file | ||
|
||
3. Configure your Bing Search API key in `configs/tools/websearch.yml`: | ||
- Set your Bing API key through environment variable or by directly modifying the yml file | ||
```bash | ||
export bing_api_key="your_bing_api_key" | ||
``` | ||
|
||
4. Update settings in the generated `container.yaml`: | ||
- Modify Redis connection settings: | ||
- Set the host, port and credentials for your Redis instance | ||
- Configure both `redis_stream_client` and `redis_stm_client` sections | ||
- Update the Conductor server URL under conductor_config section | ||
- Adjust any other component settings as needed | ||
|
||
## Running the Example | ||
|
||
3. Run the outfit recommendation with switch example: | ||
|
||
For terminal/CLI usage: | ||
```bash | ||
python run_cli.py | ||
``` | ||
|
||
For app/GUI usage: | ||
```bash | ||
python run_app.py | ||
``` | ||
|
||
|
||
|
||
## Troubleshooting | ||
|
||
If you encounter issues: | ||
|
||
- Verify Conductor and Redis are running and accessible | ||
- Check your OpenAI API key and Bing API key are valid | ||
- Check Redis Stream client and Redis STM client configuration | ||
|
||
- Ensure all dependencies are installed correctly | ||
- Review logs for any error messages | ||
|
||
|
||
## Building the Example | ||
|
||
Coming soon! This section will provide detailed instructions for building the step2_outfit_with_switch example step by step. | ||
|
Oops, something went wrong.