DATAGO-83398: Use PYPI token for release action #31
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add ability to stream partial LLM responses to a named flow * Add a configurable message that is sent to the slack channel that sent a message to slack_input * Add the ability to clear the ack message and stream partial results to that ack message
…hHistory - also started on converting absolute paths for imports to relative ones
The import statement in `main.py` was updated to remove the relative path and use an absolute import for `SolaceAiConnector` from `solace_ai_connector.solace_ai_connector`. This change improves the code organization and ensures proper module resolution.
…ensible. This is part of a cleanup before releasing to Pypi
…do a directory search looking for modules. We are more explicit now Also promote the gen_docs tool to an installed script so that it can be used for plugins
Adjusted the importing of modules from config files to be more friendly for plugins and added a 'component_package' properly that will be auto-installed if it is specified and the package is not present.
…_kv_store This commit updates the component_base.py file to include the flow_lock_manager and flow_kv_store attributes. These attributes are necessary for components to access the flow-level lock manager and key-value store. By adding these attributes, components can now utilize the lock manager to synchronize access to shared resources and the key-value store to store and retrieve data across multiple components within the flow. This change improves the flexibility and functionality of the component_base.py file. Changed the default location of the trust_store for the Solace API to be provided by the certifi module. Added a configuration item for the ChatModel with History component to be able to limit the size of entries being added to the chat history
This commit updates the trust_store_path for the Solace API in the solace_messaging.py file. The trust_store_path is now provided by the certifi module, which ensures that the default location of the trust store is used. This change improves the security and reliability of the Solace API integration. ```
DATAGO-78654: Add ci
…calar values after processing (#5)
* Cleanup of some lingering prints and add a couple more tests * Print cleanup * Exclude coverage.py result from sonarqube --------- Co-authored-by: Edward Funnekotter <[email protected]> Co-authored-by: Edward Funnekotter <[email protected]>
* Add reusable workflows * Run docker tests in parallel
Better_streaming
Set the log level to the min between STDOUT and file
Added request_response component and added config to select temporary queues for broker_input
* First commit of big refactor * refactor: improve event handling in TestOutputComponent * refactor: handle both Message and Event objects in TestInputComponent * fix: handle both Event and Message objects in TestOutputComponent * feat: add Message import to utils_for_test_files * feat: add send_message method to ComponentBase * fix: wrap message in Event before enqueueing to next component * refactor: move timer_manager creation to SolaceAiConnector and improve error handling * feat: add logging for timer manager stop process * perf: optimize TimerManager to reduce CPU usage * fix: improve shutdown process for timer thread * perf: Remove unnecessary sleep in TimerManager run loop * refactor: remove unused send_message method from ComponentBase * refactor: improve clean exit process and resource cleanup * fix: import log module to resolve undefined name errors * refactor: update aggregate component to handle timeout consistently * refactor: update aggregate component output to return list * refactor: simplify aggregation logic and improve method naming * fix: reset current_aggregation after returning aggregation * All tests now pass * Warning cleanup * chore: one additional small fix to account for input components that might return None * refactor: rename Event class 'payload' to 'data' * refactor: Update get_message_from_flow to use event.data instead of event.payload * Switched the contained data in the Event class to be called data rather than payload to avoid confusion with message payload * Remove commented code * feat: add OpenAI chat model components with and without history * refactor: replace LangChain with native OpenAI library for chat models * feat: add history max time and clear history options to OpenAI chat model * feat: implement clear_history_but_keep_depth functionality * more changes * feat: add method to clear chat history while retaining specified depth * fix: correct data member references in clear_history_but_keep_depth method * refactor: implement thread-safe history storage using KV store * feat: implement history_max_time parameter for message pruning * feat: add time-based history pruning to OpenAIChatModelWithHistory * refactor: optimize history management in OpenAIChatModelWithHistory * feat: implement time-based pruning in conversation history * refactor: optimize chat history management * feat: add history age-out functionality with hourly cleanup timer * clean up some spacing * feat: add base_url configuration for OpenAIChatModelBase * feat: add streaming functionality to OpenAIChatModelBase * feat: update chat history handling for system messages * refactor: ensure first history message is user message in clear_history_but_keep_depth * refactor: update clear_history_but_keep_depth to ensure depth includes last user message * Make sure that the first message in the history has a user role when trimming the history * Merge branch 'main' into open_ai_components * chore: Some small formatting changes * chore: Add some protection around clearing the history to make sure the first message is never a non-user message.
- Added clarifications - Added instruction for creating a local solace broker - Update Makefile and Dockerfile with minor details - Fixed bug with writing stream file
Updated documentations + File log bug fix - Added clarifications - Added instruction for creating a local solace broker - Update Makefile and Dockerfile with minor details - Fixed bug with writing stream file
efunneko
added a commit
that referenced
this pull request
Sep 5, 2024
…UUID Invoke Function, Added assembly component (#36) * Examples Update + Code Refactor (#25) * Removed StorageManager * Added examples for OpenAI, Bedrock, Anthropic, and VertexAI * Updating old examples (1/2) * Updating old examples (2/2) * Added support for temporary queue + UUID queue name (#26) * Add assembly component and auto-generated documents (#27) * Added the assembly component * Auto-generated documents * Added type check * Update the cache service expiry logic + Update the assembly component to use cache expiry for timeout * Moved assembly to the correct place * Added MoA Example + UUID Invoke Function (#28) * MoA example: Broadcast to multiple agents * Added MoA event manager, added uuid invoke_function + test, updated auto-generated docs * Added assembly layer to MoA example * Update documentation for new users + Refactored component_input & source_expression (#29) * Refactored component_input to input_selection * Updated, added, and enhanced the documentation with new users in mind * Refactored source_expression function to evaluate_expression (backward compatible) * Added tips and tricks section + info and examples on custom modules * tiny format update * tiny update * Fixed solace disconnection issues on shutting down (#30) * Add RAG example for AI connector + delete action for vector index (#31) * Added a RAG example for AI connector * Added delete option to vectordb * Changed id to ids * chore: Refactor make_history_start_with_user_message method (#32) Fix the method to not trim the first entry if it is a "system" role --------- Co-authored-by: Edward Funnekotter <[email protected]>
efunneko
added a commit
that referenced
this pull request
Sep 26, 2024
…ved docs and examples and better testing (#39) * Examples Update + Code Refactor (#25) * Removed StorageManager * Added examples for OpenAI, Bedrock, Anthropic, and VertexAI * Updating old examples (1/2) * Updating old examples (2/2) * Added support for temporary queue + UUID queue name (#26) * Add assembly component and auto-generated documents (#27) * Added the assembly component * Auto-generated documents * Added type check * Update the cache service expiry logic + Update the assembly component to use cache expiry for timeout * Moved assembly to the correct place * Added MoA Example + UUID Invoke Function (#28) * MoA example: Broadcast to multiple agents * Added MoA event manager, added uuid invoke_function + test, updated auto-generated docs * Added assembly layer to MoA example * Update documentation for new users + Refactored component_input & source_expression (#29) * Refactored component_input to input_selection * Updated, added, and enhanced the documentation with new users in mind * Refactored source_expression function to evaluate_expression (backward compatible) * Added tips and tricks section + info and examples on custom modules * tiny format update * tiny update * Fixed solace disconnection issues on shutting down (#30) * Add RAG example for AI connector + delete action for vector index (#31) * Added a RAG example for AI connector * Added delete option to vectordb * Changed id to ids * chore: Refactor make_history_start_with_user_message method (#32) Fix the method to not trim the first entry if it is a "system" role * Keep history depth needs to be a positive integer and test refactor (#33) * chore: Refactor clear_history_but_keep_depth method to handle negative depth values * chore: small change to how this is solved * chore: one more try * refactor: move utils_for_test_files.py to solace_ai_connector module * refactor: removed the orginal utils_for_test_files.py * refactor: update import statements in test files * refactor: add sys.path.append("src") to test files * refactor: standardize import order and sys.path.append in test files * refactor: a bit more test infrastructure changes * feat: allow component_module to accept module objects directly * feat: add types module import to utils.py * test: add static import and object config test * refactor: update test_static_import_and_object_config to use create_test_flows * refactor: Improve test structure and remove duplicate test case * fix: remove duplicate import of yaml module * refactor: Modify test config to use dict instead of YAML string * refactor: convert config_yaml from string to dictionary * refactor: update static import test to use pass_through component * test: Add delay component message passing test * feat: add test for delay component message processing * feat: Added a new test function (test_one_component) to make it very easy to just run some quick tests on a single input -> expected output tests on a single component * feat: added input_transforms to the test_one_component so that input transforms can be tested with it * chore: a bit of cleanup and new tests for test_one_component * chore: rename test_one_component because it was being picked up as a test by the pytest scanner * fix: fixed a typo * Fix for anthropic example (#35) * Updating version dependency (#37) * Fixed url and file name in getting started (#38) * Add guide for RAG (#39) * Added guide for RAG * update wording * Added link to other docs from RAG guide (#40) * chore: added a timeout setting for running component tests so that you can test situations where you don't expect any output (#34) * AI-124: Add a feature to provide simple blocking broker request/response ability for components (#42) * feat: add request_response_controller.py * feat: implement RequestResponseFlowManager and RequestResponseController classes * style: format code with black and improve readability * feat: implement RequestResponseController for flow-based request-response handling * feat: implement RequestResponseController for handling request-response patterns * fix: import SolaceAiConnector for type checking * refactor: restructure Flow class and improve code organization * feat: implement multiple named RequestResponseControllers per component * refactor: initialize request-response controllers in ComponentBase * test: add request_response_controller functionality tests * feat: finished implementation and added some tests * refactor: rename RequestResponseController to RequestResponseFlowController * refactor: rename RequestResponseController to RequestResponseFlowController * refactor: some name changes * fix: update test function names for RequestResponseFlowController * refactor: more name changes * Ed/req_resp_examples_and_fixes (#41) * feat: Added a request_response_flow example and fixed a few issues along the way * feat: Reworked the broker_request_response built-in ability of components to be simpler. Instead of having to have a defined flow and then name that flow, it will automatically create a flow with a single broker_request_response component in it. Now there is a straightforward interating function call to allow components to issue a request and get streaming or non-streaming responses from that flow. * chore: fix the request_response example and remove the old one * docs: add broker request-response configuration * docs: added advanced_component_features.md * docs: add broker request-response configuration details * docs: add payload encoding and format to broker config * docs: add cache service and timer manager to advanced_component_features.md * docs: add configuration requirement for broker request-response * docs: update broker request-response section with configuration info * docs: a bit more detail about do_broker_request_response * docs: add link to advanced features page in table of contents * docs: add link to advanced features page * docs: reorder table of contents in index.md * docs: add custom components documentation * docs: Remove advanced component features from table of contents * docs: clean up a double inclusion of the same section * docs: small example change * chore: remove dead code * chore: add some extra comments to explain some test code * docs: Update description of STDIN input component Update the description of the STDIN input component to clarify that it waits for its output message to be acknowledged before prompting for the next input. This change is made in the `stdin_input.py` file. * chore: add is_broker_request_response_enabled method * chore: Some changes after review * feat: AI-129: add ability to specify a default value for a an environment variable in a .yaml config file (#43) * DATAGO-85484 Bump min python version --------- Co-authored-by: Cyrus Mobini <[email protected]> Co-authored-by: Art Morozov <[email protected]> Co-authored-by: Art Morozov <[email protected]>
cyrus2281
added a commit
that referenced
this pull request
Dec 3, 2024
* Added a RAG example for AI connector * Added delete option to vectordb * Changed id to ids
cyrus2281
added a commit
that referenced
this pull request
Dec 3, 2024
…ved docs and examples and better testing (#39) * Examples Update + Code Refactor (#25) * Removed StorageManager * Added examples for OpenAI, Bedrock, Anthropic, and VertexAI * Updating old examples (1/2) * Updating old examples (2/2) * Added support for temporary queue + UUID queue name (#26) * Add assembly component and auto-generated documents (#27) * Added the assembly component * Auto-generated documents * Added type check * Update the cache service expiry logic + Update the assembly component to use cache expiry for timeout * Moved assembly to the correct place * Added MoA Example + UUID Invoke Function (#28) * MoA example: Broadcast to multiple agents * Added MoA event manager, added uuid invoke_function + test, updated auto-generated docs * Added assembly layer to MoA example * Update documentation for new users + Refactored component_input & source_expression (#29) * Refactored component_input to input_selection * Updated, added, and enhanced the documentation with new users in mind * Refactored source_expression function to evaluate_expression (backward compatible) * Added tips and tricks section + info and examples on custom modules * tiny format update * tiny update * Fixed solace disconnection issues on shutting down (#30) * Add RAG example for AI connector + delete action for vector index (#31) * Added a RAG example for AI connector * Added delete option to vectordb * Changed id to ids * chore: Refactor make_history_start_with_user_message method (#32) Fix the method to not trim the first entry if it is a "system" role * Keep history depth needs to be a positive integer and test refactor (#33) * chore: Refactor clear_history_but_keep_depth method to handle negative depth values * chore: small change to how this is solved * chore: one more try * refactor: move utils_for_test_files.py to solace_ai_connector module * refactor: removed the orginal utils_for_test_files.py * refactor: update import statements in test files * refactor: add sys.path.append("src") to test files * refactor: standardize import order and sys.path.append in test files * refactor: a bit more test infrastructure changes * feat: allow component_module to accept module objects directly * feat: add types module import to utils.py * test: add static import and object config test * refactor: update test_static_import_and_object_config to use create_test_flows * refactor: Improve test structure and remove duplicate test case * fix: remove duplicate import of yaml module * refactor: Modify test config to use dict instead of YAML string * refactor: convert config_yaml from string to dictionary * refactor: update static import test to use pass_through component * test: Add delay component message passing test * feat: add test for delay component message processing * feat: Added a new test function (test_one_component) to make it very easy to just run some quick tests on a single input -> expected output tests on a single component * feat: added input_transforms to the test_one_component so that input transforms can be tested with it * chore: a bit of cleanup and new tests for test_one_component * chore: rename test_one_component because it was being picked up as a test by the pytest scanner * fix: fixed a typo * Fix for anthropic example (#35) * Updating version dependency (#37) * Fixed url and file name in getting started (#38) * Add guide for RAG (#39) * Added guide for RAG * update wording * Added link to other docs from RAG guide (#40) * chore: added a timeout setting for running component tests so that you can test situations where you don't expect any output (#34) * AI-124: Add a feature to provide simple blocking broker request/response ability for components (#42) * feat: add request_response_controller.py * feat: implement RequestResponseFlowManager and RequestResponseController classes * style: format code with black and improve readability * feat: implement RequestResponseController for flow-based request-response handling * feat: implement RequestResponseController for handling request-response patterns * fix: import SolaceAiConnector for type checking * refactor: restructure Flow class and improve code organization * feat: implement multiple named RequestResponseControllers per component * refactor: initialize request-response controllers in ComponentBase * test: add request_response_controller functionality tests * feat: finished implementation and added some tests * refactor: rename RequestResponseController to RequestResponseFlowController * refactor: rename RequestResponseController to RequestResponseFlowController * refactor: some name changes * fix: update test function names for RequestResponseFlowController * refactor: more name changes * Ed/req_resp_examples_and_fixes (#41) * feat: Added a request_response_flow example and fixed a few issues along the way * feat: Reworked the broker_request_response built-in ability of components to be simpler. Instead of having to have a defined flow and then name that flow, it will automatically create a flow with a single broker_request_response component in it. Now there is a straightforward interating function call to allow components to issue a request and get streaming or non-streaming responses from that flow. * chore: fix the request_response example and remove the old one * docs: add broker request-response configuration * docs: added advanced_component_features.md * docs: add broker request-response configuration details * docs: add payload encoding and format to broker config * docs: add cache service and timer manager to advanced_component_features.md * docs: add configuration requirement for broker request-response * docs: update broker request-response section with configuration info * docs: a bit more detail about do_broker_request_response * docs: add link to advanced features page in table of contents * docs: add link to advanced features page * docs: reorder table of contents in index.md * docs: add custom components documentation * docs: Remove advanced component features from table of contents * docs: clean up a double inclusion of the same section * docs: small example change * chore: remove dead code * chore: add some extra comments to explain some test code * docs: Update description of STDIN input component Update the description of the STDIN input component to clarify that it waits for its output message to be acknowledged before prompting for the next input. This change is made in the `stdin_input.py` file. * chore: add is_broker_request_response_enabled method * chore: Some changes after review * feat: AI-129: add ability to specify a default value for a an environment variable in a .yaml config file (#43) * DATAGO-85484 Bump min python version --------- Co-authored-by: Cyrus Mobini <[email protected]> Co-authored-by: Art Morozov <[email protected]> Co-authored-by: Art Morozov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DATAGO-83398
What is the purpose of this change?
PYPI currently does not allow trusted publishers for reusable workflows
So we need to use PYPI API token to release using the action
Anything reviews should focus on/be aware of?
See also
PR to solace-ai-connector