-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: Milvus error handling #292
Conversation
* add release branch to PR triggers * omit vectar dest e2e test
* create deterministic id for upload use * fix id in sql connector
Few fsspec connectors: SFTP, Azure, Box and GCS havedate_modified and date_created fields of FileDataSourceMetadata class were of type float | None instead of str | None, modified code creating the metadata to cast float timestamps to strings. --------- Co-authored-by: Filip Knefel <[email protected]>
* Snowflake query data binding fix * Enable SingleSource source connector entry * Fix Snowflake nan issue * Make Singlestore connector more robust against SQL injection * Clean sql upload and add query debug log * Make SQLite connector more robust against sql injection * SQL injection fixes; Changelog and version update * Optimize memory usage of Snowflake uploader * Changelog update: Optimize memory usage of Snowflake uploader
…es will default to the fixture caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to function scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future."
@@ -74,3 +74,6 @@ omit=[ | |||
"unstructured_ingest/ingest_backoff/*", | |||
"unstructured_ingest/enhanced_dataclass/*" | |||
] | |||
|
|||
[tool.pytest.ini_options] | |||
asyncio_default_fixture_loop_scope="function" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without it was getting
.venv/lib/python3.10/site-packages/pytest_asyncio/plugin.py:208: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset.
The event loop scope for asynchronous fixtures will default to the fixture caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to function scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session"
) | ||
except MilvusException as milvus_exception: | ||
raise DestinationConnectionError( | ||
f"failed to precheck Milvus: {str(milvus_exception.message)}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have to use str(milvus_exception.message)
, otherwise just for str(milvus_exception)
was getting
Failed to redact: 2024-12-05 17:16:18,380 MainProcess ERROR Step precheck failure: MilvusUploader: [DestinationConnectionError] failed to precheck milvus: <MilvusException: (code=<bound method _InactiveRpcError.code of <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "Token not found or unauthorized: The provided token does not exist in our records, or you do not have the necessary permissions to use this token. Please ensure you have the correct token."
debug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-12-05T17:16:18.378661811+01:00", grpc_status:2, grpc_message:"Token not found or unauthorized: The provided token does not exist in our records, or you do not have the necessary permissions to use this token. Please ensure you have the correct token."}"
It adds more context to the messages that are returned to UI.
Was:
Example of what will be added at end:
the dim (3072) of field data(embeddings) is not equal to schema dim (1536): invalid parameter[expected=1536][actual=3072])
Full example:
2024-12-05 15:14:03,249 MainProcess ERROR Step precheck failure: MilvusUploader: [DestinationConnectionError] failed to precheck Milvus: Retry run out of 5 retry times, message=Token not found or unauthorized: The provided token does not exist in our records, or you do not have the necessary permissions to use this token. Please ensure you have the correct token.