Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Search] Use Connector API for update_service_type #176306

Conversation

jedrazb
Copy link
Member

@jedrazb jedrazb commented Feb 6, 2024

Summary

Use connectors API for update service type operation, it includes 2 actions:

  • first clear the connector config
  • then update service type

Added unit tests. Also tested manually in serverless (this action is only used in serverless):

  1. Configure and start the google drive connector
  2. Sync data to search-xd index
  3. Stop the connector service
  • It's important that connector framework is stopped before changing service type, otherwise the framework might repopulate config fields once the config is cleared
  1. Change service type to sharepoint_online and update the connector config with new service type (everything else remains unchanged)
  2. Make sure that config doesn't include old config keys (from g drive)
  3. Start connectors framework
  4. Sync data from sharepoint
  5. Do it once again for fun
Screenshot 2024-02-06 at 15 51 49 Screenshot 2024-02-06 at 15 59 20

Checklist

@jedrazb jedrazb added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting Team:EnterpriseSearch v8.13.0 labels Feb 6, 2024
@jedrazb jedrazb marked this pull request as ready for review February 6, 2024 16:53
@jedrazb jedrazb requested a review from a team February 6, 2024 16:53
@jedrazb
Copy link
Member Author

jedrazb commented Feb 7, 2024

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
enterpriseSearch 2.7MB 2.7MB -412.0B

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream


export const updateConnectorServiceType = async (
client: ElasticsearchClient,
connectorId: string,
serviceType: string
) => {
const connectorResult = await fetchConnectorById(client, connectorId);

if (connectorResult?.value) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it no longer necessary to confirm the connector doc exists first?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, if document doesn't exist we return 404 from Connector API.

In status quo the check makes sure that we don't write partial update to the index directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be, it's fine if this errors out if the connector has been deleted. The only reason we're checking there is so we wouldn't have to do null checks inside the if block and can spread the value easily.

Copy link
Member

@sphilipse sphilipse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


export const updateConnectorServiceType = async (
client: ElasticsearchClient,
connectorId: string,
serviceType: string
) => {
const connectorResult = await fetchConnectorById(client, connectorId);

if (connectorResult?.value) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be, it's fine if this errors out if the connector has been deleted. The only reason we're checking there is so we wouldn't have to do null checks inside the if block and can spread the value easily.

@jedrazb jedrazb merged commit cc00868 into elastic:main Feb 7, 2024
18 checks passed
fkanout pushed a commit to fkanout/kibana that referenced this pull request Feb 7, 2024
## Summary

Use connectors API for update service type operation
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this pull request Feb 15, 2024
## Summary

Use connectors API for update service type operation
fkanout pushed a commit to fkanout/kibana that referenced this pull request Mar 4, 2024
## Summary

Use connectors API for update service type operation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:EnterpriseSearch v8.13.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants