[Upgrade Assistant] Update logic for handling reindex failures

[alisonelizabeth]

Fixes #124153

The Upgrade Assistant was incorrectly marking a reindexing process as a failure. This PR adjusts the logic for determining a reindex failure to only check for the existence of failures in the task API response.

It also fixes a minor spacing issue when rendering an error callout.

How to test

This is a hard one to add automated tests for (open to suggestions!), as it involves restarting Kibana to reproduce.

I was able to reproduce the bug (pre change) fairly consistently using the manual steps documented in #123817 (comment). With this change, the reindexing process should no longer fail.

To trigger an actual reindexing failure (post change):

1. Start up 6.x Elasticsearch and create some indices that contain a property with a text type. For example:

PUT /test/_doc/1
{
  "field": "my_string"
}

Alternatively, you can use this zip file, which already contains some indices: 6.8.16-data.zip

2. Start Elasticsearch with the 6.x snapshot created in step 1: yarn es snapshot --license=trial -E path.data=./path_to_6.x
3. Hack the UA code so that it creates the destination index with a bad mapping. This will trigger a reindex failure:

      createIndex = await esClient.indices.create({
        index: newIndexName,
        body: {
          settings,
          mappings: {
            properties: {
              field: { type: 'date' },
            },
          },
        },
      });

4. Start Kibana and navigate to Stack Management -> Upgrade Assistant -> ES deprecations
5. Start reindexing one of the indices. Note reindexing should fail and trigger error callout.

Screenshots

Before/after of spacing issue.

Before:

After:

[alisonelizabeth]

This is not a reliable check, as the documents could end up in an updated state. This could occur if Kibana is restarted and a user resumes the reindex process in Upgrade Assistant, which actually kicks off the reindex again. As the code is here, we would actually throw an error when there was not.

[alisonelizabeth]

More info on the task response body here: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-api-response-body

[alisonelizabeth]

@elasticmachine merge upstream

[elasticmachine]

Pinging @elastic/platform-deployment-management (Team:Deployment Management)

[alisonelizabeth]

@elasticmachine merge upstream

[alisonelizabeth]

@elasticmachine merge upstream

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: e3a0b1f
• Storybooks Preview

Metrics [docs]

✅ unchanged

History

• 💚 Build #22199 succeeded 9554953
• 💔 Build #22164 failed 8ea0b6d
• 💔 Build #22139 failed a32ac7e
• 💚 Build #21689 succeeded 519cb6c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[sebelga]

Code LGTM! I tried to reproduce locally the issue reverting your changes but could not get the error to be thrown. I tried both stopping Kibana at the beginning of the reindexing and in the middle of the 3rd step ("Reindex documents"). In both cases UA resumed and completed the reindex.

The code change make sense to me though. But it would be great to get to the bottom as exactly when this issue occurs. Could it be ES related?

[alisonelizabeth]

Thanks for the review @sebelga!

The code change make sense to me though. But it would be great to get to the bottom as exactly when this issue occurs. Could it be ES related?

The issue is likely occurring because we are actually kicking off another reindex operation when the user clicks "resume" in the UI so the documents may have already been created, and end up in an updated state in the ES task response.