Add allow_delete parameter to Deprovision API

[dbwiddis]

Description

Adds an allow_delete parameter to the Deprovision Workflow API which can contain a comma-delimited list of resource ids which may be deleted.

When the parameter does not contain resources that require it, the deprovision API returns a 403 response identifying the remaining resources.

Added DeleteIndexStep, DeleteIngestPipelineStep, and DeleteSearchPipelineStep which require the use of this parameter.

Added a denylist to prevent the use of these steps when provisioning workflows, and filtered the Get Workflow Steps API return to remove them from that list as well

Issues Resolved

• [FEATURE] Support optional force deletion of certain resources #748
• [FEATURE] Implement Notification for Manual Deletion of Certain Resources #579

Documentation

Please review:

• Add documentation for Deprovision Workflow API allow_delete parameter documentation-website#7639

Note to Reviewers

This implementation causes a change in API behavior.

Current behavior is to simply remove these unimplemented deletions from the resource list and silently fail, giving a 200 (OK) on successful deprovision while the resources remain. (As a concrete example, if an index was created and included in the created resources list, deprovisioning would simply remove it from the resources list without taking any action (deleting the index).)

This is misleading, as an attempt to provision again will fail with a pre-existing index, so I think the change is justified, but should be made clear to anyone using existing scripting relying on this behavior.

I think 403 (Forbidden) is the best response type here, but I'm open to discussion here if you have a better proposal.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 93.08176% with 11 lines in your changes missing coverage. Please review.

Project coverage is 75.41%. Comparing base (4a62386) to head (7de36c3).

Files	Patch %	Lines
...search/flowframework/workflow/DeleteIndexStep.java	89.65%	2 Missing and 1 partial ⚠️
...owframework/workflow/DeleteIngestPipelineStep.java	89.65%	2 Missing and 1 partial ⚠️
...owframework/workflow/DeleteSearchPipelineStep.java	89.65%	2 Missing and 1 partial ⚠️
...wframework/rest/RestDeprovisionWorkflowAction.java	66.66%	0 Missing and 1 partial ⚠️
...pensearch/flowframework/workflow/WorkflowStep.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #763      +/-   ##
============================================
+ Coverage     74.61%   75.41%   +0.79%     
- Complexity      783      816      +33     
============================================
  Files            84       88       +4     
  Lines          3873     3998     +125     
  Branches        356      362       +6     
============================================
+ Hits           2890     3015     +125     
  Misses          825      825              
  Partials        158      158              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ohltyler]

At a high level LGTM, thanks for adding this! Meets the frontend requirements and will make deprovisioning/reprovisioning a much smoother experience. I concur with a 403 response on resources left over, I think that makes the most sense.

[owaiskazi19]

Looks good overall with one comment

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/flow-framework/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/flow-framework/backport-2.x
# Create a new branch
git switch --create backport/backport-763-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 7e7e99305d88b9f8ece719fd0b3b2183e8a6b82b
# Push it to GitHub
git push --set-upstream origin backport/backport-763-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/flow-framework/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-763-to-2.x.