Added Create index workflow step

[owaiskazi19]

Description

• Added Create index and Delete index workflow steps
• Tests

Request

{
                    "id": "create_index",
                    "type": "create_index",
                    "user_inputs": {
                        "index_name": "append-4",
                        "configurations": {
                            "settings": {
                                "index": {
                                    "number_of_shards": 2,
                                    "number_of_replicas": 1
                                }
                            },
                            "mappings": {
                                "_doc": {
                                    "properties": {
                                        "age": {
                                            "type": "integer"
                                        }
                                    }
                                }
                            },
                            "aliases": {
                                "sample-alias1": {}
                            }
                        }
                    }
                }

Response for Create Index Provisioning

[2024-03-13T01:20:08,706][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting create_index.
[2024-03-13T01:20:08,709][INFO ][o.o.p.PluginsService     ] [ip-172-31-56-214] PluginService:onIndexModule index:[append-4/stmxO9k-SqyUqqf1RC6dRQ]
[2024-03-13T01:20:08,712][INFO ][o.o.c.m.MetadataCreateIndexService] [ip-172-31-56-214] [append-4] creating index, cause [api], templates [], shards [2]/[1]
[2024-03-13T01:20:08,725][INFO ][o.o.p.PluginsService     ] [ip-172-31-56-214] PluginService:onIndexModule index:[append-4/stmxO9k-SqyUqqf1RC6dRQ]
[2024-03-13T01:20:08,773][INFO ][o.o.f.w.CreateIndexStep  ] [ip-172-31-56-214] Created index: append-4
[2024-03-13T01:20:08,785][INFO ][o.o.f.i.FlowFrameworkIndicesHandler] [ip-172-31-56-214] updated resources created of sN5lNY4B10dFQcqyDKnd
[2024-03-13T01:20:08,785][INFO ][o.o.f.w.CreateIndexStep  ] [ip-172-31-56-214] successfully updated resource created in state index: .plugins-flow-framework-state
[2024-03-13T01:20:08,785][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished create_index.
[2024-03-13T01:20:08,785][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] Provisioning completed successfully for workflow sN5lNY4B10dFQcqyDKnd
[2024-03-13T01:20:08,810][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] updated workflow sN5lNY4B10dFQcqyDKnd state to COMPLETED

Response for Deprovisioning

[2024-03-13T19:28:30,990][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting (deprovision_create_index) create_index.
[2024-03-13T19:28:30,993][INFO ][o.o.c.m.MetadataDeleteIndexService] [ip-172-31-56-214] [append-5/i53ATtamSneP0CUF7EMAZQ] deleting index
[2024-03-13T19:28:31,016][INFO ][o.o.f.w.DeleteConnectorStep] [ip-172-31-56-214] Deleted index: append-5

Issues Resolved

Resolves #103

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 82.60870% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 72.55%. Comparing base (10915aa) to head (2aaa63d).

Files	Patch %	Lines
...search/flowframework/workflow/CreateIndexStep.java	80.00%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #574      +/-   ##
============================================
+ Coverage     72.48%   72.55%   +0.06%     
- Complexity      657      659       +2     
============================================
  Files            80       80              
  Lines          3416     3399      -17     
  Branches        270      268       -2     
============================================
- Hits           2476     2466      -10     
+ Misses          822      816       -6     
+ Partials        118      117       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[joshpalis]

@owaiskazi19 have you tested this create index step to attach a default search pipeline using a pipeline_id from a previous step. This should be part of the settings of the index configuration, just wanted to double check

[amitgalitz]

@owaiskazi19 have you tested this create index step to attach a default search pipeline using a pipeline_id from a previous step. This should be part of the settings of the index configuration, just wanted to double check

+1

[owaiskazi19]

@owaiskazi19 have you tested this create index step to attach a default search pipeline using a pipeline_id from a previous step. This should be part of the settings of the index configuration, just wanted to double check

@joshpalis @amitgalitz I tested with ingest pipeline, but should be the same for search as well. I have pasted the template, response and mapping here.

[dbwiddis]

Code LGTM but see comment.