Java high-level REST client completeness for xpack APIs

[elasticmachine]

Original comment by @javanna:

This is the xpack side of #27205 . This is a meta issue to track completeness of the Java REST high-level Client in terms of supported xpack API. The following list includes all the REST API that xpack exposes to date, and that are also exposed by the Transport Client. The ones marked as done are already supported by the high-level REST client, while the others need to be added. Every group is sorted based on an estimation around how important the API is, from more important to less important. Each API is also assigned a rank from 1 to 3 that expresses how difficult adding support for it is expected to be.

Please Note The existing actions are in x-pack/ which is governed by the Elastic License, not the Apache License. Engineers who take these should strive to pull the Actions into Apache License controlled territory, so the high level rest client does not use any Elastic License bits. These should live in x-pack/protocol, which is OSS.

Please also note The APIs themselves should follow the spec for how they are named. This means they should be in the XPackClient, not the HighLevelRestClient. This would mean you end up with hlRestClient.xpack().watcher() as an example

Please Please also also note Prepend your commit msgs with HLRC:

Watcher API

• put watch * Rest HL client: Add put watch action #32026 @spinscale
• get watch * HLRC: Add get watch API #35531 @jimczi
• delete watch * HLRC: Add delete watch action #32337 @ywelsch
• execute watch * HLRC: execute watch API #35868 @romseygeek
• ack watch * Add support for 'ack watch' to the HLRC. #33962 @jtibshirani
• activate watch * HLRC: Add activate watch action #33988 @iverase
• deactivate watch * HLRC: Deactivate Watch API #34192 @not-napoleon
• watcher stats ** @imotov
• watcher stop * [HLRC] Add Start/Stop Watch Service APIs. #34317 @jdconrad
• watcher start * [HLRC] Add Start/Stop Watch Service APIs. #34317 @jdconrad

Machine Learning API

• ml info action @vladimirdolzhenko [HLRC] XPack ML info action #35777
• Create calendar ** (highest prio) @davidkyle HLRC: ML PUT Calendar #33362
• Delete calendar * @davidkyle HLRC: Delete ML calendar #33775
• Add job to calendar * [ @benwtrent | HLRC: ML Add Job to Calendar API #35666 ]
• Delete job from calendar * [ @benwtrent | HLRC: ML Delete job from calendar #35713 ]
• Add scheduled events to calendar * [ @benwtrent | HLRC ML Add Event To Calendar API #35704 ]
• Delete scheduled events from calendar * [ @benwtrent | HLRC: ML Delete event from Calendar #35760 ]
• Get calendars * @davidkyle HLRC: Get ML calendars #33760
• Get scheduled events * [ @benwtrent | HLRC: ML Get Calendar Events #35747 ]
• Create datafeed ** (highest prio) [HLRC][ML] Add ML put datafeed API to HLRC #33603 @dimitris-athanasiou
• Delete datafeed * [HLRC][ML] Add ML delete datafeed API to HLRC #33667 @dimitris-athanasiou
• Start datafeed * [ @benwtrent | HLRC: ML start data feed API #33898 ]
• Stop datafeed * [ @benwtrent | HLRC: ML Stop datafeed API #33946 ]
• Get datafeed info * [HLRC][ML] Add ML get datafeed API to HLRC #33715 @dimitris-athanasiou
• Get datafeed statistics ** [ @benwtrent | HLRC: ML Adding get datafeed stats API #34271 ]
• Preview datafeed * [ @benwtrent | HLRC: ML Add preview datafeed api #34284 ]
• Update datafeed * [ @benwtrent | HLRC: Adding Update datafeed API #34882 ]
• Create job ** (highest prio) [ @droberts195, @benwtrent | Adding ML HLRC wrapper and put_job API call #32726 ]
• Delete job * [ @benwtrent | HLRC: adding machine learning delete job #32820 ]
• Open job * [ @benwtrent | HLRC: adding machine learning open job #32860 ]
• Close job * [ @benwtrent | HLRC: ML Close Job #32943 ]
• Get job info * [ @benwtrent | HLRC: Add ML Get Job #32960 ]
• Get job statistics ** [ @benwtrent | HLRC: Adding ML Job stats #33183 ]
• Flush job * [ @benwtrent | HLRC: ML Flush job #33187]
• Post data to job ** [ @benwtrent | HLRC: ML Post Data #33443 ]
• Update job * (done after put job API) [ @benwtrent | HLRC: ML Update Job #33392 ]
• Forecast job * [ @benwtrent | HLRC: ML Forecast Job #33506 ]
• Delete Forecast * [ @benwtrent | HLRC: ML Delete Forecast API #33526 ]
• Delete model snapshot * @edsavage [HLRC][ML] Add ML delete model snapshot API #35537
• Get model snapshot info * @edsavage [HLRC][ML] Add ML get model snapshots API #35487
• Revert model snapshot * @edsavage [HLRC][ML] Add ML revert model snapshot API #35750
• Update model snapshot ** @edsavage [HLRC][ML] Add ML update model snapshot API (#35537) #35694
• Get buckets * HLRC: Add ML Get Buckets API #33056 @dimitris-athanasiou
• Get overall buckets * HLRC: Add ML get overall buckets API #33297 @dimitris-athanasiou
• Get categories * HLRC: Add ML get categories API #33465 @edsavage
• Get influencers * HLRC: Add ML get influencers API #33389 @dimitris-athanasiou
• Get records * HLRC: Add ML Get Records API #33085 @dimitris-athanasiou
• Find file structure * [HLRC][ML] Add ML find file structure API #35833 @droberts195
• Put filter * [ @benwtrent | HLRC: Add ML API PUT filter #35175 ]
• Get filters * [ @benwtrent | HLRC: Adding ml get filters api #35502 ]
• Update filters * [ @benwtrent | HLRC: Adding ML Update Filter API #35522 ]
• Delete filters * HLRC: Add ML delete filter action #35382 @iverase

Machine Learning REST only API

• delete expired data * @edsavage [HLRC][ML] Add delete expired data API #35906

Security API

• clear cache * HLRC: add support for the clear realm cache API #35163 @jaymode
• has privileges ** HLRC: Add "_has_privileges" API to Security Client #35479 @tvernum
• put role ** [HLRC] Put Role #36209 @albertzaharovits
• get roles ** 35787 @jkakavas
• clear role cache * @bleskes, HLREST: Add Clear Roles Cache API #34187
• delete role * HLRC: Delete role API #34620 @ywelsch - Has no API docs in the api ref
• create role mapping ** Create/Update role mapping API #34171 @bizybot
• modify role mapping ** Create/Update role mapping API #34171 @bizybot
• delete role mapping * [HLRC] Add support for Delete role mapping API #34531 @bizybot
• get role mappings ** [HLRC] Add support for get role mappings API #34637 @bizybot
• get ssl certificates ** HLRC: Get SSL Certificates API #34135 @jkakavas
• post create token * HLRC: Add security Create Token API #34791 @tvernum
• delete invalidate token * HLRC: Add InvalidateToken security API #35114 @tvernum
• put user ** (highest prio) HLRest: add xpack put user API #32332 @jaymode
• get users ** HLRC: Add get users action #36332 @nknize
• delete user * HLRC: Add delete user action #35294 @iverase
• disable user HLRC: add enable and disable user API support #33481 @jaymode
• enable user HLRC: add enable and disable user API support #33481 @jaymode
• change password HLRC: add change password API support #33509 * @jkakavas
• xpack.security.get_privileges [HLRC] Add support for get application privileges API #35556 @jkakavas
• xpack.security.delete_privileges. Add Delete Privileges API to HLRC #35454 @tlrx
• xpack.security.put_privileges [HLRC] Add support for put privileges API #35679 @bizybot
• xpack.security.get_user_privileges HLRC: Implement get-user-privileges API #36292 @tvernum

Security REST only API

• authenticate * (highest prio) HLRest: add security authenticate API #33552 @albertzaharovits

Miscellaneous API

• xpack info * HLREST: Add x-pack-info API #31870 @nik9000
• xpack usage ** HLRC: Add xpack usage api #31975 @rjernst

Graph API

• graph explore *** GraphClient for the high level REST client and associated tests #32366 @markharwood

Licensing API

• put license ** Rest HL client: Add put license action #32214 @imotov
• get license ** @imotov
• delete license * HLRC: Add Delete License API #32586 @ywelsch
• post start trial * add start trial API to HLRC #33406 @andyb-elastic
• get trial status * HLRC: add support for get license basic/trial status API #33176 @javanna
• post start basic * HLRC: Add support for XPack Post Start Basic Licence API #33606 @vladimirdolzhenko
• get basic status * HLRC: add support for get license basic/trial status API #33176 @javanna

Rollup

• Create Job Add create rollup job api to high level rest client #33521 @tlrx
• Delete Job Add delete rollup job support to HL REST Client #34066 @pcsanwald
• Start Job Add start rollup job support to HL REST Client #34623 (@cbuescher)
• Stop Job Add stop rollup job support to HL REST Client #34702 (@cbuescher)
• Get Job * HLRC: Add get rollup job #33921 (@nik9000)
• Get Rollup Capabilities Add GetRollupCaps API to high level rest client #32880 (@polyfractal)
• Get Rollup Index Capabilities [HLRC] Add GetRollupIndexCaps API #35102 * (@polyfractal)
• Rollup Search ** HLRC: Add rollup search #36334 (@nik9000)

Migration

• get Migration Assistance * HLRC: migration get assistance API #32744 @javanna
• upgrade Migration ** HLRC: migration api - upgrade #34898 @pgomulka
• get Deprecation Info * HLRC: Get Deprecation Info API #36279 @hub-cap

Monitoring API

• monitoring bulk

Index Lifecycle Management

See #33100

CCR

See #33824

[elasticmachine]

Original comment by @pickypg:

The _xpack/monitoring/_bulk API is not intended to be supported by outside clients. Its use is only meant for Kibana, Beats, APM Server, and Logstash.

Ideally we will even be removing it once we get to Metricbeat-based monitoring data.

[elasticmachine]

Original comment by @javanna:

thanks for clarifying @pickypg !

[elasticmachine]

Original comment by @tvernum:

@javanna What does

Security Transport Client only API

mean - is that intending to say that there is no REST interface, in which case has privileges is handled by rest action RestHasPrivilegesAction

[elasticmachine]

Original comment by @javanna:

@tvernum I couldn't find the corresponding REST action for some reason. I am going to edit the description and move has privileges with the other API.

[elasticmachine]

Original comment by @javanna:

I updated the description of the issue by assigning each API a rank from 1 to 3 based on how difficult it should be to add support for it to the high-level REST client. Criterias were mainly how big the request is to serialize and how big the response is to parse back.

[elasticmachine]

Original comment by @tvernum:

@javanna Did you have a plan about how we'd ship an X-Pack enabled client?
From the javadocs on the existing OSS client, it seems like the best course of action is a single

public class RestXPackClient extends RestHighLevelClient {

But there's options to have separate clients per feature (I can't see a reason to do that) or to take the same approach as the current SecurityClient and wrap rather than extend the RestHighLevelClient.

[elasticmachine]

Original comment by @javanna:

@tvernum I envisioned extending like you mentioned above, wrapping is an option too though. I don't have a strong opinion, we should give it a try and see how things look.

[nik9000]

One down!