Update cat API docs with guidance on intended use cases

[cjcenizal]

Per conversation with @jaymode, the cat API is intended for consumption via Console and the command line. It's not intended for consumers who are writing application code.

Kibana consumes the cat API at numerous points (example: elastic/kibana#56532). To help Kibana engineers (and other application developers) migrate away from cat and avoid consuming it in the future, I think we should update the docs with a clear explanation of the purpose behind the cat API, including details on why it's problematic to consume it in application code, and advise application developers to use the other APIs.

[elasticmachine]

Pinging @elastic/es-core-features (:Core/Features/CAT APIs)

[elasticmachine]

Pinging @elastic/es-docs (>docs)

[jaymode]

including details on why it's problematic to consume it in application code

The Elasticsearch team discussed this a few months back during a discussion regarding system indices. Ideally by keeping the system and hidden indices out of the cat APIs, we can keep the APIs simpler. Additionally, we would like some additional freedom in the cat API output such that we could add a new column (may be in the middle of existing columns), which may more easily be handled by the human eye than an application (depending on how its parsing is implemented). If we have applications depending on the cat API and we don't expect them to be depending on the cat API, a change we make could break the application which is something we would like to avoid.

@jasontedor may have other points and be able to articulate the reasoning better than I can.

[jasontedor]

I think our [docs] allude to the reason that @jaymode expands on well:

JSON is great…​ for computers. Even if it’s pretty-printed, trying to find relationships in the data is tedious. Human eyes, especially when looking at a terminal, need compact and aligned text. The cat API aims to meet this need.

The cat API is meant for human consumption, and interaction with via command-line tools. It is not meant for consumption in applications, it's not designed/developed for that, and we don't intend for it to be.