Documentation improvement

[dai-chen]

Currently the SQL plugin documentation is only a README with reference to technical docs https://opendistro.github.io/for-elasticsearch-docs/docs/sql/. However, both are missing important details and not sufficient to be served as a good reference for user when they have trouble. Particularly, here are things missing and could be improved:

1. Description: name and briefly describe its functionality.
2. Syntax: rigor syntactical form in grammar.
3. Semantics: semantical requirement, ex. specification for function.
4. Use Cases: typical use case and example with response to help understand.
5. Explain: ES DSL generated by our plugin for each sample query.
6. Constraints: any case we don't support for now for some reason.

Apart from these items, we can also improve our documentation in other aspect, for example enable Wiki on GitHub to host our development docs in https://github.com/opendistro-for-elasticsearch/sql/tree/master/docs/dev. This is more focused on internal implementation as plugin developer and contributor facing design docs instead of user facing guidance. Because of its different intention, it is not supposed to have overlap with existing technical documentation.

[dai-chen]

After scoping this work can be divided into 2 parts: docs for developer and for user. For the former, we plan to add more design docs and instructions for contributor manually so they can get involved easily.

As for user facing docs, we evaluated different options such as executable documentation and generation from code etc. At this stage our goal is to generate docs with small efforts and in sync with existing codebase as our architecture is tend to change. And we don't plan to use these docs as integration test for now.

Based on our requirements, we decide to add a set of "Scaffold" to automated docs generation from existing codebase. The generated docs still have the flexibility to be executable for testing and can be polished further by tech writer.

[dai-chen]

Here is an incomplete list of what we want to cover in docs for user. 4 sub issues will be created soon.

• Interfaces
  • Endpoint
    • Query via GET/POST
    • Explain
  • JDBC protocol
    • Request & response format
    • Error response format
    • Prepared statement
  • Other formats
    • Default DSL
    • CSV
    • Raw
• Administration
  • Configuration
    • Plugin settings
  • Monitoring
    • Stats
• DQL (Data Query Language)
  • Basics
    • SELECT
    • FROM
    • WHERE
    • GROUP BY
    • HAVING
    • ORDER BY
    • LIMIT
  • Functions
    • SQL functions
    • ES special functions
  • Full text
  • Complex
    • PartiQL support
    • Subquery
    • Join
    • Multi-query
  • Metadata:
    • SHOW/DESCRIBE
• DML (Data Manipulation Language)
  • DELETE

[dai-chen]

In my PR opendistro/for-elasticsearch-docs#146, Pandoc seems not adapt to our need very well to convert ReST to Markdown, particularly for TOC, table and code block. I did post-processing manually this time. For more docs generated in future, we should consider adding Markdown generator in our code rather than fully depends on Pandoc.

Anyway, the docs for plugin basic usage is merged and up now: https://opendistro.github.io/for-elasticsearch-docs/docs/sql/endpoints/.

[dai-chen]

Phase 2 is done: #363

[dai-chen]

Phase 3: #380