diff --git a/README.md b/README.md index cbf13ce..95a0682 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,15 @@ # Open Distro Elasticsearch SQL CLI +ODFE: Open Distro for Elasticsearch -SQL CLI is a stand alone Python application and can be launched by a wake word `escli`. It serves as a support only for -[Open Distro SQL plugin for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/). You can move -it around to other machines without having to install Elasticsearch on them. +ODFE SQL CLI is a stand alone Python application and can be launched by a wake word `odfesql`. It serves as a support only for +[Open Distro SQL plugin for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/). You can run +it on any OS we support, and connect to any valid remote endpoint without installing Elasticsearch. ## Installation -- `pip install escli` -- escli is compatible with Python 3, and python 2 is going away soon https://pythonclock.org/ +- `pip install odfesql` +- odfe sql cli is compatible with Python 3, because Python 2 is no longer maintained since 01/01/2020 https://pythonclock.org/ ## Configuration @@ -25,10 +26,11 @@ See the file itself for a description of all available options. - Enable horizontal display (by default) and vertical display when output is too wide - Pagination for long output - Syntax highlighting -- Connect to Elasticsearch node/cluster with authentication on either ES localhost, Open Distro ES, or AES, +- Connect to Elasticsearch node/cluster with/without security on either **ES localhost, Open Distro ES, or AWS Elasticsearch Domain**. +Refer to [test plan](./tests/test_plan.md) on how to connect to different instance with/without security - Load Config file - Run single query from Command Line with parameters - - *endpoint:* no need to specify a parameter, anything follow by wake word `escli` should be the endpoint. + - *endpoint:* no need to specify a parameter, anything follow by wake word `odfesql` should be the endpoint. By default, it’s http://localhost:9200 - *--help:* help page for options and params - *-q:* follow by a single query user wants to run. @@ -45,7 +47,8 @@ See the file itself for a description of all available options. ## Basic Usage -![](./screenshots/usage.gif) +- The CLI supports all types of query that ODFE SQL supports. See [ODFE SQL basic usage](https://github.com/opendistro-for-elasticsearch/sql#basic-usage) +- ![](./screenshots/usage.gif) @@ -69,30 +72,3 @@ See the [LICENSE](./LICENSE.TXT) file for our project's licensing. We will ask y ## Copyright Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. - - - -## Development Guide -### Build the application in development -- `pip install virtualenv` -- `virtualenv venv` to create virtual environment for **Python 3** -- `source ./venv/bin/activate` activate virtual env. -- `cd` into project root folder. -- `pip install --editable .` will install all dependencies from `setup.py`. - -### Run CLI -- Start an Elasticsearch instance from either local, Docker with Open Distro SQL plugin, or AWS Elasticsearch -- To launch the cli, use wake word `escli` followed by endpoint of your running ES instance. If not specifying -any endpoint, it uses http://localhost:9200 by default. - -### Testing Procedure -- Prerequisites - - Build the application - - Start a local Elasticsearch instance with - [Open Distro SQL plugin for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/) installed - and listening at http://localhost:9200. -- `pip install -r requirements-dev.txt` Install test frameworks including Pytest and mock. -- `cd` into `tests` and run `pytest` - -### Style -- Use [black](https://github.com/psf/black) to format code, with option of `--line-length 120` diff --git a/development_guide.md b/development_guide.md new file mode 100644 index 0000000..f571062 --- /dev/null +++ b/development_guide.md @@ -0,0 +1,59 @@ +## Development Guide +### Development Environment Set Up +- `pip install virtualenv` +- `virtualenv venv` to create virtual environment for **Python 3** +- `source ./venv/bin/activate` activate virtual env. +- `cd` into project root folder. +- `pip install --editable .` will install all dependencies from `setup.py`. + +### Run CLI +- Start an Elasticsearch instance from either local, Docker with Open Distro SQL plugin, or AWS Elasticsearch +- To launch the cli, use wake word `odfesql` followed by endpoint of your running ES instance. If not specifying +any endpoint, it uses http://localhost:9200 by default. + +### Testing +- Prerequisites + - Build the application + - Start a local Elasticsearch instance (OSS) with + [Open Distro SQL plugin for Elasticsearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/) installed + and listening at http://localhost:9200. +- Pytest + - `pip install -r requirements-dev.txt` Install test frameworks including Pytest and mock. + - `cd` into `tests` and run `pytest` +- Refer to [test_plan](./tests/test_plan.md) for manual test guidance. + +### Style +- Use [black](https://github.com/psf/black) to format code, with option of `--line-length 120` + +## Release guide + +- Package Manager: pip +- Repository of software for Python: PyPI + +### Workflow + +1. Update version number + 1. Modify the version number in `_init_.py` under `escli` package. It will be used by `setup.py` for release. +2. Create/Update `setup.py` (if needed) + 1. For more details refer to https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py +3. Update README.md, Legal and copyright files(if needed) + 1. Update README.md when there is a critical feature added. + 2. Update `THIRD-PARTY` files if there is a new dependency added. +4. Generate distribution archives + 1. Make sure you have the latest versions of `setuptools` and `wheel` installed: `pip install --user --upgrade setuptools wheel` + 2. Run this command from the same directory where `setup.py` is located: `python3 setup.py sdist bdist_wheel` +5. Upload the distribution archives to TestPyPI + 1. Register an account on [testPyPI](https://test.pypi.org/) + 2. `pip install --user —upgrade twine ` + 3. `python3 -m twine upload --repository-url https://test.pypi.org/legacy/ dist/*` +6. Install your package from TestPyPI and do manual test + 1. `pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple odfe-sql` +7. Upload to PyPI + 1. Register an account on [PyPI](https://pypi.org/), note that these are two separate servers and the credentials from the test server are not shared with the main server. + 2. Use `twine upload dist/*` to upload your package and enter your credentials for the account you registered on PyPI. +8. Install your package from PyPI using `pip install [your-package-name]` + +### Reference +- https://medium.com/@joel.barmettler/how-to-upload-your-python-package-to-pypi-65edc5fe9c56 +- https://packaging.python.org/tutorials/packaging-projects/ +- https://packaging.python.org/guides/using-testpypi/ \ No newline at end of file diff --git a/escli/__init__.py b/escli/__init__.py index fe9ada0..77afd36 100644 --- a/escli/__init__.py +++ b/escli/__init__.py @@ -12,4 +12,4 @@ See the License for the specific language governing permissions and limitations under the License. """ -__version__ = "1.0.7" +__version__ = "1.0" diff --git a/escli/essqlcli.py b/escli/essqlcli.py index b6446d2..acb5332 100644 --- a/escli/essqlcli.py +++ b/escli/essqlcli.py @@ -45,7 +45,7 @@ class ESSqlCli: - """ESSqlCli instance is used to build and run the ES SQL CLI.""" + """ESSqlCli instance is used to build and run the ODFE SQL CLI.""" def __init__(self, esclirc_file=None, always_use_pager=False): # Load conf file @@ -123,7 +123,7 @@ def run_cli(self): while True: try: - text = self.prompt_app.prompt(message="escli> ") + text = self.prompt_app.prompt(message="odfesql> ") except KeyboardInterrupt: continue # Control-C pressed. Try again. except EOFError: diff --git a/setup.py b/setup.py index 4fe681a..9196f33 100644 --- a/setup.py +++ b/setup.py @@ -41,7 +41,7 @@ long_description = fh.read() setup( - name="escli", + name="odfesql", author="Zhongnan", author_email="zhongnan.su@outlook.com", version=version, @@ -53,7 +53,7 @@ long_description=long_description, long_description_content_type="text/markdown", install_requires=install_requirements, - entry_points={"console_scripts": ["escli=escli.main:cli"]}, + entry_points={"console_scripts": ["odfesql=escli.main:cli"]}, classifiers=[ "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", diff --git a/tests/test_plan.md b/tests/test_plan.md new file mode 100644 index 0000000..68abac1 --- /dev/null +++ b/tests/test_plan.md @@ -0,0 +1,36 @@ +# Test Plan + The purpose of this checklist is to guide you through the basic usage of ODFE SQL CLI, as well as a manual test process. + +## Display + * [ ] Test pagination with different output length / width. + * [ ] Test table formatted output. + * [ ] Test successful conversion from horizontal to vertical display with confirmation. + * [ ] Test warning message when output > 200 rows of data. (Limited by ODFE SQL syntax) + + +## Connection +* [ ] Test connection to a local Elasticsearch instance + * [ ] Standard Elastic version, with/without authentication by [X-pack security](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/security-getting-started.html) + * [ ] OSS version, no authentication + * [ ] OSS version, install [ODFE Security plugin](https://opendistro.github.io/for-elasticsearch-docs/docs/install/plugins/) to enable authentication and SSL +* [ ] Test connection to [AWS Elasticsearch domain](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html) (with ODFE SQL plugin installed by default) +* [ ] Test connection fail when connecting to invalid endpoint +* [ ] Test reconnection when connection lost during execution + + +## Execution +* [ ] Test successful execution given a query +* [ ] Test unsuccessful execution with an invalid SQL query +* [ ] Test load config file + + +## Nested commands with params +* [ ] Test dump query result to a file +* [ ] Test explain option `-e` +* [ ] Test query and format option `-q`, `-f` +* [ ] Test vertical option `-v` + + +## OS and Python compatibility +* [ ] Manually test on Linux(Ubuntu), Windows and mac-os. +* [ ] Test against python 3.X versions \ No newline at end of file diff --git a/tests/test_plan.wiki b/tests/test_plan.wiki deleted file mode 100644 index b5785f1..0000000 --- a/tests/test_plan.wiki +++ /dev/null @@ -1,29 +0,0 @@ - = Test Plan = - - == Display == - * [ ] Test pagination with different output length / width. - * [ ] Test table formatted output. - * [ ] Test successful conversion from horizontal to vertical with confirmation. - * [ ] Test output message when large output > 200 rows of data. - - - == Connection == - * [ ] Test connecting to a local database. - * [ ] Test connection fail when connect to invalid endpoint - * [ ] Test reconnection when connection lost during execution - * [ ] Test client selection when connect to different type of endpoint(Open Distro, Local ES, AES) - - == Execution == - * [ ] Test successful execution given a query. - * [ ] Test unsuccessful execution with a invalid SQL query. - * [ ] Test load config - - - == Nested commands with params == - * [ ] Test write query result to a file. - * [ ] Test explain option - * [ ] Test query and format option - * [ ] Test vertical option - - == Cross-Platform availability == - * [ ] Manually test on Linux, Windows and mac-os. \ No newline at end of file