Merge branch 'update-ansible-docs-remove-ewc' of pgithub.com:winem/st…

…2docs into update-ansible-docs-remove-ewc

Dockerfile

@@ -1,7 +1,7 @@
 FROM ubuntu:18.04
 
 RUN apt-get -qq update && apt-get -q install -y \
-    git \
+    curl git \
     libffi-dev libldap2-dev libsasl2-dev libssl-dev \
     python3-dev python3-pip python-virtualenv
 

Makefile

@@ -29,7 +29,13 @@ comma := ,
 COMPONENT_PYTHONPATH = $(subst $(space_char),:,$(realpath $(COMPONENTS)))
 
 REQUIREMENTS := requirements.txt st2/requirements.txt
-PIP_VERSION := 20.0.2
+DOCS_VERSION := $(shell cat version.txt | cut -d '.' -f 1,2)
+ST2_BRANCH := v$(DOCS_VERSION)
+ifneq (,$(findstring dev,$(ST2_BRANCH)))
+	ST2_BRANCH := master
+endif
+
+PIP_VERSION := $(shell curl --silent https://raw.githubusercontent.com/StackStorm/st2/$(ST2_BRANCH)/Makefile | grep 'PIP_VERSION ?= ' | awk '{ print $$3 }')
 PIP_OPTIONS := $(ST2_PIP_OPTIONS)
 
 ifndef PIP_OPTIONS

docs/source/index.rst

@@ -24,10 +24,6 @@ Contents:
     datastore
     chatops/index
 
-.. include:: _includes/flow.rst
-
-.. include:: _includes/solutions.rst
-
 .. toctree::
     :maxdepth: 2
     :caption: Advanced Topics

docs/source/inquiries.rst

@@ -57,7 +57,9 @@ The ``inquirer`` runner imposes a number of parameters that are, in turn, requir
 +-------------+---------------------------------------------------------+
 | roles       | A list of RBAC roles that are permitted to respond to   |
 |             | the action. Defaults to empty list, which permits all   |
-|             | roles. **This requires enterprise features**            |
+|             | roles. **This requires enterprise features on           |
+|             | StackStorm 3.2 and before**, but is available on        |
+|             | StackStorm 3.4 and later if :doc:`rbac` is enabled.     |
 +-------------+---------------------------------------------------------+
 | users       | A list of users that are permitted to respond to        |
 |             | the action. Defaults to empty list, which permits all   |

docs/source/install/config/config.rst

@@ -32,8 +32,8 @@ The ``username`` and ``password`` properties are optional.
 .. _ref-mongo-ha-config:
 
 |st2| also supports `MongoDB replica sets
-<https://docs.mongodb.com/v3.4/core/replication-introduction/>`_ using `MongoDB URI string
-<https://docs.mongodb.com/v3.4/reference/connection-string/>`_.
+<https://docs.mongodb.com/manual/replication//>`_ using `MongoDB URI string
+<https://docs.mongodb.com/manual/reference/connection-string/>`_.
 
 In :github_st2:`/etc/st2/st2.conf <conf/st2.prod.conf>` include the following section:
 
@@ -43,10 +43,10 @@ In :github_st2:`/etc/st2/st2.conf <conf/st2.prod.conf>` include the following se
     host = mongodb://<#MDB_NODE_1>,<#MDB_NODE_2>,<#MDB_NODE_3>/?replicaSet=<#MDB_REPLICA_SET_NAME>
 
 * You can also add ports, usernames and passwords, etc to your connection string. See
-  https://docs.mongodb.com/v3.4/reference/connection-string/
+  https://docs.mongodb.com/manual/reference/connection-string/
 
 * To understand more about setting up a MongoDB replica set, see
-  https://docs.mongodb.com/v3.4/tutorial/deploy-replica-set/
+  https://docs.mongodb.com/manual/replication/
 
 |st2| also supports SSL/TLS to encrypt MongoDB connections. A few extra properties need be added to
 the configuration apart from the ones outlined above.
@@ -84,6 +84,43 @@ In :github_st2:`/etc/st2/st2.conf <conf/st2.prod.conf>` include the following se
   * Build MongoDB from source to enable SSL/TLS support. See
     https://github.com/mongodb/mongo/wiki/Build-Mongodb-From-Source for more information.
 
+|st2| also supports transport / network level compression since |st2| v3.5.0. Example below shows
+which configuration options need to be set to enable compression:
+
+In :github_st2:`/etc/st2/st2.conf <conf/st2.prod.conf>` include the following section:
+
+.. code-block:: ini
+
+    [database]
+    ...
+    compressors = <zstd|zlib>
+    zlib_compression_level = 6
+
+* ``compressors`` - A comma delimited list of compression algorithm client supports for network /
+  transport level compression which should be advertised to the server. Actual algorithm used will
+  then be determined based  algorithm which is supported both by the client and the server.
+* ``zlib_compression_level`` - Compression level to use (possible values are from ``-1`` to ``9``)
+  when ``compressors`` is set to ``zlib``.
+
+.. note::
+
+    Zstandard (zstd) compression is only supported by MongoDB server version 4.0 and above.
+
+    Compression is always a trade off - CPU cycles for compression and decompression operation are
+    exchanged for less bytes transfered over the network.
+
+    How compression affects actual database operation durations and throughput is very much
+    workload and deployment specific (spare CPU cycles, network setup, available bandwidth, is the
+    NIC being saturated, is MongoDB hosted on the same server / AZ, etc).
+
+    Our micro benchmarks showed that enabling zstandard compression seems to have no perceived
+    impact on database read and write operation duration, but your milleage may vary.
+
+    Compression may come especially handy when working with large Trigger Instance and Execution
+    objects (aka executions which produce large textual results). Executions and Trigger Instances
+    usually contain large textual result which compresses very well and can result in substantial
+    reduction in network traffic.
+
 Configure RabbitMQ
 ------------------
 
@@ -118,6 +155,13 @@ or
    [messaging]
    url = amqp://guest:[email protected]:5671/?ssl=true
 
+.. note::
+
+    Keep in mind that if you want to use custom ssl options (e.g. use a key and cert file or use a
+    custom ca cert), you need to enable ssl via messaging.ssl option and not via connection URL
+    string. When SSL is enabled via connection URL string default SSL options will be used which
+    can't be changed.
+
 In addition to encrypted connection to RabbitMQ, some other SSL related options which are
 documented below are also supported:
 
@@ -144,12 +188,38 @@ documented below are also supported:
 * ``login_method`` - Login method to use. Default is ``PLAIN``. Other possible
   options are ``AMQPLAIN`` and ``EXTERNAL``.
 
+.. note::
+
+    If you want to use custom SSL settings (e.g. using a different ca bundle or similar) you
+    should specify all those options as part of the st2.conf and also do the same for enabling ssl
+    using ``messaging.ssl`` option. Combining URL and config parameters for SSL doesn't work - if
+    you enable ssl as part of the  URL it will use default SSL settings, but you won't be able to
+    specify a custom value for ``cert_reqs`` ``ca_certs`` and other SSL related options.
+
 .. note::
 
    RabbitMQ doesn't expose an SSL / TLS listener by default and needs to be configured to enable
    TLS support. For more information, refer to the official documentation -
    `Enabling TLS Support in RabbitMQ <https://www.rabbitmq.com/ssl.html#enabling-tls>`_.
 
+|st2| also supports compressing payloads which are sent over the message bus since |st2| v3.5.0.
+Example below shows which configuration options need to be set to enable compression:
+
+.. code-block:: ini
+
+    [messaging]
+    ...
+    compression = <zstd|lzma|bz2|gzip>
+
+.. note::
+
+    Compression is always a trade off - CPU cycles for compression and decompression operation are
+    exchanged for less bytes transfered over the network.
+
+    How compression affects actual message bus operation durations and throughput is very much
+    workload and deployment specific (spare CPU cycles, network setup, available bandwidth, is the
+    NIC being saturated, is RabbitMQ hosted on the same server / AZ, etc).
+
 .. _ref-rabbitmq-cluster-config:
 
 |st2| also supports `RabbitMQ cluster <https://www.rabbitmq.com/clustering.html>`_.
@@ -167,7 +237,6 @@ In :github_st2:`/etc/st2/st2.conf <conf/st2.prod.conf>` include the following se
   https://www.rabbitmq.com/clustering.html
 * RabbitMQ HA guide - https://www.rabbitmq.com/ha.html
 
-
 .. _config-configure-ssh:
 
 Configure SSH

docs/source/install/k8s_ha.rst

@@ -69,7 +69,8 @@ You can configure:
 - number of replicas for each component
 - st2 auth secrets
 - st2.conf settings
-- RBAC roles, assignments and mappings (enterprise only)
+- RBAC roles, assignments and mappings (enterprise only for StackStorm v3.2 and before, open source
+  for StackStorm v3.4 and later)
 - custom st2 packs and its configs
 - SSH private key
 - K8s resources and settings to control pod/deployment placement
@@ -317,7 +318,3 @@ and ecourage discussions in `Slack <https://stackstorm.com/community-signup>`_ `
 .. only:: community
 
     .. include:: /__engage_community.rst
-
-.. only:: enterprise
-
-    .. include:: /__engage_enterprise.rst

docs/source/install/overview.rst

@@ -72,12 +72,14 @@ installed independently.
 --------------------------------------
 * **nginx** provides SSL termination, redirects HTTP to HTTPS, serves WebUI static components, and
   reverse-proxies REST API endpoints to st2* web services.
-* **StackStorm WebUI** (st2web, and Workflow Designer, for Extreme Workflow Composer) are
-  installed at ``/opt/stackstorm/static/webui`` and configured via ``webui/config.js``. ``st2web``
-  comes in its own ``deb`` and ``rpm`` package. In StackStorm versions earlier than 3.3 Workflow Designer was deployed as part of the
-  ``bwc-enterprise`` package. They are HTML5 applications, served as static HTML, and call |st2|
-  st2auth and st2api REST API endpoints. NGINX proxies inbound requests to ``/api`` and ``/auth``
-  to the st2api and st2auth services respectively.
+* **StackStorm WebUI** (st2web, including Workflow Designer) are installed at
+  ``/opt/stackstorm/static/webui`` and configured via ``webui/config.js``. ``st2web``
+  comes in its own ``deb`` and ``rpm`` package. In StackStorm versions earlier than 3.3 Workflow
+  Designer was deployed as part of the ``bwc-enterprise`` package. They are HTML5 applications,
+  served as static HTML, and call |st2| st2auth and st2api REST API endpoints. NGINX proxies
+  inbound requests to ``/api`` and ``/auth`` to the st2api and st2auth services respectively.
+  With StackStorm version 3.4 and later, the workflow designer is entirely integrated into st2web,
+  and the ``bwc-enterprise`` package is no longer distributed.
 
 4. st2chatops - ChatOps components
 ----------------------------------
@@ -96,7 +98,7 @@ The required dependencies are RabbitMQ, and MongoDB. The optional dependencies a
 
   - nginx for SSL termination, reverse-proxying API endpoints and serving static HTML.
   - Redis or Zookeeper for concurrency policies (see :doc:`/reference/policies`).
-  - LDAP for |ewc| LDAP authentication.
+  - LDAP authentication.
 
 
 Multi-box/HA deployment

docs/source/install/rhel8.rst

@@ -75,8 +75,10 @@ Install MongoDB, RabbitMQ:
 
   sudo yum -y install crudini
   sudo yum -y install mongodb-org 
+  curl -sL https://packagecloud.io/install/repositories/rabbitmq/erlang/script.rpm.sh | sudo bash
   curl -sL https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.rpm.sh | sudo bash
   sudo yum makecache -y --disablerepo='*' --enablerepo='rabbitmq_rabbitmq-server'
+  sudo yum -y install erlang
   sudo yum -y install rabbitmq-server
   sudo systemctl start mongod rabbitmq-server
   sudo systemctl enable mongod rabbitmq-server

docs/source/install/uninstall.rst

@@ -148,6 +148,7 @@ below. Only execute the instructions for your distribution.
 
     sudo rm -f /etc/yum.repos.d/mongodb-org* /etc/yum.repos.d/StackStorm*
     sudo rm -f /etc/yum.repos.d/pgdg-94* /etc/yum.repos.d/nginx* /etc/yum.repos.d/nodesource*
+    sudo rm -f /etc/yum.repos.d/rabbitmq_erlang* /etc/yum.repos.d/*rabbitmq-server*
 
 
 5. Clean Up Remaining Content

docs/source/overview.rst

@@ -88,7 +88,3 @@ What's Next?
 .. only:: community
 
     .. include:: __engage_community.rst
-
-.. only:: enterprise
-
-    .. include:: __engage_enterprise.rst

docs/source/start.rst

@@ -54,6 +54,12 @@ You will almost certainly have authentication enabled. The easiest way to login
 
 This will obtain an authentication token, and cache it.
 
+The following will display the authentication token. 
+
+.. code-block:: bash
+
+    st2 auth st2admin -p 'Ch@ngeMe'
+
 There are other options for authentication: check the :doc:`docs<authentication>` for more details.
 
 Work with Actions
@@ -198,11 +204,13 @@ Deploy a Rule
     # Get the rule that was just created
     st2 rule get examples.sample_rule_with_webhook
 
-Once the rule is created, the webhook begins to listen on ``https://{host}/api/v1/webhooks/{url}``.
-Fire the POST, check out the file and see that it appends the payload if the name=Joe.
+Once the rule is created, the webhook begins to listen on ``https://{host}/api/v1/webhooks/{url}``. Fire the POST, check out ``/home/stanley/st2.webhook_sample.out``, and see that it appends the payload to the file.
 
 .. code-block:: bash
 
+    # Obtain authentication token
+    st2 auth st2admin -p 'Ch@ngeMe'
+    
     # Post to the webhook
     curl -k https://localhost/api/v1/webhooks/sample -d '{"foo": "bar", "name": "st2"}' -H 'Content-Type: application/json' -H 'X-Auth-Token: put_token_here'