Logstash 6.x doc changes

[dedemorton]

6.3 Release

• Monitoring UI (https://github.com/elastic/x-pack-kibana/pull/4711)
  • New screenshot for PQ disk stats
  • New screenshot for pipeline viewer - time series viz in detail drawer
• Pipeline-to-Pipeline Communication (Doc for pipeline-to-pipeline communication #9385)
• Centralized pipeline management - new pipeline settings (https://github.com/elastic/x-pack-kibana/pull/4620, https://github.com/elastic/x-pack-logstash/issues/327). (WIP - PR Add settings to centralized pipeline mgmt doc #9649)
• Centralized Management errors out if ES security is disabled. Note: Security disabled by default with Trial license in 6.2 (https://github.com/elastic/private-logstash/pull/34)
• ArcSight module documentation updates ([opening x-pack] ArcSight module documentation updates #9472)

Between 6.3 and 6.4

• Azure plugins and monitoring module (WIP Add azure module doc #9510)
• Cherrypick PQ upgrade info to 6.x. (Originally only merged into 6.2 and 6.3) - Content in Add instructions for upgrading PQ pre-6.3.0 #9580. Notes in [DOC] Adding release notes for 6.3.0 #9644

6.4 Release

• SNMP Poller input plugin
• Centralized management v2 updates - https://github.com/elastic/x-pack-logstash/issues/327, https://github.com/elastic/x-pack-kibana/issues/4334
• Update setup of monitoring and centralized management UIs with impending changes

Other

• Document how to use TLS/SSL settings with LS modules (Document how to use TLS/SSL settings with modules #9290 )
• Modules generator and contributor guide
• Document permissions/roles required for running X-Pack features (Document permissions and/or role required for user to run X-Pack features #9146)
• Document running Logstash on Windows - Document running Logstash as a Windows service using nssm #4005
• Improve the getting started guide; perhaps have a quickstart (e.g. modules) and general getting started
  • Include info about modules so users know there are easier ways to get started.
  • Mention multiple pipelines and centralized pipeline management so users know that the example configs shown in the getting started guide are simplified and that other options exist for specifying configs.
  • Decide whether to add more input examples (jdbc, http, ES + SJ, etc)
• (Dev) Create doc for setting java heap size ([Docs] Add a separate topic on setting heap size #7727, Dead references to LS_HEAP_SIZE #6970), and reference that page from the performance troubleshooting section.
  • Dead references to LS_HEAP_SIZE (Dead references to LS_HEAP_SIZE #6970) Fix [Docs] Remove LS_HEAP_SIZE and reference JVM.options instead #9335
• Add pipelines.yml to Config Settings Files page and other places (Should this page mention pipelines.yml? #8564)
  • Update config files list (add pipelines.yml and jvm.log4j2.properties; update jvm.options) Add file names to list of LS config files #9321
  • Running LS from command line
  • Advanced pipeline topic (expand info)
• Document plugin installation in deb/rpm environs: document plugin installation in rpm/deb environments #8040
• Clarify info in plugins-filters-dissect.html
• Add content from blogs to Logstash docs (Add content from blogs to Logstash docs #6283)
• Decide on best approach for referencing older release versions in docs. Clean up accordingly.
• Performance tuning info from blog to doc (https://github.com/elastic/blog/issues/201)

Backlog

• Fix quality issues in plugin docs, as time allows (see Quality issues in the plugin docs (incorrect asciidoc formatting, missing content) #6693).
• Document that config directory should only contain config files: Logstash parse .rb files indie pipeline dir as configuration files #9613
• Make sure monitoring APIs are up-to-date wrt examples of what they return.
  • New fields for Elasticsearch output plugin level stats
• Add section "Adding & Collecting New Plugin Metrics" - adding guidance on how to add metrics to collect in plugins along with new metrics style guide Style guide practices when adding metrics to logstash plugins #5716
• Pull topics about contributing to Logstash into a separate book (developer guide). Include info about generating a module and the Event API. The Event API docs should probably appear in both the LS Ref and the Dev Guide. Quick fix: move topic about generating modules to the contributing to logstash section.
• Clean up cross references to match link style used in other books in the library. See Cleanup cross references by removing trailing slashes #7571
• Make plugin descriptions consistent across plugin docs. See Make plugin descriptions consistent across the plugin docs #6086
• Update plugin docs to use the new Event API (looks like we missed some examples in the plugin docs). For example Update docs for elasticsearch filter plugin #6459
• Plugin docs:
  • "master" version of docs needs to be fixed to show latest version of all plugins. The other versions will continue to show the plugin version which was bundled with the release.
• New windows doc page - just under the install scope

Completed

6.2

• Add docs for JDBC_static plugin when content is available. Also update "Transforming Data" section to mention JDBC_static plugin
• Document secret store
• Publish versioned plugin reference
• Enhance docs about pipeline viewer to show new overview page (backport to 6.2).
• Edit JDBC_static docs.
• [DeDe] Fix defaults shown for default codec in redis input and output plugin docs. This will require some conditional coding. See Documentation on default codec is wrong. It lists "plain" but is using "json" logstash-plugins/logstash-output-redis#47 and Documentation on default codec is wrong. It lists "plain" but is using "json". logstash-plugins/logstash-input-redis#57

6.1 GA

• Make sure usage of Cloud ID is clearly documented
• Clarify docs about tuning options. See related issue: Clarify tuning options for PQ in documentation #7027

[dedemorton]

Closing because this is an old tracking document.