Beats doc updates in 6.x

[dedemorton]

THIS DOCUMENT IS OBSOLETE. Please contact me for a pointer to the new location of the doc roadmap. All unresolved items from this list have either been moved to the roadmap, added to the backlog, or closed because they were deemed no longer necessary.

Resolved

• Resolve Provide more detailed documentation on log levels #861)
• [DeDe] Make sure GS guides accurately reflect requirement for using sudo (Most of the Beats require to sudo commands. Exception is Filebeat and Heartbeat (only if icmp is not configured)). Look at the soln docs as a source of truth for recommendations.
• Random edits:
• Check FAQ meta issue and determine if any items from the issue should be added to the docs for the next release: Add FAQ topics to the Beats doc #3599 - [x] Config file format (backlog)
  • Filebeat Modules tutorial - possibly rewrite as task-based topics
  • Jolokia module docs
• Document include_fields filter usage #3340 (problem here might be that it's not obvious that processors can be applied to modules because we point off to another topic that describes the shared options. Instead of doing this, maybe we should put the config options in a shared file that gets pulled into the docs for all modules. https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-metricbeat.html
• Document authentication/SSL support for http, jolokia and other metricbeats #4268 (carlos said he might take a crack at this one)
• Beats users permissions for the default modules using ingest pipelines #4826 (need to document permissions for things like running ingest pipelines, setting up dashboards, index patterns, etc).
• Clean up CLI docs so that the user sees one list of commands rather than a list separated into two sections (for beats that add their own options). This will involve some conditional coding to implement.
• REMOVED Organize FAQ topics (either alphabetically or logically). See Minor: organize FAQ topics logically #2258 - I'm removing this from my list in favor of more important work because it's low-value. Maybe someday.

6.4

• Add missing config info (paths) to the module docs for the Kibana and Elasticsearch modules: Docs for Kibana and elasticsearch modules are missing config details #7835
• [DeDe] Update dashboards. Need to show the latest version of the dashboards.
• Follow up: document http endpoint. See Document HTTP Endpoint #3933. (backport to 6.2, if possible)
• Follow up: document headers and ssl for metricsets. See Document headers and ssl for metricsets #3948 (backport to 6.2, if possible)
• [DeDe] Filebeat system module startup instructions (Brandon) Filebeat system module startup instructions, dashboard name, and time range #4567 (backport to 6.2, if possible)
• Document backoff setting. See Document backoff setting #5419. Fixed in Document backoff options #6903
• Old defect triage. Resolved 20+ defects.

6.3

• Make sure these get reviewed:
  • Spooling to disk: Add file spool to queue docs #6902
  • Update default value for flush.timeout and flush.min_events #7192
  • Backoff settings: Document backoff options #6903
• [DeDe] Refactor prospector/intput docs. Add a list of prospectors/inputs in the Filebeat docs. Currently, we have a list of the type, but it would be nice to have more visibility (maybe add a separate section about from where to get the logs, via udp, redis slow logs). Also see [WIP] Rename prospector to input #5944 for work related to renaming prospectors to inputs. The UDP input, in particular, needs more detail about configuration (see UDP prospector type with plain harvester #4452) Also see Meta: Refactor prospector to input #6122 and Update docs to reflect prospector > input refactoring #6375 and Feature: TCP Input #6266
• Change title of topic about ingest node to say "Parse data" instead of "Parse logs" because "logs" does not make sense in all contexts where ingest node is used with beats.
• Update Metricbeat getting started to show how to enable/disable by using the modules.d directory. See Update getting started docs to show modules.d for enabling modules #5325
• [DeDe] Need Beats users to know about the additional commands for importing dashboards that are covered here: https://www.elastic.co/guide/en/beats/devguide/current/import-dashboards.html#import-dashboards. Right now the info is only obvious to developers...users need to know these options, too. Make it more clear in the Getting Started guides that these options exist. (backport to 6.2, if possible)
• Add pointer to SSL settings in the Heartbeat monitor docs. See Heartbeat monitor SSL settings docs #5599
• Document beat user permission required to use ingest pipelines (backport to 6.0+, if possible). See Beats users permissions for the default modules using ingest pipelines #4826
• REMOVED Improve retrievability of reference info about config options. It's still too hard to find stuff. (We are improving the search, so I'm closing this one)

6.0+

Marking these items as 6.0 so we remember to backport them.

• Clarify docs about config reloading. See Clarify documentation around reloading configs #5676.
• [Tudor] Filebeat modules: Machine learning jobs. See Filebeat modules: Machine learning jobs #4506. setup command is documented, but we probably need to add info about required privileges plus maybe have some conceptual info.
  language indicating the version of kibana that's required to see the dashboards.
• Follow up. Make sure that all PRs labeled as needs_docs have documentation, specifically:
  • Use packetbeat protocols to publish beat.Event to new pipeline Use packetbeat protocols to publish beat.Event to new pipeline #4599 (backport in Cherry-pick #4753 to 6.0: Remove settings from docs #4878 )
  • [urso] Convert heartbeat to new publisher pipeline Convert heartbeat to new publisher pipeline #4591 (backport in Cherry-pick #4753 to 6.0: Remove settings from docs #4878 )

6.1+ (list is a work-in-progress...created as a place holder)

• Convert format of the Filebeat modules doc for Kafka, PostgresSQL, and Traefik (any modules added after 6.0) to match the other module docs.
• Conditionally code pre-release content. See Conditionally code apt/yum install instuctions for pre-releases #5652

6.2

• Make sure the docs get updated with info about the password keystore. See Allow user to locally obfuscate secret in a keystore #5687
• Document autodiscover for kubernetes: Add autodiscover for kubernetes #6055. See Add kubernetes autodiscover docs #6065

[dedemorton]

I'm closing this meta issue because I will no longer be using it to track the doc roadmap starting with Beats 6.5 and later.