Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

[logstash] add security example #392

Merged
merged 6 commits into from
Jun 18, 2020
Merged

Conversation

jmlrt
Copy link
Member

@jmlrt jmlrt commented Dec 2, 2019

This PR add a security example to use Logstash helm chart and Elasticsearch helm chart with TLS encryption.

Fix #587

  • Chart version not bumped (the versions are all bumped and released at the same time)
  • README.md updated with any new values or changes
  • Updated template tests in ${CHART}/tests/*.py
  • Updated integration tests in ${CHART}/examples/*/test/goss.yaml

@rhizoet
Copy link

rhizoet commented Apr 30, 2020

Any news to this?

I've used the example for my logstash setup. But I get an Error and because of this logstash become not ready:

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.9.0.jar) to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
[2020-04-30T10:19:27,135][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.6.2"}
[2020-04-30T10:19:29,214][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"signed fields invalid"}
[2020-04-30T10:19:29,299][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[2020-04-30T10:19:31,000][INFO ][org.reflections.Reflections] Reflections took 82 ms to scan 1 urls, producing 20 keys and 40 values 
[2020-04-30T10:19:31,924][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][main] A gauge metric of an unknown type (org.jruby.RubyArray) has been created for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.
[2020-04-30T10:19:31,928][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x793149aa run>"}
[2020-04-30T10:19:33,529][INFO ][logstash.inputs.beats    ][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-04-30T10:19:33,603][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-04-30T10:19:33,797][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-04-30T10:19:33,807][INFO ][org.logstash.beats.Server][main] Starting server on port: 5044
[2020-04-30T10:19:34,317][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-04-30T10:19:59,298][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"signed fields invalid"}

I've created the certs and credentials as stated in the Makefile from Elasticsearch. So nothing else done. Config is the same as in this PR.

Any idea what went wrong?

@jmlrt jmlrt marked this pull request as ready for review June 18, 2020 08:28
@jmlrt jmlrt requested a review from a team June 18, 2020 08:28
fatmcgav
fatmcgav previously approved these changes Jun 18, 2020
Copy link
Contributor

@fatmcgav fatmcgav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@fatmcgav fatmcgav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jmlrt jmlrt merged commit 7f101f2 into elastic:master Jun 18, 2020
@jmlrt jmlrt deleted the logstash-security branch June 18, 2020 12:33
@jmlrt
Copy link
Member Author

jmlrt commented Jun 18, 2020

@rhizoet The PR is merged, FYI logstash-output-elasticsearch doesn't seem to accept .pem certificate. I updated the Elasticsearch Makefile to use .crt certificate instead.

jmlrt added a commit that referenced this pull request Jun 18, 2020
This commit add a security example to use Logstash Helm chart and Elasticsearch Helm chart with TLS encryption.
jmlrt added a commit that referenced this pull request Jun 18, 2020
This commit add a security example to use Logstash Helm chart and Elasticsearch Helm chart with TLS encryption.
@jmlrt
Copy link
Member Author

jmlrt commented Jun 18, 2020

backported to 6.8, 7.8and 7.x branches

jmlrt added a commit that referenced this pull request Jun 18, 2020
This commit add a security example to use Logstash Helm chart and Elasticsearch Helm chart with TLS encryption.
@jmlrt jmlrt mentioned this pull request Jun 18, 2020
@jmlrt jmlrt mentioned this pull request Oct 28, 2020
This was referenced Nov 17, 2020
This was referenced Nov 19, 2020
@jmlrt jmlrt mentioned this pull request Feb 8, 2021
This was referenced Mar 15, 2021
@jmlrt jmlrt mentioned this pull request May 25, 2021
@jmlrt jmlrt mentioned this pull request Mar 8, 2022
@jmlrt jmlrt mentioned this pull request Apr 21, 2022
This was referenced Sep 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[logstash] mounting PEM certificate to connect to Elasticsearch via TLS isn't working
3 participants