Add "humio" as a supported distribution

[jmountifield]

The OpenSearch LogStash plugin is currently unable to send data to Humio's bulk endpoint because "humio" is not recognised as a valid distribution, and (currently) Humio advertises its bulk API as version 6.2.0.

Having tested the current Humio versions with the plugin we believe it is compatible, we would like to add "humio" as a valid distribution here

We will look to increment the Humio advertised major version for this API to >= 7 but this is not something that can be done quickly as significant compatibility testing is required. We would like to see our uses be able to make use of the OpenSearch plugin asap.

[VijayanB]

@jmountifield Thanks for your interest in contributing to logstash-output-openserach plugin. Does "humio" uses OpenSearch behind the service?

[jmountifield]

Hi @VijayanB, thanks for the reply.

Does "humio" uses OpenSearch behind the service?

No, Humio itself does not make use of OpenSearch or ElasticSearch technology in our product.

Humio provides log/data management capabilities, and to ensure that as many users as possible can utilise the platform we have pursued an approach of adopting open APIs for ingest in addition to the native APIs. This includes the elasticsearch bulk api, and Splunk HEC.

Since the recent changes by Elastic we are no longer compatible with native LogStash and we would like to enourage our customer base currently using logstash to adopt the OpenSearch plugin.

We do not request any promotion of Humio as a target platform (i.e. I don't think we want/need any reference to Humio in the OpenSearch documentation) and at a future point I expect we can remove the specific reference to Humio as we upgrade to version 7+ of the bulk API.

[stockholmux]

I'm curious about checking distribution. @VijayanB What's the need to check for OpenSearch in distribution?

Would a warning be sufficient if it's not OpenSearch?

[jmountifield]

Hi team, I've closed my original PR in favour of what is proposed in #65 and I'd love to see that change included, we have some existing LogStash users that are currently blocked from upgrading their LogStash environments.

I had a look through CONTRIBUTING and RELEASING and I was wondering what sort of expectations the community can have around this change being included in a 1.1.0 release in the near future?

[stockholmux]

@jmountifield I can't answer that directly (curious what @VijayanB has to say), but your message does make me want to clear something up - this repo is independently versioned from OpenSearch itself. So while OpenSearch 1.1.0 will be out in a couple of weeks, this plugin may not be on the same timeline (could be sooner or later!).

[VijayanB]

@jmountifield Like @stockholmux mentioned, there is no relation between this plugin version and OpenSearch version. We can release this plugin independently. I will update here on expected release timeline soon.

[jmountifield]

Thanks for the feedback @stockholmux and @VijayanB. I didn't have an expectation that the release cycles were linked, but its good to get that called out specifically.

I'm really asking because I am trying to determine if we need to try and publish a temporary build of the plugin for our users to make use of in the interim. This isn't intended to induce any feelings of obligation on your part, this project is awesome and I think many are very grateful that you're taking it on (even if they don't realise it yet).

[VijayanB]

@jmountifield We are planning to release by Sep 21st. Will that be helpful?

[VijayanB]

@jmountifield We released 1.1.0 version. Can you install it from rubygems.org and verify whether you were able to connect to humio? Thanks.

[jmountifield]

@VijayanB yes, tested and working great!

For reference, I tested with this configuration:

output{
  opensearch{
    hosts => ["https://<HUMIO_HOST>:443/api/v1/ingest/elastic-bulk"]
    user => "sandbox"
    password => "<INGEST_TOKEN>"
    ssl => true
    ssl_certificate_verification => true
    manage_template => false
    http_compression => true
  }
}