[Meta] Platform Support

[jcgraybill]

Is your feature request related to a problem? Please describe

There's ongoing work to use OpenSearch and OpenSearch dashboards in environments beyond Linux/x64 and Linux/ARM64, c.f. #33, #38, #101. Most of OpenSearch and its plugins run in the JVM, and most of OpenSearch Dashboards and its plugins run in Node.JS, so packaging the build artifacts from Linux alongside runtimes for Windows, MacOS, FreeBSD, etc will make most OpenSearch features available to those platforms. WORA!

Making OpenSearch and OpenSearch Dashboards match the Linux experience on these platforms would require some additional work, including:

1. Being incorporated into the Build/CI framework, so that the same integration tests, performance tests, and backwards-compatibility tests are run in these environments.
2. Incorporating native or operating-system-specific components like the native K-NN library, the performance monitor plugin, and the OpenSearch Dashboards Reporting plugin's chromium dependency.
3. Porting/testing all of the shell scripts that ship with OpenSearch as necessary.
4. Creating documentation for administrating this software in the new enviroments, if that differs.
5. Making the process of installing and running OpenSearch idiomatic for that operating system. On Linux, we offer a .tgz and (soon) RPM/DEB packages, but these aren't typically the way FreeBSD, Windows, and Macintosh users install or run software.
6. Confirming that all dependencies are the same between platforms. Node.JS 10, for example, isn't available for all operating systems, so running OpenSearch Dashboards might require using a different version of Node.JS than we currently build and test against. This could lead to unexpected behavior in OpenSearch Dashboards.

The lead time on some of these tasks could be relatively long. Some of them will matter very much to some users, and less to other users. One could imagine a user, for example, who wouldn't want to use OpenSearch with production data on Windows until it's demonstrated to pass its full test suite when running in Windows. Another user might appreciate the convenience of being able to use OpenSearch for development with test data on their MacOS laptop right away. Other users may want to use OpenSearch on their preferred operating system while some of these tasks are still outstanding in order to give feedback on what works and doesn't.

Describe the solution you'd like

Let's use this issue to discuss and collect feedback on the questions:

1. Should we produce and make downloadable OpenSearch and OpenSearch Dashboards artifacts for Windows and MacOS (and in the future, other operating systems or architectures) early, while some number of these gaps exist between them and the Linux artifacts?
2. Are there any of the gaps listed above (or others I didn't list) that we would block on?
3. If we do produce OS-specific artifacts that have gaps from the Linux releases, how do we describe this to users? Do we list them separately on the downloads page? Do we label them ("alpha", "beta", "port", "experimental")? Do we caveat or make recommendations about whether they should be used for production workloads?

• Support FreeBSD #101
• [Meta] OpenSearch / OpenSearch Dashboards - Windows distribution (X64) #33
• OpenSearch / OpenSearch Dashboards - macOS distribution (X64) #38
• [Meta] OpenSearch / OpenSearch Dashboards - Support for RPM distribution #27

[stockholmux]

I have thoughts on If we do produce OS-specific artifacts that have gaps from the Linux releases.

• Labeling of these should be done carefully. A good example: the volume of questions about Open Distro on Windows being for a 'non-production environment' was pretty frequent, confusing to users, and the answers were non-specific and unsatisfying.
• Labeling something as "alpha" or "beta" already has meaning and it could confuse folks if we use it multiple ways.
• There is likely not a perfect term here, so I don't think the average user will instantly understand any term we use.

I don't want to over index on labeling the artifacts this way - the more we add to this the less usable our downloads page will be and we're likely to prevent rather than enable folks trying OpenSearch.

Here is what I would suggest:

• Anything that passes tests AND has a verifiable build supply chain (no Bob's Basement Build allowed), can be on the downloads page.
• The Linux artifacts should be considered to be the bar. If any artifact doesn't meet that bar (it's missing something, something doesn't work, etc.) is placed in a list off the downloads page: let's call it the "artifact caveats page." This deficit can be reported by the folks producing the build or by the community. If an artifact does not meet the bar, a little link is placed near the download button linking to the caveat entry.

This way we're not lumping in an artifact that doesn't have, say, something trivial like colour logging vs something that misses features. This could even be the project's own artifacts - so a Mac Arm build could have a caveat that something doesn't work as an example.

[schmitch]

btw. we are currently using elasticsearch and wanted to make ths witch to opensearch. however not having at least a stable windows distribution means that we can't migrate that easily since we also have lots of windows customers.

we would not mind at all if the windows release does not have the same featureset, as long as it is clearly stated what is different and of course nothing should be broken.

[tsiege]

Homebrew support for local development would be amazing. I'm currently working off of docker. Improving local development is crucial for me. I used to be able to dev against an Elasticsearch run in brew, and it was a great experience that never gave me any issues. Having that for Opensearch would be a game changer for me.

Getting the docker instance running locally where it can consistently start, stop, and easily connect to for local development is really crucial to me. Docker is hard to debug, and makes it challenging for me and for on boarding new devs.

[dblock]

@tsiege There's actually a community contributed Homebrew formulae already, check out #38 (comment) and https://github.com/Homebrew/homebrew-core/blob/master/Formula/opensearch.rb. I have some concerns given that Homebrew builds from source and this doesn't include plugins, but it's a thing.

[stockholmux]

@schmitch When you say you have 'lots of windows customers' - are they currently running Elasticsearch production instances on Windows?

[gruselglatz]

Maybe i've overlooked something but there is already a deb buildstep [1]. But why is there no download for the deb packages?

Sry if i missed something :)
[1] https://github.com/opensearch-project/OpenSearch/blob/b78176afefd8c09a19de6bc52907b40c8c4fedf2/distribution/packages/build.gradle#L353

[AMoo-Miki]

The FreeBSD port of OpenSearch Dashboards relaxes its NodeJS requirements. Additionally, OSD's helpers have a few guardrails against platforms other than linux, windows, and darwin.