Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support the RPM distribution builds #1117

Closed
4 tasks done
Tracked by #27
gaiksaya opened this issue Nov 22, 2021 · 6 comments
Closed
4 tasks done
Tracked by #27

Support the RPM distribution builds #1117

gaiksaya opened this issue Nov 22, 2021 · 6 comments
Assignees
@peterzhuamazon
Labels
enhancement New Enhancement good first issue Good for newcomers v1.3.0

Comments

@gaiksaya
Copy link
Member

gaiksaya commented Nov 22, 2021 •
edited by peterzhuamazon
Loading

  • Assemble process will have new param --distribution to specify which distribution to build
  • Uncouple distribution creation with min tarball/zip file extension
  • RPM will be built based on min tar + plugin zip installation + fpm bundle pkg
  • User should be able to use ./assemble builds/manifests.yml --distribution rpm to build rpm and get a rpm file in return for installation.
  • We design it to run on a specific Docker Image with ruby, rpm, fpm installed, not able to run directly on macOS directly. We will make changes to the docker files as well
  • Part of the pre/post scripts and sysvinit/systemd will be taken from ODFE.
  • Also need these scripts specifically for pkg from PA: https://github.com/opensearch-project/performance-analyzer/tree/main/packaging

Acceptance criteria:

  • The RPM package builds for both arm64 and x64.
  • able to install the package with no errors.
  • able to start the service through systemd.
  • able to remove installed/uninstall package from host.
@gaiksaya gaiksaya mentioned this issue Nov 22, 2021
Closed
22 tasks
@zylzulu
Copy link
Contributor

zylzulu commented Nov 23, 2021 •
edited
Loading

[Update] The .rpm package is able to be built after run ./assemble.sh builds/manifest.yml --distribution rpm. Here is the log for it:

Installing dependencies in . ...
Installing dependencies from Pipfile.lock (c3492c)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
Running ./src/run_assemble.py builds/manifest.yml --distribution rpm ...
2021-11-23 19:42:39 INFO     Bundling OpenSearch (x64) on linux into /usr/share/opensearch/opensearch-rpm-test-in-docker/dist ...
2021-11-23 19:42:39 INFO     Copied min bundle to /tmp/tmpnuezu6mb/opensearch-min-1.1.0-linux-x64.tar.gz.
2021-11-23 19:42:39 INFO     Extracting dist into /tmp/tmpnuezu6mb.
2021-11-23 19:42:43 INFO     Extracted dist into /tmp/tmpnuezu6mb.
2021-11-23 19:42:43 INFO     Executing "bash /usr/share/opensearch/opensearch-rpm-test-in-docker/scripts/components/OpenSearch/install.sh -v 1.1.0 -p linux -a x64 -f /usr/share/opensearch/opensearch-rpm-test-in-docker/builds -o /tmp/tmpnuezu6mb/opensearch-1.1.0" in /tmp/tmpnuezu6mb/opensearch-1.1.0
+ getopts :h:v:s:o:p:a:f: arg
+ case $arg in
+ VERSION=1.1.0
+ getopts :h:v:s:o:p:a:f: arg
+ case $arg in
+ PLATFORM=linux
+ getopts :h:v:s:o:p:a:f: arg
+ case $arg in
+ ARCHITECTURE=x64
+ getopts :h:v:s:o:p:a:f: arg
+ case $arg in
+ ARTIFACTS=
+ getopts :h:v:s:o:p:a:f: arg
+ case $arg in
+ OUTPUT=/tmp/tmpnuezu6mb/opensearch-1.1.0
+ getopts :h:v:s:o:p:a:f: arg
+ '[' -z 1.1.0 ']'
+ '[' -z '' ']'
+ SNAPSHOT=false
+ '[' -z linux ']'
+ '[' -z x64 ']'
++ dirname /usr/share/opensearch/opensearch-rpm-test-in-docker/scripts/components/OpenSearch/install.sh
+ DIR=/usr/share/opensearch/opensearch-rpm-test-in-docker/scripts/components/OpenSearch
+ echo /usr/share/opensearch/opensearch-rpm-test-in-docker/scripts/components/OpenSearch
/usr/share/opensearch/opensearch-rpm-test-in-docker/scripts/components/OpenSearch
+ cd /usr/share/opensearch/opensearch-rpm-test-in-docker/scripts/components/OpenSearch
+ cp ../../../scripts/legacy/tar/linux/opensearch-tar-install.sh /tmp/tmpnuezu6mb/opensearch-1.1.0/
2021-11-23 19:42:43 INFO     Installed plugins: []
2021-11-23 19:42:43 INFO     build for rpm distribution.
Setting workdir {:workdir=>"/tmp", :level=>:info}
Setting from flags: architecture=x86_64 {:level=>:info}
Setting from flags: description=opensearch rpm 1.1.0 {:level=>:info}
Setting from flags: epoch= {:level=>:info}
Setting from flags: iteration= {:level=>:info}
Setting from flags: license=ASL 2.0 {:level=>:info}
Setting from flags: maintainer=OpenSearch {:level=>:info}
Setting from flags: name=opensearch {:level=>:info}
Setting from flags: url=https://opensearch.org/ {:level=>:info}
Setting from flags: vendor=OpenSearch {:level=>:info}
Setting from flags: version=1.1.0 {:level=>:info}
Converting dir to rpm {:level=>:info}
Removing excluded path {:path=>"usr/share/opensearch/config", :matches=>"usr/share/opensearch/config", :level=>:info}
Removing excluded path {:path=>"usr/share/opensearch/data", :matches=>"usr/share/opensearch/data", :level=>:info}
Reading template {:path=>"/usr/local/rvm/gems/ruby-2.3.3/gems/fpm-1.13.0/templates/rpm.erb", :level=>:info}
Running rpmbuild {:args=>["rpmbuild", "-bb", "--define", "buildroot /tmp/package-rpm-build-f2196c68ea37e8f65a71ae6af7d51f1994d5295b261fc543120cbbe925f7/BUILD", "--define", "_topdir /tmp/package-rpm-build-f2196c68ea37e8f65a71ae6af7d51f1994d5295b261fc543120cbbe925f7", "--define", "_sourcedir /tmp/package-rpm-build-f2196c68ea37e8f65a71ae6af7d51f1994d5295b261fc543120cbbe925f7", "--define", "_rpmdir /tmp/package-rpm-build-f2196c68ea37e8f65a71ae6af7d51f1994d5295b261fc543120cbbe925f7/RPMS", "--define", "_tmppath /tmp", "/tmp/package-rpm-build-f2196c68ea37e8f65a71ae6af7d51f1994d5295b261fc543120cbbe925f7/SPECS/opensearch.spec"], :level=>:info}
Executing(%prep): /bin/sh -e /tmp/rpm-tmp.RB7rG8 {:level=>:info}
Executing(%build): /bin/sh -e /tmp/rpm-tmp.9tQS9R {:level=>:info}
Executing(%install): /bin/sh -e /tmp/rpm-tmp.vrj5DB {:level=>:info}
Processing files: opensearch-1.1.0-1.x86_64 {:level=>:info}
Provides: opensearch = 1.1.0-1 opensearch(x86-64) = 1.1.0-1 {:level=>:info}
Requires(interp): /bin/sh /bin/sh /bin/sh /bin/sh {:level=>:info}
Requires(rpmlib): rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(CompressedFileNames) <= 3.0.4-1 {:level=>:info}
Requires(pre): /bin/sh {:level=>:info}
Requires(post): /bin/sh {:level=>:info}
Requires(preun): /bin/sh {:level=>:info}
Requires(postun): /bin/sh {:level=>:info}
Wrote: /tmp/package-rpm-build-f2196c68ea37e8f65a71ae6af7d51f1994d5295b261fc543120cbbe925f7/RPMS/x86_64/opensearch-1.1.0-1.x86_64.rpm {:level=>:info}
Executing(%clean): /bin/sh -e /tmp/rpm-tmp.vkDat9 {:level=>:info}
Created package {:path=>"/usr/share/opensearch/opensearch-rpm-test-in-docker/opensearch-1.1.0-linux-x64.rpm"}
2021-11-23 19:43:31 INFO     Published /usr/share/opensearch/opensearch-rpm-test-in-docker/dist/opensearch-1.1.0-linux-x64.rpm.
2021-11-23 19:43:31 INFO     Package for rpm
2021-11-23 19:43:31 INFO     Done.

@zylzulu zylzulu mentioned this issue Nov 24, 2021
Closed
1 task
@peternied peternied mentioned this issue Dec 3, 2021
Closed
@zelinh zelinh changed the title Automate the RPM distribution builds Support the RPM distribution builds Dec 6, 2021
@peterzhuamazon
Copy link
Member

We are able to install the rpm on centos7 host and start the service with success:

[centos@ip-172-31-22-63 system]$ sudo systemctl restart opensearch
[centos@ip-172-31-22-63 system]$ sudo systemctl status opensearch
● opensearch.service - OpenSearch
   Loaded: loaded (/etc/systemd/system/opensearch.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2021-12-07 00:54:42 UTC; 33s ago
     Docs: https://opensearch.org/
 Main PID: 1056 (java)
   CGroup: /system.slice/opensearch.service
           └─1056 /usr/share/opensearch/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -X...

Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,610][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] Auditing will watch {} for read requests.
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,611][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] Auditing read operation requests from kibanaserver users is disabled.
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,611][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] Auditing only metadata information for write request is enabled.
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,611][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] Auditing diffs for write requests is disabled.
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,611][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] Auditing write operation requests from kibanaserver users is disabled.
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,611][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] Auditing will watch <NONE> for write requests.
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,612][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] .opendistro_security is used as internal security index.
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,612][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] Internal index used for posting audit logs is null
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,612][INFO ][o.o.s.c.ConfigurationRepository] [ip-172-31-22-63.ec2.internal] Hot-reloading of audit configuration is enabled
Dec 07 00:54:54 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:54:54,613][INFO ][o.o.s.c.ConfigurationRepository] [ip-172-31-22-63.ec2.internal] Node 'ip-172-31-22-63.ec2.internal' initialized

[centos@ip-172-31-22-63 system]$ sudo systemctl stop opensearch
[centos@ip-172-31-22-63 system]$ sudo systemctl status opensearch
● opensearch.service - OpenSearch
   Loaded: loaded (/etc/systemd/system/opensearch.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: https://opensearch.org/

Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd[1]: Stopping OpenSearch...
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:56:04,772][INFO ][o.o.s.a.r.AuditMessageRouter] [ip-172-31-22-63.ec2.internal] Closing AuditMessageRouter
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:56:04,773][INFO ][o.o.s.a.s.SinkProvider   ] [ip-172-31-22-63.ec2.internal] Closing InternalOpenSearchSink
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:56:04,773][INFO ][o.o.s.a.s.SinkProvider   ] [ip-172-31-22-63.ec2.internal] Closing DebugSink
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:56:04,773][INFO ][o.o.n.Node               ] [ip-172-31-22-63.ec2.internal] stopping ...
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:56:04,790][INFO ][o.o.n.Node               ] [ip-172-31-22-63.ec2.internal] stopped
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:56:04,790][INFO ][o.o.n.Node               ] [ip-172-31-22-63.ec2.internal] closing ...
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:56:04,795][INFO ][o.o.s.a.i.AuditLogImpl   ] [ip-172-31-22-63.ec2.internal] Closing AuditLogImpl
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd-entrypoint[1056]: [2021-12-07T00:56:04,797][INFO ][o.o.n.Node               ] [ip-172-31-22-63.ec2.internal] closed
Dec 07 00:56:04 ip-172-31-22-63.ec2.internal systemd[1]: Stopped OpenSearch.
[centos@ip-172-31-22-63 system]$ vi opensearch.service

It still has some parts to improve but we are able to see service run without error:

[centos@ip-172-31-22-63 opensearch]$ curl https://localhost:9200 -u admin:admin --insecure
{
  "name" : "ip-172-31-22-63.ec2.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "Lls4272hRkOzJFUe7u1Omw",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.2.0",
    "build_type" : "tar",
    "build_hash" : "c459282fd67ddb17dcc545ec9bcdc805880bcbec",
    "build_date" : "2021-11-29T23:26:06.112383Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

@zylzulu
Copy link
Contributor

zylzulu commented Dec 8, 2021 •
edited
Loading

We are able to install Opensearch Dashboards on CentOS7 and start the service.

[centos@ip-172-31-22-63 ~]$ sudo systemctl start opensearch-dashboards
[centos@ip-172-31-22-63 ~]$ sudo systemctl status opensearch-dashboards
● opensearch-dashboards.service - "OpenSearch Dashboards"
   Loaded: loaded (/etc/systemd/system/opensearch-dashboards.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2021-12-08 00:42:40 UTC; 12s ago
 Main PID: 25193 (node)
   CGroup: /system.slice/opensearch-dashboards.service
           └─25193 /usr/share/opensearch-dashboards/bin/../node/bin/node /usr/share/opensearch-dashboards/bin/../src/cli/dist -c /etc/opensearch-dashboards/opensearch_dashboards.yml

Dec 08 00:42:40 ip-172-31-22-63.ec2.internal systemd[1]: Started "OpenSearch Dashboards".

[centos@ip-172-31-22-63 ~]$ sudo systemctl stop opensearch-dashboards
[centos@ip-172-31-22-63 ~]$ sudo systemctl status opensearch-dashboards
● opensearch-dashboards.service - "OpenSearch Dashboards"
   Loaded: loaded (/etc/systemd/system/opensearch-dashboards.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

Dec 07 23:28:04 ip-172-31-22-63.ec2.internal opensearch-dashboards[20508]: {"type":"log","@timestamp":"2021-12-07T23:28:04Z","tags":["error","opensearch","data"],"pid":20508,"message":"[ConnectionE....0.1:9200"}
Dec 07 23:28:07 ip-172-31-22-63.ec2.internal opensearch-dashboards[20508]: {"type":"log","@timestamp":"2021-12-07T23:28:07Z","tags":["error","opensearch","data"],"pid":20508,"message":"[ConnectionE....0.1:9200"}
Dec 07 23:28:07 ip-172-31-22-63.ec2.internal systemd[1]: Stopping "OpenSearch Dashboards"...
Dec 07 23:28:07 ip-172-31-22-63.ec2.internal opensearch-dashboards[20508]: {"type":"log","@timestamp":"2021-12-07T23:28:07Z","tags":["info","plugins-system"],"pid":20508,"message":"Stopping all plugins."}
Dec 07 23:28:07 ip-172-31-22-63.ec2.internal opensearch-dashboards[20508]: {"type":"log","@timestamp":"2021-12-07T23:28:07Z","tags":["info","savedobjects-service"],"pid":20508,"message":"Starting s...igrations"}
Dec 07 23:28:07 ip-172-31-22-63.ec2.internal opensearch-dashboards[20508]: {"type":"log","@timestamp":"2021-12-07T23:28:07Z","tags":["warning","savedobjects-service"],"pid":20508,"message":"Unable ... request."}
Dec 07 23:28:07 ip-172-31-22-63.ec2.internal systemd[1]: Stopped "OpenSearch Dashboards".
Dec 07 23:49:42 ip-172-31-22-63.ec2.internal systemd[1]: Started "OpenSearch Dashboards".
Dec 08 00:36:53 ip-172-31-22-63.ec2.internal systemd[1]: Stopping "OpenSearch Dashboards"...
Dec 08 00:36:53 ip-172-31-22-63.ec2.internal systemd[1]: Stopped "OpenSearch Dashboards".

[centos@ip-172-31-22-63 ~]$ curl localhost:5601/api/status
{
   "name": "ip-172-31-22-63.ec2.internal",
   "uuid": "502c843c-dd3d-4164-bff1-65e516c409ad",
   "version": {
      "number": "1.2.0",
      "build_hash": "caf668e73304bac890f41c37cd6c3a41257cd289",
      "build_number": 1,
      "build_snapshot": false
   },
   "status": {
      "overall": {
         "since": "2021-12-08T00:58:42.046Z",
         "state": "green",
         "title": "Green",
         "nickname": "Looking good",
         "icon": "success",
         "uiColor": "secondary"
      },
    ......
}

@gaiksaya gaiksaya added enhancement New Enhancement good first issue Good for newcomers v1.3.0 labels Dec 10, 2021
@zylzulu
Copy link
Contributor

zylzulu commented Dec 13, 2021 •
edited
Loading

Currently, we are able to install the RPM package on the AL2 host, and start, stop the opensearch service.

[ec2-user@ip-172-31-65-37 ~]$ sudo systemctl start opensearch
[ec2-user@ip-172-31-65-37 ~]$ sudo systemctl status opensearch
● opensearch.service - OpenSearch
   Loaded: loaded (/etc/systemd/system/opensearch.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-12-09 18:49:25 UTC; 3 days ago
     Docs: https://opensearch.org/
 Main PID: 9740 (java)
   CGroup: /system.slice/opensearch.service
           └─9740 /usr/share/opensearch/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -...

Dec 09 18:49:25 ip-172-31-65-37.ec2.internal systemd[1]: Started OpenSearch.
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: An illegal reflective access operation has occurred
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: Illegal reflective access by io.protostuff.runtime.PolymorphicThrowableSchema (file:/usr/share/opensearch/plugins/open...owable.cause
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: Please consider reporting this to the maintainers of io.protostuff.runtime.PolymorphicThrowableSchema
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: All illegal access operations will be denied in a future release
Hint: Some lines were ellipsized, use -l to show in full.

[ec2-user@ip-172-31-65-37 ~]$ sudo systemctl stop opensearch
[ec2-user@ip-172-31-65-37 ~]$ sudo systemctl status opensearch
● opensearch.service - OpenSearch
   Loaded: loaded (/etc/systemd/system/opensearch.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: https://opensearch.org/

Dec 09 18:46:13 ip-172-31-65-37.ec2.internal systemd[1]: Stopping OpenSearch...
Dec 09 18:46:13 ip-172-31-65-37.ec2.internal systemd[1]: Stopped OpenSearch.
Dec 09 18:49:25 ip-172-31-65-37.ec2.internal systemd[1]: Started OpenSearch.
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: An illegal reflective access operation has occurred
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: Illegal reflective access by io.protostuff.runtime.PolymorphicThrowableSchema (file:/usr/share/opensearch/plugins/open...owable.cause
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: Please consider reporting this to the maintainers of io.protostuff.runtime.PolymorphicThrowableSchema
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Dec 09 18:49:34 ip-172-31-65-37.ec2.internal systemd-entrypoint[9740]: WARNING: All illegal access operations will be denied in a future release
Dec 13 06:02:34 ip-172-31-65-37.ec2.internal systemd[1]: Stopping OpenSearch...
Dec 13 06:02:34 ip-172-31-65-37.ec2.internal systemd[1]: Stopped OpenSearch.
Hint: Some lines were ellipsized, use -l to show in full.

[ec2-user@ip-172-31-65-37 ~]$ curl https://localhost:9200 -u admin:admin --insecure
{
  "name" : "ip-172-31-65-37.ec2.internal",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "dc3UAvusS0WE2KMPT_ffJQ",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.2.0",
    "build_type" : "tar",
    "build_hash" : "c459282fd67ddb17dcc545ec9bcdc805880bcbec",
    "build_date" : "2021-11-29T23:26:06.112383Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

Besides, logs have been redirected to /var/log/opensearch

[ec2-user@ip-172-31-65-37 opensearch]$ ll
total 164
-rw-r--r-- 1 opensearch opensearch 11692 Dec 10 00:04 opensearch-2021-12-09-1.json.gz
-rw-r--r-- 1 opensearch opensearch  7555 Dec 10 00:04 opensearch-2021-12-09-1.log.gz
-rw-r--r-- 1 opensearch opensearch  2396 Dec 11 00:04 opensearch-2021-12-10-1.json.gz
-rw-r--r-- 1 opensearch opensearch  1816 Dec 11 00:04 opensearch-2021-12-10-1.log.gz
-rw-r--r-- 1 opensearch opensearch  2344 Dec 12 00:04 opensearch-2021-12-11-1.json.gz
-rw-r--r-- 1 opensearch opensearch  1767 Dec 12 00:04 opensearch-2021-12-11-1.log.gz
-rw-r--r-- 1 opensearch opensearch  2351 Dec 13 00:04 opensearch-2021-12-12-1.json.gz
-rw-r--r-- 1 opensearch opensearch  1734 Dec 13 00:04 opensearch-2021-12-12-1.log.gz
-rw-r--r-- 1 opensearch opensearch  1638 Dec 13 06:05 opensearch_deprecation.json
-rw-r--r-- 1 opensearch opensearch  1902 Dec 13 06:05 opensearch_deprecation.log
-rw-r--r-- 1 opensearch opensearch     0 Dec  9 18:44 opensearch_index_indexing_slowlog.json
-rw-r--r-- 1 opensearch opensearch     0 Dec  9 18:44 opensearch_index_indexing_slowlog.log
-rw-r--r-- 1 opensearch opensearch     0 Dec  9 18:44 opensearch_index_search_slowlog.json
-rw-r--r-- 1 opensearch opensearch     0 Dec  9 18:44 opensearch_index_search_slowlog.log
-rw-r--r-- 1 opensearch opensearch 36431 Dec 13 06:06 opensearch.log
-rw-r--r-- 1 opensearch opensearch 76417 Dec 13 06:06 opensearch_server.json
[ec2-user@ip-172-31-65-37 opensearch]$ cat opensearch.log
[2021-12-13T00:04:36,248][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:09:36,250][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:14:36,250][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:19:36,250][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:24:36,251][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:29:36,251][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:34:36,251][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:39:36,251][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:44:36,252][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:49:36,163][INFO ][o.o.a.t.CronTransportAction] [ip-172-31-65-37.ec2.internal] Start running AD hourly cron.
[2021-12-13T00:49:36,163][INFO ][o.o.a.t.ADTaskManager    ] [ip-172-31-65-37.ec2.internal] Start to maintain running historical tasks
[2021-12-13T00:49:36,164][INFO ][o.o.a.c.HourlyCron       ] [ip-172-31-65-37.ec2.internal] Hourly maintenance succeeds
[2021-12-13T00:49:36,252][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:54:36,252][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T00:59:36,253][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T01:04:36,253][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T01:09:36,253][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T01:14:36,253][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
[2021-12-13T01:19:36,254][INFO ][o.o.j.s.JobSweeper       ] [ip-172-31-65-37.ec2.internal] Running full sweep
......

@zylzulu
Copy link
Contributor

zylzulu commented Dec 13, 2021

Opensearch Dashboards is able to be started and stoped on the AL2 host

[ec2-user@ip-172-31-65-37 ~]$ sudo systemctl start opensearch-dashboards
[ec2-user@ip-172-31-65-37 ~]$ sudo systemctl status opensearch-dashboards
● opensearch-dashboards.service - "OpenSearch Dashboards"
   Loaded: loaded (/etc/systemd/system/opensearch-dashboards.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-12-09 18:48:32 UTC; 3 days ago
 Main PID: 9687 (node)
   CGroup: /system.slice/opensearch-dashboards.service
           └─9687 /usr/share/opensearch-dashboards/bin/../node/bin/node /usr/share/opensearch-dashboards/bin/../src/cli/dist -c /etc/opensearch-dashboards/opensearch_dashboards.yml

Dec 09 18:48:32 ip-172-31-65-37.ec2.internal systemd[1]: Started "OpenSearch Dashboards".

[ec2-user@ip-172-31-65-37 ~]$ sudo systemctl stop opensearch-dashboards
[ec2-user@ip-172-31-65-37 ~]$ sudo systemctl status opensearch-dashboards
● opensearch-dashboards.service - "OpenSearch Dashboards"
   Loaded: loaded (/etc/systemd/system/opensearch-dashboards.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

Dec 09 18:48:32 ip-172-31-65-37.ec2.internal systemd[1]: Started "OpenSearch Dashboards".
Dec 13 06:15:39 ip-172-31-65-37.ec2.internal systemd[1]: Stopping "OpenSearch Dashboards"...
Dec 13 06:15:39 ip-172-31-65-37.ec2.internal systemd[1]: Stopped "OpenSearch Dashboards".

[ec2-user@ip-172-31-65-37 ~]$ curl localhost:5601/api/status
{
   "name":"ip-172-31-65-37.ec2.internal",
   "uuid":"39a18af1-2c45-4429-b803-ed8e57bb6688",
   "version":{
      "number":"1.2.0",
      "build_hash":"caf668e73304bac890f41c37cd6c3a41257cd289",
      "build_number":1,
      "build_snapshot":false
   },
   "status":{
      "overall":{
         "since":"2021-12-13T06:17:08.311Z",
         "state":"green",
         "title":"Green",
         "nickname":"Looking good",
         "icon":"success",
         "uiColor":"secondary"
      },
      "statuses":[
         {
            "id":"core:[email protected]",
            "message":"OpenSearch is available",
            "since":"2021-12-13T06:17:08.311Z",
            "state":"green",
            "icon":"success",
            "uiColor":"secondary"
         },
......

Logs are in /var/log/opensearch-dashboards

[ec2-user@ip-172-31-65-37 opensearch-dashboards]$ ll
total 24
-rw-r--r-- 1 opensearch-dashboards opensearch-dashboards 23652 Dec 13 06:17 opensearch-dashboards.log
[ec2-user@ip-172-31-65-37 opensearch-dashboards]$ cat opensearch-dashboards.log
{"type":"log","@timestamp":"2021-12-09T18:48:34Z","tags":["info","plugins-service"],"pid":9687,"message":"Plugin \"visTypeXy\" is disabled."}
{"type":"log","@timestamp":"2021-12-09T18:48:35Z","tags":["info","plugins-system"],"pid":9687,"message":"Setting up [44] plugins: [alertingDashboards,usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,securityDashboards,reportsDashboards,indexManagementDashboards,anomalyDetectionDashboards,dashboard,visualizations,visTypeVega,visTypeTimeline,timeline,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,ganttChartDashboards,visualize,queryWorkbenchDashboards,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,observabilityDashboards,discover,savedObjectsManagement,bfetch]"}
{"type":"log","@timestamp":"2021-12-09T18:48:35Z","tags":["info","savedobjects-service"],"pid":9687,"message":"Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations..."}
......

@peterzhuamazon peterzhuamazon mentioned this issue Jan 10, 2022
Closed
1 task
@peterzhuamazon peterzhuamazon linked a pull request Jan 10, 2022 that will close this issue
[WIP] Add RPM Assemble code with fpm to assemble_workflow #1443
Closed
1 task
@dblock dblock mentioned this issue Jan 11, 2022
Open
4 tasks
@gaiksaya
Copy link
Member Author

Duplicate issue of #27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@peterzhuamazon peterzhuamazon

Labels
enhancement New Enhancement good first issue Good for newcomers v1.3.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[WIP] Add RPM Assemble code with fpm to assemble_workflow peterzhuamazon/opensearch-build
3 participants
@zylzulu @gaiksaya @peterzhuamazon