Could not fetch specs from https://rubygems.org/

[eromoe]

Because china block the official source, we use https://ruby.taobao.org/ to instead.
But logstash do not use the source I set:

D:\logstash-1.5.5\bin>gem sources -l
*** CURRENT SOURCES ***

https://ruby.taobao.org/

D:\logstash-1.5.5\bin>plugin install --no-verify logstash-input-mongodb
io/console not supported; tty will not be manipulated
Installing logstash-input-mongodb
Error Bundler::HTTPError, retrying 1/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 2/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 3/10
Could not fetch specs from https://rubygems.org/
.....

[purbon]

@eromoe you need to change your Gemfile, the line where it says sources. Then with the --no-verify should do the work. You can also change the source globally by using gem source ....

there is one PR waiting to be merge that will fix that issue for you #3583

let me know if what I say helps you.

[purbon]

@eromoe if you change your source as you said, or also changed the sources in the Gemfile, this error you see is strange. I can not reproduce it and I guess is due to some other hiccups, might be some cache, who knows. As you said in your last comment, not sure why is removed, it works for you. Can you confirm this again?

My output doing the same thing as you, but with a local gem repo, is:

skywalker% DEBUG=true bin/plugin install logstash-output-mongodb
DEBUG: exec /Users/purbon/work/logstash/build/logstash-3.0.0.dev/vendor/jruby/bin/jruby --1.9 -J-XX:+UseParNewGC -J-XX:+UseConcMarkSweepGC -J-Djava.awt.headless=true -J-XX:CMSInitiatingOccupancyFraction=75 -J-XX:+UseCMSInitiatingOccupancyOnly -J-Xmx1g /Users/purbon/work/logstash/build/logstash-3.0.0.dev/lib/pluginmanager/main.rb install logstash-output-mongodb
Validating logstash-output-mongodb
Installing logstash-output-mongodb
Installation successful
Bundler output
HTTP GET http://localhost:9292/api/v1/dependencies
HTTP 200 OK
Fetching gem metadata from http://localhost:9292/
Query List: ["file-dependencies", "ci_reporter_rspec", "simplecov", "coveralls", "tins", "rspec", "logstash-devutils", "benchmark-ips", "octokit", "stud", "fpm", "rubyzip", "gems", "flores", "logstash-input-heartbeat", "logstash-output-zeromq", "logstash-codec-collectd", "logstash-output-xmpp", "logstash-codec-dots", "logstash-codec-edn", "logstash-codec-edn_lines", "logstash-codec-fluent", "logstash-codec-es_bulk", "logstash-co

as you can see the data is fetched not from rubygems, but from the local gem repo.

Closing this for now, feel free to reopen if you can provide more information.

[Neo23x0]

I am using version 2.1.1 on SLES and still have this issue that even when I use "--local" and "--no-verify" it tries to connect to "https://rubygems.org".

/opt/logstash/bin> rpm -q logstash
logstash-2.1.1-1.noarch
/opt/logstash/bin> sudo ./plugin install --local --no-verify /secmon/plugins/logstash-output-syslog-2.1.0.gem
Installing logstash-output-syslog
Error Bundler::HTTPError, retrying 1/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 2/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 3/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 4/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 5/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 6/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 7/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 8/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 9/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 10/10
Could not fetch specs from https://rubygems.org/
Too many retries, aborting, caused by Bundler::HTTPError
ERROR: Installation Aborted, message: Could not fetch specs from https://rubygems.org/

[purbon]

@Neo23x0 the right command to install from an offline package would be

$> plugin install --local --no-verify logstash-output-syslog

after properly installing locally the package generated by `bin/plugin pack``

your command is actually trying to install the gem plugin directly as a local gem, so it try to fetch dependencies from the internet and not locally as expected.

I see this as an actual improvement for the plugin manager, can you please open a new issue with it?

Thanks a lot!

[Neo23x0]

Actually trying to install the gem directly was the second try. (and yes unpack runs without errors)

/opt/logstash/bin> sudo ./plugin install --local --no-verify logstash-output-syslog
Installing logstash-output-syslog
Error Bundler::HTTPError, retrying 1/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 2/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 3/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 4/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 5/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 6/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 7/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 8/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 9/10
Could not fetch specs from https://rubygems.org/
Error Bundler::HTTPError, retrying 10/10
Could not fetch specs from https://rubygems.org/
Too many retries, aborting, caused by Bundler::HTTPError
ERROR: Installation Aborted, message: Could not fetch specs from https://rubygems.org/

Unpack

/opt/logstash/bin> sudo ./plugin unpack /home/secmon/plugins_package.tar.gz
Unpacking /home/secmon/plugins_package.tar.gz
Unpacked at /opt/logstash/vendor/cache
The unpacked plugins can now be installed in local-only mode using bin/plugin install --local [plugin name]

logstash-output-syslog-2.1.0.gem is in the root of the .tar.gz

[purbon]

Interesting, this should obviously not happen. Will investigate.

• Can you let me know a bit more about your env?
• os version, java version, jruby version, ....
• do you have a package installed? look for a dir under vendor/cache

thanks a lot for your report.

[Neo23x0]

System:

SuSE Enterprise Linux (SLES) 12 (x64)
ruby-2.1-1.6.x86_64

java -version

java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470_27sr3fp10-20150708_01(SR3 FP10))
IBM J9 VM (build 2.7, JRE 1.7.0 Linux amd64-64 Compressed References 20150630_255653 (JIT enabled, AOT enabled)
J9VM - R27_Java727_SR3_20150630_2236_B255653
JIT - tr.r13.java_20150623_94888.01
GC - R27_Java727_SR3_20150630_2236_B255653_CMPRSS
J9CL - 20150630_255653)
JCL - 20150628_01 based on Oracle jdk7u85-b15

I have sudo rights on bin/plugins but the system administrator executed the commands on my request with the same results.
I'll ask him if he can give me a directory listing of the cache directory. (root:root)

[purbon]

@Neo23x0 interesting bug, I just tried with a centos6 and this is what I get (machine completely offline) and using the rpm logstash package.

[vagrant@localhost logstash]$ sudo ./bin/plugin install --local --no-verify logstash-output-syslog
Installing logstash-output-syslog
Plugin not found, aborting
ERROR: Installation Aborted, message: Could not find gem 'logstash-output-syslog (>= 0) java' in any of the gem sources listed in your Gemfile or installed on this machine.
[vagrant@localhost logstash]$ yum list | grep logstash
Failed to set locale, defaulting to C
logstash.noarch                            1:2.1.1-1                    @logstash-2.1

[Neo23x0]

logstash-output-syslog is not in the default packages.

I used a package system, which is Ubuntu as described here:
https://www.elastic.co/guide/en/logstash/current/offline-plugins.html

I installed logstash-output-syslog there (online) then packed and unpacked on the target machine (offline), which is SLES 12. On the target machine, logstash-output-syslog should be in the local cache folder after "unpack" and would therefore not produce the error "Plugin not found, aborting" as in your case, right?

[purbon]

this helps a lot, thanks a lot for your feedback. Danke sehr! I will reproduce.

[Neo23x0]

De nada ;)
I was debugging the issue myself on the Ubuntu machine, which is a virtual machine and blocked all Internet traffic on this host with LittleSnitch - when I tried the "unpack" and "install --local" not a single request was intercepted. (> maybe it is somehow relevant if a system had Internet access once)

[purbon]

@Neo23x0 if everything went right, you have to see this:

[vagrant@localhost logstash]$ ls -la vendor/cache/ | grep syslog
-rw-r--r-- 1 root     root         8704 Jan 21 15:34 logstash-filter-syslog_pri-2.0.2.gem
-rw-r--r-- 1 root     root        11776 Jan 21 15:34 logstash-input-syslog-2.0.2.gem
-rw-r--r-- 1 root     root         9216 Jan 21 15:34 logstash-output-syslog-2.1.0.gem

in your target machine cache directory.

if this happen, logstash is expected to pick it up from there and install it without issues.

[purbon]

maybe if a system had Internet access once is somehow relevant

this is not relevant to my understanding, if no internet there is no internet.

[purbon]

@Neo23x0 I'm lost with this issue, just tried all your workflow with same config and OS, no connection to outside world for sure, and

[vagrant@localhost logstash]$ ls -la vendor/cache/ | grep syslog
-rw-r--r-- 1 root     root         8704 Jan 21 15:34 logstash-filter-syslog_pri-2.0.2.gem
-rw-r--r-- 1 root     root        11776 Jan 21 15:34 logstash-input-syslog-2.0.2.gem
-rw-r--r-- 1 root     root         9216 Jan 21 15:34 logstash-output-syslog-2.1.0.gem
[vagrant@localhost logstash]$ ls
CHANGELOG.md  CONTRIBUTORS  Gemfile  Gemfile.jruby-1.9.lock  LICENSE  NOTICE.TXT  bin  lib  plugins_package.tar.gz  vendor
[vagrant@localhost logstash]$  sudo ./bin/plugin install --local --no-verify logstash-output-syslog
Installing logstash-output-syslog
Installation successful

everything installed as expected from local repo.

there have to be something else happening here. can you share:

• the content of vendor/cache.
• the output of env.

[Neo23x0]

I got a directory listing of the folder /opt/logstash/vendor/cache/

total 54652
-rw-r----- 1 root root    99840 Jan 21 16:02 addressable-2.3.8.gem
-rw-r----- 1 root root    15360 Jan 21 16:02 arr-pm-0.0.10.gem
-rw-r----- 1 root root    24064 Jan 21 16:02 atomic-1.1.99-java.gem
-rw-r----- 1 root root    12800 Jan 21 16:02 avl_tree-1.2.1.gem
-rw-r----- 1 root root    32256 Jan 21 16:02 awesome_print-1.6.1.gem
-rw-r----- 1 root root     4608 Jan 21 16:02 aws-sdk-2.1.36.gem
-rw-r----- 1 root root   483328 Jan 21 16:02 aws-sdk-core-2.1.36.gem
-rw-r----- 1 root root    44032 Jan 21 16:02 aws-sdk-resources-2.1.36.gem
-rw-r----- 1 root root   763392 Jan 21 16:02 aws-sdk-v1-1.66.0.gem
-rw-r----- 1 root root    88576 Jan 21 16:02 backports-3.6.7.gem
-rw-r----- 1 root root    71680 Jan 21 16:02 bindata-2.1.0.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 buftok-0.2.0.gem
-rw-r----- 1 root root    20992 Jan 21 16:02 cabin-0.7.2.gem
-rw-r----- 1 root root    29184 Jan 21 16:02 childprocess-0.5.8.gem
-rw-r----- 1 root root    80384 Jan 21 16:02 cinch-2.3.1.gem
-rw-r----- 1 root root    24576 Jan 21 16:02 clamp-0.6.5.gem
-rw-r----- 1 root root    95232 Jan 21 16:02 coderay-1.1.0.gem
-rw-r----- 1 root root   111104 Jan 21 16:02 concurrent-ruby-0.9.2-java.gem
-rw-r----- 1 root root   112128 Jan 21 16:02 domain_name-0.5.25.gem
-rw-r----- 1 root root   446976 Jan 21 16:02 edn-1.1.0.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 elasticsearch-1.0.15.gem
-rw-r----- 1 root root    75776 Jan 21 16:02 elasticsearch-api-1.0.15.gem
-rw-r----- 1 root root    34816 Jan 21 16:02 elasticsearch-transport-1.0.15.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 equalizer-0.0.10.gem
-rw-r----- 1 root root    35840 Jan 21 16:02 faraday-0.9.2.gem
-rw-r----- 1 root root    13312 Jan 21 16:02 ffi-1.9.10-java.gem
-rw-r----- 1 root root    47104 Jan 21 16:02 ffi-rzmq-2.0.4.gem
-rw-r----- 1 root root    13312 Jan 21 16:02 ffi-rzmq-core-1.0.4.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 file-dependencies-0.1.6.gem
-rw-r----- 1 root root     6144 Jan 21 16:02 filesize-0.0.4.gem
-rw-r----- 1 root root    17408 Jan 21 16:02 filewatch-0.7.1.gem
-rw-r----- 1 root root   114176 Jan 21 16:02 fpm-1.3.3.gem
-rw-r----- 1 root root    14336 Jan 21 16:02 gelf-1.3.2.gem
-rw-r----- 1 root root     7680 Jan 21 16:02 gelfd-0.2.0.gem
-rw-r----- 1 root root    84992 Jan 21 16:02 gems-0.8.3.gem
-rw-r----- 1 root root    66560 Jan 21 16:02 geoip-1.6.1.gem
-rw-r----- 1 root root     7168 Jan 21 16:02 gmetric-0.1.3.gem
-rw-r----- 1 root root    20480 Jan 21 16:02 hipchat-1.5.2.gem
-rw-r----- 1 root root    38400 Jan 21 16:02 hitimes-1.2.3-java.gem
-rw-r----- 1 root root    66560 Jan 21 16:02 http-0.9.8.gem
-rw-r----- 1 root root    37888 Jan 21 16:02 http-cookie-1.0.2.gem
-rw-r----- 1 root root    13824 Jan 21 16:02 http-form_data-1.0.1.gem
-rw-r----- 1 root root   256000 Jan 21 16:02 http_parser.rb-0.6.0-java.gem
-rw-r----- 1 root root    81920 Jan 21 16:02 httparty-0.13.7.gem
-rw-r----- 1 root root    60416 Jan 21 16:02 i18n-0.6.9.gem
-rw-r----- 1 root root    22528 Jan 21 16:02 jar-dependencies-0.3.1.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 jls-grok-0.11.2.gem
-rw-r----- 1 root root    12800 Jan 21 16:02 jls-lumberjack-0.0.26.gem
-rw-r----- 1 root root    18944 Jan 21 16:02 jmespath-1.1.3.gem
-rw-r----- 1 root root  1516032 Jan 21 16:02 jrjackson-0.3.8.gem
-rw-r----- 1 root root 13627904 Jan 21 16:02 jruby-kafka-1.5.0-java.gem
-rw-r----- 1 root root  4348928 Jan 21 16:02 jruby-openssl-0.9.14-java.gem
-rw-r----- 1 root root    89600 Jan 21 16:02 json-1.8.3-java.gem
-rw-r----- 1 root root    18944 Jan 21 16:02 logstash-codec-collectd-2.0.2.gem
-rw-r----- 1 root root     7680 Jan 21 16:02 logstash-codec-dots-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-codec-edn-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-codec-edn_lines-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-codec-es_bulk-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-codec-fluent-2.0.2-java.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-codec-graphite-2.0.2.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-codec-json-2.0.4.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-codec-json_lines-2.0.3.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-codec-line-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-codec-msgpack-2.0.2-java.gem
-rw-r----- 1 root root    17408 Jan 21 16:02 logstash-codec-multiline-2.0.6.gem
-rw-r----- 1 root root    13824 Jan 21 16:02 logstash-codec-netflow-2.0.3.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-codec-oldlogstashjson-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-codec-plain-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-codec-rubydebug-2.0.5.gem
-rw-r----- 1 root root    87552 Jan 21 16:02 logstash-core-2.1.1-java.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-filter-anonymize-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-filter-checksum-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-filter-clone-2.0.4.gem
-rw-r----- 1 root root    10240 Jan 21 16:02 logstash-filter-csv-2.1.1.gem
-rw-r----- 1 root root    13824 Jan 21 16:02 logstash-filter-date-2.1.1.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-filter-dns-2.0.2.gem
-rw-r----- 1 root root     7680 Jan 21 16:02 logstash-filter-drop-2.0.2.gem
-rw-r----- 1 root root    10752 Jan 21 16:02 logstash-filter-fingerprint-2.0.3.gem
-rw-r----- 1 root root 13906944 Jan 21 16:02 logstash-filter-geoip-2.0.5.gem
-rw-r----- 1 root root    15872 Jan 21 16:02 logstash-filter-grok-2.0.3.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-filter-json-2.0.3.gem
-rw-r----- 1 root root    12800 Jan 21 16:02 logstash-filter-kv-2.0.3.gem
-rw-r----- 1 root root    11264 Jan 21 16:02 logstash-filter-metrics-3.0.0.gem
-rw-r----- 1 root root    12288 Jan 21 16:02 logstash-filter-multiline-2.0.3.gem
-rw-r----- 1 root root    14848 Jan 21 16:02 logstash-filter-mutate-2.0.3.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-filter-ruby-2.0.3.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-filter-sleep-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-filter-split-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-filter-syslog_pri-2.0.2.gem
-rw-r----- 1 root root    10240 Jan 21 16:02 logstash-filter-throttle-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-filter-urldecode-2.0.2.gem
-rw-r----- 1 root root    22016 Jan 21 16:02 logstash-filter-useragent-2.0.4.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-filter-uuid-2.0.3.gem
-rw-r----- 1 root root    10240 Jan 21 16:02 logstash-filter-xml-2.1.1.gem
-rw-r----- 1 root root    28672 Jan 21 16:02 logstash-input-beats-2.1.2.gem
-rw-r----- 1 root root    18944 Jan 21 16:02 logstash-input-couchdb_changes-2.0.2.gem
-rw-r----- 1 root root    11264 Jan 21 16:02 logstash-input-elasticsearch-2.0.3.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-input-eventlog-3.0.1.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-input-exec-2.0.4.gem
-rw-r----- 1 root root    14848 Jan 21 16:02 logstash-input-file-2.1.3.gem
-rw-r----- 1 root root    12800 Jan 21 16:02 logstash-input-ganglia-2.0.4.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-input-gelf-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-input-generator-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-input-graphite-2.0.5.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-input-heartbeat-2.0.2.gem
-rw-r----- 1 root root    11264 Jan 21 16:02 logstash-input-http-2.1.0.gem
-rw-r----- 1 root root    10240 Jan 21 16:02 logstash-input-imap-2.0.3.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-input-irc-2.0.3.gem
-rw-r----- 1 root root    16384 Jan 21 16:02 logstash-input-jdbc-3.0.0.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 logstash-input-kafka-2.0.3.gem
-rw-r----- 1 root root   454144 Jan 21 16:02 logstash-input-log4j-2.0.5-java.gem
-rw-r----- 1 root root    13312 Jan 21 16:02 logstash-input-lumberjack-2.0.5.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-input-pipe-2.0.2.gem
-rw-r----- 1 root root    10752 Jan 21 16:02 logstash-input-rabbitmq-3.1.2.gem
-rw-r----- 1 root root    12288 Jan 21 16:02 logstash-input-redis-2.0.2.gem
-rw-r----- 1 root root    15872 Jan 21 16:02 logstash-input-s3-2.0.4.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-input-snmptrap-2.0.2.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 logstash-input-sqs-2.0.3.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-input-stdin-2.0.2.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 logstash-input-syslog-2.0.2.gem
-rw-r----- 1 root root    14336 Jan 21 16:02 logstash-input-tcp-3.0.1.gem
-rw-r----- 1 root root    12800 Jan 21 16:02 logstash-input-twitter-2.2.0.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-input-udp-2.0.3.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-input-unix-2.0.4.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-input-xmpp-2.0.3.gem
-rw-r----- 1 root root    10752 Jan 21 16:02 logstash-input-zeromq-2.0.2.gem
-rw-r----- 1 root root    10240 Jan 21 16:02 logstash-mixin-aws-2.0.2.gem
-rw-r----- 1 root root    10752 Jan 21 16:02 logstash-mixin-http_client-2.2.0.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-mixin-rabbitmq_connection-2.3.0-java.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 logstash-output-cloudwatch-2.0.2.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-output-csv-2.0.3.gem
-rw-r----- 1 root root    28672 Jan 21 16:02 logstash-output-elasticsearch-2.4.0-java.gem
-rw-r----- 1 root root    10240 Jan 21 16:02 logstash-output-email-3.0.2.gem
-rw-r----- 1 root root     7680 Jan 21 16:02 logstash-output-exec-2.0.2.gem
-rw-r----- 1 root root    12800 Jan 21 16:02 logstash-output-file-2.2.3.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-output-ganglia-2.0.2.gem
-rw-r----- 1 root root    10752 Jan 21 16:02 logstash-output-gelf-2.0.3.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 logstash-output-graphite-2.0.3.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-output-hipchat-3.0.2.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 logstash-output-http-2.1.0.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-output-irc-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-output-juggernaut-2.0.2.gem
-rw-r----- 1 root root    12288 Jan 21 16:02 logstash-output-kafka-2.0.1.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-output-lumberjack-2.0.4.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-output-nagios-2.0.2.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-output-nagios_nsca-2.0.3.gem
-rw-r----- 1 root root     7680 Jan 21 16:02 logstash-output-null-2.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-output-opentsdb-2.0.2.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-output-pagerduty-2.0.2.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-output-pipe-2.0.2.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-output-rabbitmq-3.0.7-java.gem
-rw-r----- 1 root root    10752 Jan 21 16:02 logstash-output-redis-2.0.2.gem
-rw-r----- 1 root root    15360 Jan 21 16:02 logstash-output-s3-2.0.4.gem
-rw-r----- 1 root root    10240 Jan 21 16:02 logstash-output-sns-3.0.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-output-sqs-2.0.2.gem
-rw-r----- 1 root root    10240 Jan 21 16:02 logstash-output-statsd-2.0.5.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-output-stdout-2.0.4.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 logstash-output-syslog-2.1.0.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 logstash-output-tcp-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-output-udp-2.0.2.gem
-rw-r----- 1 root root     8192 Jan 21 16:02 logstash-output-xmpp-2.0.2.gem
-rw-r----- 1 root root     9728 Jan 21 16:02 logstash-output-zeromq-2.0.2.gem
-rw-r----- 1 root root    28160 Jan 21 16:02 logstash-patterns-core-2.0.2.gem
-rw-r----- 1 root root    12288 Jan 21 16:02 lru_redux-1.1.0.gem
-rw-r----- 1 root root   336384 Jan 21 16:02 mail-2.6.3.gem
-rw-r----- 1 root root    53760 Jan 21 16:02 manticore-0.5.2-java.gem
-rw-r----- 1 root root   451072 Jan 21 16:02 march_hare-2.15.0-java.gem
-rw-r----- 1 root root    13312 Jan 21 16:02 memoizable-0.4.2.gem
-rw-r----- 1 root root    13312 Jan 21 16:02 method_source-0.8.2.gem
-rw-r----- 1 root root    20992 Jan 21 16:02 metriks-0.9.9.7.gem
-rw-r----- 1 root root   147456 Jan 21 16:02 mime-types-2.99.gem
-rw-r----- 1 root root   352768 Jan 21 16:02 mimemagic-0.3.1.gem
-rw-r----- 1 root root    23552 Jan 21 16:02 minitar-0.5.4.gem
-rw-r----- 1 root root   826880 Jan 21 16:02 msgpack-jruby-1.4.1-java.gem
-rw-r----- 1 root root    25088 Jan 21 16:02 multi_json-1.11.2.gem
-rw-r----- 1 root root    20992 Jan 21 16:02 multi_xml-0.5.5.gem
-rw-r----- 1 root root    11264 Jan 21 16:02 multipart-post-2.0.0.gem
-rw-r----- 1 root root    14336 Jan 21 16:02 murmurhash3-0.1.6-java.gem
-rw-r----- 1 root root    19456 Jan 21 16:02 naught-1.1.0.gem
-rw-r----- 1 root root  2484736 Jan 21 16:02 nokogiri-1.6.7.1-java.gem
-rw-r----- 1 root root    57344 Jan 21 16:02 octokit-3.8.0.gem
-rw-r----- 1 root root        0 Jan 21 16:17 output.txt
-rw-r----- 1 root root     8192 Jan 21 16:02 polyglot-0.3.5.gem
-rw-r----- 1 root root   135680 Jan 21 16:02 pry-0.10.3-java.gem
-rw-r----- 1 root root   120832 Jan 21 16:02 puma-2.11.3-java.gem
-rw-r----- 1 root root   228864 Jan 21 16:02 rack-1.6.4.gem
-rw-r----- 1 root root    75776 Jan 21 16:02 redis-3.2.2.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 ruby-maven-3.3.8.gem
-rw-r----- 1 root root  8104448 Jan 21 16:02 ruby-maven-libs-3.3.3.gem
-rw-r----- 1 root root    98816 Jan 21 16:02 rubyzip-1.1.7.gem
-rw-r----- 1 root root    56320 Jan 21 16:02 rufus-scheduler-3.0.9.gem
-rw-r----- 1 root root    15872 Jan 21 16:02 sawyer-0.6.0.gem
-rw-r----- 1 root root  1331200 Jan 21 16:02 sequel-4.30.0.gem
-rw-r----- 1 root root     8704 Jan 21 16:02 simple_oauth-0.3.1.gem
-rw-r----- 1 root root    23552 Jan 21 16:02 slop-3.6.0.gem
-rw-r----- 1 root root   205824 Jan 21 16:02 snmp-1.2.0.gem
-rw-r----- 1 root root     9216 Jan 21 16:02 spoon-0.0.4.gem
-rw-r----- 1 root root    11264 Jan 21 16:02 statsd-ruby-1.2.0.gem
-rw-r----- 1 root root    15872 Jan 21 16:02 stud-0.0.22.gem
-rw-r----- 1 root root   227840 Jan 21 16:02 thread_safe-0.3.5-java.gem
-rw-r----- 1 root root    78336 Jan 21 16:02 treetop-1.4.15.gem
-rw-r----- 1 root root    75776 Jan 21 16:02 twitter-5.15.0.gem
-rw-r----- 1 root root   147456 Jan 21 16:02 tzinfo-1.2.2.gem
-rw-r----- 1 root root   310784 Jan 21 16:02 tzinfo-data-1.2015.7.gem
-rw-r----- 1 root root   120832 Jan 21 16:02 unf-0.1.4-java.gem
-rw-r----- 1 root root    41984 Jan 21 16:02 user_agent_parser-2.3.0.gem
-rw-r----- 1 root root    33280 Jan 21 16:02 win32-eventlog-0.6.5.gem
-rw-r----- 1 root root    11776 Jan 21 16:02 xml-simple-1.1.5.gem
-rw-r----- 1 root root   175616 Jan 21 16:02 xmpp4r-0.5.gem

[Neo23x0]

Oh wait - there is one more thing to mention:
The system is connected to the local network, but unable to lookup rubygems.org via DNS but has a http_proxy / https_proxy configured. The only thing is that the proxy system intercepts HTTPS traffic and presents a signed certificate from an untrusted CA.
I'll try to unset the proxy environment variables.

[Neo23x0]

Is there a fail over to the online download if the local gem cannot be found?

[untergeek]

@Neo23x0 There's one more thing you brought up that may be playing into this (emphasis on may):

java -version

java version "1.7.0"
Java(TM) SE Runtime Environment (build pxa6470_27sr3fp10-20150708_01(SR3 FP10))
IBM J9 VM (build 2.7, JRE 1.7.0 Linux amd64-64 Compressed References 20150630_255653 (JIT enabled, AOT enabled)
J9VM - R27_Java727_SR3_20150630_2236_B255653
JIT - tr.r13.java_20150623_94888.01
GC - R27_Java727_SR3_20150630_2236_B255653_CMPRSS
J9CL - 20150630_255653)
JCL - 20150628_01 based on Oracle jdk7u85-b15

Logstash and Elasticsearch do not support the IBM JDK/JRE. I know some users are able to use it, but with mixed results. This adds a level of complexity because it will be hard for us to accurately reproduce your results.

[purbon]

I agree with you @untergeek, @Neo23x0 this was never tested with IBM jdk, are you able to test with oracles or openjdk?

[Neo23x0]

So, @purbon you tested on a SLES Linux with Oracle JDK yesterday?

[Neo23x0]

Do I need the JDK for Logstash?
I've found a tutorial for the JRE on SLES Linux:
http://www.unixmen.com/install-java-8-opensuse-13-x-sles-servers/

[purbon]

@Neo23x0 yes, with oracle jdk and openjdk. Will test today with IBM jdk to see if this is actually the issue, generally speaking is a good guess as we have seen very strange behaviour with this jdk.

JRE is enough to run logstash.

[Neo23x0]

Ha - check this out.

• SLES 12 - fresh install with OpenJDK
• logstash-2.1.1-1.noarch.rpm
• plugins_pack.tar.gz downloaded
• plugin unpack
• plugin install --local --no-verify > requests Internet access to https:://bundler.rubygems.org and https://rubygems.org (not in this screenshot)

[purbon]

Interesting,
thanks a lot for your feedback and intense testing, this is much
appreciate it. I will certainly work on a solution to this, looks there is
something fishy going on.

• purbon

On Fri, Jan 22, 2016 at 11:17 AM Florian Roth [email protected]
wrote:

Ha - check this out.

• SLES 12 - fresh install with OpenJDK
• plugins_pack.tar.gz downloaded
• plugin unpack
• plugin install --local --no-verify > requests Internet access to
  https:://bundler.rubygems.org and https://rubygems.org (not in this
  screenshot)

[image: screen shot 2016-01-22 at 11 14 53]
https://cloud.githubusercontent.com/assets/2851492/12507986/77c80212-c0f9-11e5-8ef3-2819f67aec21.png

—
Reply to this email directly or view it on GitHub
#4189 (comment).

[purbon]

I must say, I've never heard about a host named bundler.rubygems.org :-(

• purbon

On Fri, Jan 22, 2016 at 11:27 AM Pere Urbon-Bayes [email protected]
wrote:

Interesting,
thanks a lot for your feedback and intense testing, this is much
appreciate it. I will certainly work on a solution to this, looks there is
something fishy going on.

• purbon

On Fri, Jan 22, 2016 at 11:17 AM Florian Roth [email protected]
wrote:

Ha - check this out.

• SLES 12 - fresh install with OpenJDK
• plugins_pack.tar.gz downloaded
• plugin unpack
• plugin install --local --no-verify > requests Internet access to
  https:://bundler.rubygems.org and https://rubygems.org (not in this
  screenshot)

[image: screen shot 2016-01-22 at 11 14 53]
https://cloud.githubusercontent.com/assets/2851492/12507986/77c80212-c0f9-11e5-8ef3-2819f67aec21.png

—
Reply to this email directly or view it on GitHub
#4189 (comment).

[Neo23x0]

Damnit !
Same result with Oracle JRE.

Connects to
https://bundler.rubygems.org
https://rubygems.org

[Neo23x0]

I'll try to figure out what the actual problem is.
It seems to be related with gem installation:
rubygems/bundler#2769

[Neo23x0]

Oh damnit ... even though the installation has these errors, the plugin seems to install and is usable on my test machine. Now I'll try to reproduce this in production.

[Neo23x0]

Works also on production system.
So - logstash-output-syslog plugin can be used although installation terminates with an error.

BTW other issue that made debugging harder: The plugin installation with "sudo" caused the gem and gemspecs files to be written as "root:root" with no read rights for everyone. This caused logstash that runs as "logstash" user in our environment to ignore the module on initialization printing the "missing plugin" message to stdout.