Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backup fails after updating to 0.17 #583

Closed
roedie opened this issue Oct 10, 2016 · 8 comments
Closed

Backup fails after updating to 0.17 #583

roedie opened this issue Oct 10, 2016 · 8 comments
Labels

Comments

@roedie
Copy link
Contributor

roedie commented Oct 10, 2016

Today I've updated oxidized to version 0.17 and somehow my smartedges will not backup anymore. Debug output gives:

Puma 2.16.0 starting...
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://0.0.0.0:8888
usage: ssh [-1246AaCfgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec]
           [-D [bind_address:]port] [-E log_file] [-e escape_char]
           [-F configfile] [-I pkcs11] [-i identity_file]
           [-L [bind_address:]port:host:hostport] [-l login_name] [-m mac_spec]
           [-O ctl_cmd] [-o option] [-p port]
           [-Q cipher | cipher-auth | mac | kex | key]
           [-R [bind_address:]port:host:hostport] [-S ctl_path] [-W host:port]
           [-w local_tun[:remote_tun]] [user@]hostname [command]
D, [2016-10-10T21:03:50.918496 #23621] DEBUG -- : x.x.x.x raised Net::SSH::Disconnect with msg "connection closed by remote host"
D, [2016-10-10T21:03:50.918652 #23621] DEBUG -- : lib/oxidized/node.rb: Oxidized::SSH failed for se100
D, [2016-10-10T21:03:50.918790 #23621] DEBUG -- : lib/oxidized/job.rb: Config fetched for se100 at 2016-10-10 19:03:50 UTC
W, [2016-10-10T21:03:51.903758 #23621]  WARN -- : se100 status no_connection, retry attempt 1

My routerdb looks like:

#0   :1    :2    :3    :4       :5       :6     :7    :8             :9
#name:model:group:input:username:password:enable:proxy:ssh_encryption:ssh_kext
se100:ipos:somegroup:ssh::::::

I'm a bit lost why this is happening. Especially with the ssh error right in the beginning. Any pointers or ideas?

Sander

@ytti
Copy link
Owner

ytti commented Oct 10, 2016

I'm traveling until 21st or so, currently on somewhere over greenland with terrible Internet.

Could this be caused by e638e7c - can you change the 'empty' ::: to :nil:nil:

@roedie
Copy link
Contributor Author

roedie commented Oct 11, 2016

Yes that works. But it is confusing to be honest.

I though it would be a wise idea to put nil in all unused parts of my config to make it readable. But then stuff breaks as well. So, some fields need nil and some don't. See:

D, [2016-10-11T09:06:23.238354 #31388] DEBUG -- : resolving DNS for somehost...
D, [2016-10-11T09:06:23.239934 #31388] DEBUG -- : node.rb: resolving node key 'input', with passed global value of 'ssh, telnet' and node value 'nil'
D, [2016-10-11T09:06:23.240249 #31388] DEBUG -- : node.rb: returning node key 'input' with value 'nil'
F, [2016-10-11T09:06:23.255732 #31388] FATAL -- : Oxidized crashed, crashfile written in /home/oxidized/.config/oxidized/crash
nil not found for node x.x.x.x
/var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/node.rb:142:in `block in resolve_input': nil not found for node x.x.x.x (Oxidized::MethodNotFound)
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/node.rb:140:in `map'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/node.rb:140:in `resolve_input'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/node.rb:17:in `initialize'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/nodes.rb:19:in `new'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/nodes.rb:19:in `block (2 levels) in load'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/nodes.rb:15:in `each'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/nodes.rb:15:in `block in load'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/nodes.rb:126:in `synchronize'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/nodes.rb:126:in `with_lock'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/nodes.rb:10:in `load'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/nodes.rb:121:in `initialize'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/core.rb:14:in `new'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/core.rb:14:in `initialize'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/core.rb:4:in `new'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/core.rb:4:in `new'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/lib/oxidized/cli.rb:12:in `run'
    from /var/lib/gems/2.1.0/gems/oxidized-0.17.0/bin/oxidized:9:in `<top (required)>'
    from /usr/local/bin/oxidized:23:in `load'
    from /usr/local/bin/oxidized:23:in `<main>'

While I do understand this field wants 'ssh' or 'telnet' it would be great if it would just do the defaults when it has nil in it. Since using :: in that part works that way.

Also, just removing the ::::: from the end of the router definition works as well.

@ytti
Copy link
Owner

ytti commented Oct 11, 2016

That looks weird, it looks like we're getting string 'nil', not type nil.

If it were type nil, then it should return ssh, telnet: https://github.com/ytti/oxidized/blob/master/lib/oxidized/node.rb#L203

I can't immediately figure out, why it's string "nil" not nil, like it should.

This
https://github.com/ytti/oxidized/blob/master/lib/oxidized/source/csv.rb#L35

should turn it into type nil, because of:
https://github.com/ytti/oxidized/blob/master/lib/oxidized/source/source.rb#L13

Somehow it looks like that's not working, and string 'nil' gets returned.

@roedie
Copy link
Contributor Author

roedie commented Oct 12, 2016

Please understand that I'm not the best coder and Ruby is definitely not my cup of tea, but, if I look at the following:

@cfg.vars_map.each do |key, position|
        vars[key.to_sym] = node_var_interpolate data[position]
end

Doesn't that mean that it only sends vars of vars_map to node_var_interpolate and not the vars that are in map?

So, with the following config:

    map:
      name: 0
      model: 1
      group: 2
      input: 3
      username: 4
      password: 5
    vars_map:
      enable: 6
      ssh_proxy: 7
      ssh_encryption: 8
      ssh_kex: 9
      ssh_port: 10

the settings 0 to 5 will not be send to node_var_interpolate and will just be processed as strings. That would explain what I am seeing.

AFAIK only setting 0 is mandatory and all others might have default settings which are inherited from the config and may have 'nil' in the router.db.

@ytti
Copy link
Owner

ytti commented Oct 12, 2016

Good catch, that's it!

@ytti ytti closed this as completed in 6612535 Oct 12, 2016
@roedie
Copy link
Contributor Author

roedie commented Oct 12, 2016

Woah, you're fast ;-)

@ytti
Copy link
Owner

ytti commented Oct 12, 2016

Lemme know if it works and I'll release 0.17.1

@roedie
Copy link
Contributor Author

roedie commented Oct 13, 2016

Yup, this fixes it.

MajesticFalcon added a commit to MajesticFalcon/oxidized that referenced this issue Feb 6, 2017
* Remove extra new lines added by export command

* Updated Dockerfile to use ruby 2.3

* ruby2.1 behaves oddly with this style of dependcy

fixed ytti#565

* rollback `alvarion` model

wrongly removed in 56bac97

* interpolate node variable values

Empty value was variably considered "" or nil, now it will always be ""
string.

Now "nil", "false", "true" strings are converted to their respective
types nil, false and true.

This also means we cannot have verbatim strings by that name, like if
your password was "false", you're shit out of luck. If this is the case,
we can add some __false__ hack or consider other similar solutions.

Fixes ytti#500
Fixes ytti#534

May break stuff with ""/nil changing now, but as it was not consistent
to begin with, I find it acceptable. Users now may need to manually
enter nil in some fields to regain old behaviour.

* update CHANGELOG

* bump up version

* Redoing the modifications on a up to date base

* Redoing the modifications on a up to date base

* update model names

* add links to models

* add comware link

name contained -, which my replace regexp didn't allow, thus didn't
generate link for that model

* regexp missed this model

* prompt not captured after sending enabe PW

fixes ytti#577

* The \n is not seen as a lineline with ' it need "

* Update README.md

Added correction related to issue ytti#445 and docker-compose file example

* Interpolate also node variables, not just vars map

Fixes ytti#583

* Add nginx reverse proxy example

* Add apache2 reverse proxy example

* Add key authentication to ssh input

* bump up version

* Use variable vars(:ssh_keys) for private key authentication

* Fix blank pfSense configs being saved if regex doesn't match

* Update regex to match newer pfSense config files

* Add missing dependency for Rugged

* Added support for fujitsu blade switches

* Added Support for MRV FiberDriver Linux based management module

* fetch should return string, not array

Also if group was explicitly given, we referred to wrong directory

* return nil if we can't find file

unsure if we really should, or just raise the error and let consumer of
fetch decide what to do with the error

* add support for the HPE BladeSystem

* add Ruby.gitignore from github.com/github/gitignore

* Allow model to specify SSH PTY options

* it shouldn't be a private method

* add Planet SGS switches

* add to README

* Update dnos.rb

Handles old DNOS switches that do not allow term length 0 prior to enable, and double exits in case of enable..

* expect prompt after sending enable password

* Added support for Trango/Trangolink-45

I only have the Trangolink-45 so I'm not sure which other radios this
might support.

* This adds support for the Casa C1G CMTS

This will probably work for the other Casa CMTS as well.

* This adds support for D-Link switches

This specifically supports DES-3526/3550 series switches.

* This adds support for Hatteras Networks devices

* renamed  alvarion -> alvarion.rb

I believe from this:
require File.join dir, file+'.rb'

That the only way this model would work is if it has a .rb extension.  I
don't have any of these devices to test if this works.

* add support for PLANET SG switches

So far, only SGS switches were supported.

Now, we check the model type during the 'show version' command, and only
execute the 'show transceiver details' command in case an SGS switch was
detected (as its not supported on SG models).
We will also strip lines containing the current System Time and Uptime.
These only appear on SG models, but it's a good idea to strip them anyways.

* expect prompt after entering enable password

* Update ciscosmb.rb

Adapted for SG series, not sure about SF.

Please test.

* Update ciscosmb.rb

Forgot the username prompt change.

* Update ciscosmb.rb

Added old commands in case of different CLI variations.

* Added option to disable ssl verification checks for http source

* Updated config options

* New hook: awssns - Publish messages to AWS SNS topics

* exclude time from output

* Update eos.rb

* Remove trailing whitespace and enable prompt detection

Fixes: ytti#630

* Fix suggested by ytti for issue ytti#610

* Recursively search from one dir above specified

Fixes ytti#626

* update changelogs

* bump up version

* delete secret password if it is called secret

* documention: debug inside input is now boolean

* Update ios.rb

Do the terminal configure after enable since it might be prohibited in non ena

* Hide remaining passwords

* adding zhone OLT/MX GPON/MetroE/DSL gear

* Ignore undefined UTF8 in ironware.rb

We have some Brocade MLX devices that are triggering  'raised Encoding::UndefinedConversionError with msg ""\xFF" from ASCII-8BIT to UTF-8"'.  Update the ironware.rb file to ignore undefined UTF8 as well as invalid UTF8

* Add support for telnet

* Mask out configuration date for fiberdriver

Our fiberdriver devices report a timestamp during 'show running-config', which results in a config diff on every run.  This patch removes the '+! Configuration saved on 2017/01/10 14:21:20' line from the config, as well as a couple other useless status messages

* Ensure config gets returned even if no replacements are done

* Ignore fan speed changes for ironware

Brocade ICX/FCX log fan speed changes like this:

Fan ok, speed (auto): 1<->[[2]]<->3
Fan ok, speed (auto): [[1]]<->2<->3

Remove the current fan speed from the configuration to prevent lots of useless changes from being logged.

* Add suggestion to set `ip` variable in CSV reader

We have around ~1000 nodes, and oxidized was taking 30s to do the initial config load.  After adding the 'ip' variable to our CSV, load time dropped to less than 1 second.

* Update ironware.rb

Some Brocade devices only have two possible fan speeds.  Previously, this only handled three levels of fan speeds.  Updated regex will handle both of these cases:

    Fan 1 ok, speed (auto): 1<->[[2]]
    Fan 1 ok, speed (auto): 1<->[[2]]<->3

* Tweak the white space for cleaning.

* Untested GPG work

* Syntax error

* Update

* GPG support

* GPG support working

* Requested changes

* Update

* Don't show error is unsupported

This is the proposed fix for issue ytti#669. Let me know if there is anything that I need to fix.
This appears to be working correctly for the switch I have in house (S2400-24P). However the model is not working for the Aruba Instant AP (IAP) (virtual) controller. I hope to submit a fix for those once I have it working. The problems with that device appear to be unrelated to these changes as the prompt is incorrect.

* Filter out rogue client alerts from AireOS

* Added git build

* Adjustments

* feature: support IP Addressed provided with a netmask

* Updated split to be more efficient, thanks ytti!

* Update fabricos.rb

Ignore Uptime/Power/Date fields in chassisShow and configShow

* Restore compatibility with older versions of IronWare
Remove \r from the "enable" command. Both \r and \n are interpreted the same on older versions, causing a blank line to be sent as the enable password.
Add a delay between the sending of the "logout" command and each "exit" command. Older devices don't seem to be able to keep up with the speed the commands are sent without a delay.

* Cleaner way to fix older IronWare devices
This works on older IronWare devices without needing a sleep between each command.

* added secret removal for fortiOS

* nicer regex for both password variants of fortiOS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants