[BUGError executing net.cli "show tech-support" on Cisco IOS XE device

[vriccio-auto]

Description
Trying to collect tech-support in switch (Cisco IOS XE) device.

Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info).

Steps to Reproduce the behavior
[root@salt-prod-master-1 ~]# salt 'blr0' net.cli "show tech-support"

blr0:

comment:
Cannot execute "cli" on blr0.vmware.com as svc.salt.net. Reason: Search pattern never detected in send_command: blr0-core01#!
out:

result:
False
traceback:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 205, in _send_command
output = self.device.send_command(command)
File "/usr/local/lib/python3.6/site-packages/netmiko/utilities.py", line 500, in wrapper_decorator
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/netmiko/base_connection.py", line 1537, in send_command
search_pattern
OSError: Search pattern never detected in send_command: blr02-eng-core01#

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/salt/utils/napalm.py", line 171, in call
out = getattr(napalm_device.get("DRIVER"), method)(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 2363, in cli
output = self._send_command(command)
File "/usr/local/lib/python3.6/site-packages/napalm/ios/ios.py", line 208, in _send_command
raise ConnectionClosedException(str(e))
napalm.base.exceptions.ConnectionClosedException: Search pattern never detected in send_command: blr02-eng-core01#
ERROR: Minions returned with non-zero exit code

Error in Salt- Minion (proxy process in debug mode)

[DEBUG ] LazyLoaded status.proxy_reconnect
[DEBUG ] schedule.handle_func: adding this job to the jobcache with data {'id': 'blr02-core01', 'fun': 'status.proxy_reconnect', 'fun_args': [

{'proxy_name': 'napalm'}
], 'schedule': '__proxy_keepalive', 'jid': '20210607195636776156', 'pid': 7112}
[ERROR ] Unhandled exception running status.proxy_reconnect
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/salt/utils/schedule.py", line 843, in handle_func
ret["return"] = self.functions[func](*args, **kwargs)
File "/usr/lib/python3.6/site-packages/salt/loader.py", line 1235, in call
return self.loader.run(run_func, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/salt/loader.py", line 2268, in run
return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/contextvars/init.py", line 38, in run
return callable(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/salt/loader.py", line 2283, in _run_as
return _func_or_method(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/salt/modules/status.py", line 1793, in proxy_reconnect
if proxyproxy_name + ".get_reboot_active":
File "/usr/lib/python3.6/site-packages/salt/loader_context.py", line 72, in getitem
return self.value()[item]
File "/usr/lib/python3.6/site-packages/salt/loader.py", line 1403, in getitem
func = super().getitem(item)
File "/usr/lib/python3.6/site-packages/salt/utils/lazy.py", line 110, in getitem
return self._dict[key]
KeyError: 'napalm.get_reboot_active'
[DEBUG ] schedule.handle_func: Removing /var/cache/salt/proxy/blr02-eng-core01/proc/20210607195636776156
[DEBUG ] LazyLoaded mine.update
[DEBUG ] Subprocess Thread-98-Schedule-__proxy_keepalive cleaned up
[DEBUG ] Sending global request "[email protected]"
[DEBUG ] Global request denied.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Versions Report
Salt Version:
Salt: 3003

Dependency Versions:
cffi: 1.9.1
cherrypy: unknown
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.0.1
libgit2: Not Installed
M2Crypto: 0.35.2
Mako: Not Installed
msgpack: 0.6.2
msgpack-pure: Not Installed
mysql-python: Not Installed
pycparser: 2.14
pycrypto: Not Installed
pycryptodome: 3.10.1
pygit2: Not Installed
Python: 3.6.8 (default, Nov 16 2020, 16:55:22)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 17.0.0
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.1.4

Salt Extensions:
SSEAPE: 8.4.0+8

System Versions:
dist: centos 7 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-1160.24.1.el7.x86_64
system: Linux
version: CentOS Linux 7 Core

PASTE HERE

Additional context
Add any other context about the problem here.

[welcome]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on Freenode
• Salt Project YouTube channel
• Salt Project Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[OrangeDog]

Can you fix the formatting? Probably needs some code fences: ``` (and N.B. "PASTE HERE"). You may have also deleted the <details> block from the template.

[dmurphy18]

Also missing info

napalm 3.2.0
netmiko 3.4.0

This could be a duplicate of #60061 need to dig further into this before I can tell. Note the 'get_reboot_active' error in the log file has already been addressed, see issue #60025 and PR #60244

[dmurphy18]

Adding more info

proxy:
        proxytype: napalm
        driver: ios
        host: blr02-eng-core01.vmware.com
        username: <username>
        password: <password>

[dmurphy18]

Once the issue of get_connection is fixed, see #60061, the problem appears to be netmiko/base_connection.py failing to properly scan the output data returned by the command 'show tech-support', namely missing the part of the data returned and not seeing the host name in that data.

Investigating further as to why this is occurring, as with debug code, the host name is clearly present in the output data, for example:

20777 ^M
20778 ROM: IOS-XE ROMMON^M
20779 ^M
20780 ip-10-27-57-55 uptime is 6 hours, 35 minutes^M
20781 Uptime for this control processor is 6 hours, 37 minutes^M

but the exception is generated

20751     raise IOError(
20752 OSError: Search pattern never detected in send_command: ip\-10\-27\-57\-55\#

perhaps it is that '#' symbol as part of the search pattern

[dmurphy18]

Problem (with get_connection) fixed appears to occur with netmiko from versions 3.3.0 and later. Netmiko version 3.2.0 has no IOError issues, note: that the command from Salt is 'show tech-support' and the error appears to be in netmiko/base_connection.py, but no recent changes appear there. The problem is finding the prompt for the Cisco router in the output.

Suggest using netmiko version 3.2.0 till netmiko has the problem fixed, similar to ktbyers/netmiko#2272 and ktbyers/netmiko#1949

The search pattern failure is for the enable prompt on the Cisco router.

[dmurphy18]

Closing this and marking it as a duplicate of #60061 with the note about using netmiko v3.2.0