[BUG] napalm minion KeyError: 'napalm.get_reboot_active' #60025
Assignees
Labels
Bug
broken, incorrect, or confusing behavior
Confirmed
Salt engineer has confirmed bug/feature - often including a MCVE
lazy-loader
severity-medium
3rd level, incorrect or bad functionality, confusing and lacks a work around
Milestone
Comments
|
Thank you for opening the issue. What is the relevant setup, what is the expected behavior, and what are the exact commands to reproduce the error? These details will help us reproduce the error to fix the issue. |
|
***Whats the issue ? *** How to reproduce The message appears more then once within a minute. ***expected behavior |
|
@network-shark please ensure you have the napalm module installed:
You may need to use "pip". As per these docs, napalm is a requirement, so let's start the troubleshooting there: https://docs.saltproject.io/en/latest/ref/proxy/all/salt.proxy.napalm.html#dependencies |
|
I can't start a napalm minion without installing napalm :) . Thank you , but that is not the problem. |
|
@network-shark OK -- it was worth a shot :) |
|
I think this is related to #59690 -- or at least touches the same code. The In other words, it is not handling the case where |
sagetherage
added
the
severity-medium
|
I moved to 3003 last night and hitting this as well. Makes network proxy minions completely unusable (they don't stay connected). Any chance we can get this in the point release? |
|
Here's output from a Junos minion. No commands run except to start the proxy: |
|
@ggiesen I just fired up the proxy in my lab environment I can't say how this problem behaves on long running minion processes. I got theses failures , but did not have any issues. Are you sure we're talking about the same issue ? From the output it looks like the minion is still running |
|
The minion itself is still running but not connected to the device. Edit: Let me do some further digging to validate a few things Edit 2: Junos connections don't show up in a |
|
So after doing some testing with the deltaproxy in #60177 , and then going back to an unpatched 3003, I can confirm this is a real issue and not merely a comestic one. Start the proxy minion: Proxy minion is connected: Kill the connection: Logs on proxy minion show disconnect: We see user is disconnected on router1: Some tens of seconds later and proxy minion attempts to reconnect and fails: Running Manually run Logs from proxy minion confirm reconnect: Running And we see the connection on the router: In short, if the SSH session gets disconnected, it will not auto-reconnect as it is supposed to (confirmed this works on 3001.7 and 3002.6). To get it to reconnect you must either run |
|
@ggiesen Thanks for that great debug of issue, looks like it is not related to what I originally thought, given it was working and the problem is in re-connect or similar |
|
Just to be clear, the message also appears even proxy minion is started up and connected properly. It just seems to only impact the ability for the proxy minion to reconnect. |
dmurphy18
added
the
Confirmed
|
Duplicated this with Arista EOS too, investigating |
|
Understand the issue and should have a fix next week. |
Merged
3 tasks
|
After applying the patch in #60244, I'm now getting the following: |
|
Note that the regular proxy minion still connects to the device, and will reconnect if the session is killed, but I get those tracebacks at startup. @garethgreenaway With @dmurphy18's patch, deltaproxy still never reconnects if the session is killed. |
|
@ggiesen but you don't see the traceback ad you did before? Or you didn't see that traceback? |
|
To clarify, I'm not seeing any traceback from deltaproxy. Reconnect just doesn't work, even with @garethgreenaway latest commit ( 3a577ca ). With regular proxy minion, reconnect now works, but getting the traceback in #60025 (comment) and periodically also getting: I'm applying the following patches: sudo curl 'https://raw.githubusercontent.com/saltstack/salt/aab00269fa893b3bd099f863914843aff8c49598/salt/loader.py' -o /usr/lib/python3.6/site-packages/salt/loader.py |
|
@ggiesen Looks as if you are encountering an additional problem which was hidden by the original problem (presuming you have applied the changes in #60244 correctly). I suggest that you open a new issue referencing the subsequent issue with the fix from #60244 applied, given the key error is not get_reboot_active Can you open an issue as described above and re-close this, as best practice is to open issues for new items rather than tack them on to existing or re-opened issues. |
|
@ggiesen Closing this, since your issue experienced is not the original issue. Please open a new issue for the problems you have seen as outlined in the comment above. |
|
Could someone please advise how I can get this update? Current status of /usr/lib/python3.9/site-packages/salt/modules/status.py: @dmurphy18 could you advise how to pull this? |
|
@TheBirdsNest You could manually try to apply the patches to your current salt version, but it’s not clear if this will race other issues. @ggiesen tried it and got other errors and gave up on this due to lack of time. I hope we will see these fixes in 3004 |
|
@TheBirdsNest The fix should be available in Salt 3003.1. looking at the code for salt/modules/status.py with tag v3003.1 and Also the fix was merged May 25th, 3003.1 released in mid June |
|
Thanks @dmurphy18 I just checked and its not in our release: |
|
I checked out the tagged release and it was in the copy of the code. Let me check installed code. |
|
I still see those messages on my proxy minions 3002.2 |
|
@TheBirdsNest @network-shark Apologies, I must have fat-fingered the git checkout for tag v3003.2 and got master branch when I thought I had v3003.2. You are correct, the fix will appear in the Silicon release, see a5d9545 Apologies for the confusion. I thought it had made it into 3003.2 but must have just missed the cutoff date. |
|
Just bumped into this issue, I consider it serious enough in network automation cases as the network proxies become unusable as mentioned. My running version: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
I just saw it in my debug log . Maybe it's just cosmetic .
napalm proxy minion connected to cisco csr1000v
The text was updated successfully, but these errors were encountered: