-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Minions running 3002+ throw a JSONDecodeError when called from master #60365
Comments
I tried to debug a little bit, and for some reason it seems in code:
log:
|
On a minion that works, the log looks like this:
|
For me, it looks like it is a problem with any minion running salt 3002+ in azure. |
Hey @sebastianreloaded Can you try setting your locale for your master/minion to UTF-8? |
Oh, actually looking at this a little closer -
Would you be comfortable editing that file? If you add a breakpoint and run your minion manually (with I'm guessing that there will be something extra in that stdout. Might also be some logging in place there and changing to debug logging and inspecting the log may reveal the other output. |
I don't seem to get breakpoints to work. I see the following:
But i get an exception:
I can prettyprint the cmd_ret, but that just looks like what i wrote earlier:
Do i have to configure the salt-minion differently to get pdb working? |
Okay, that's actually kind of interesting, because this line:
expects
May also try
and
I'm wondering if there's something going on with Azure where their sudo is... different 🤔 (seems weird, but at the moment I don't have a better theory) |
Hi, yeah thats what i figured, but i gave up trying to dig into The calls directly on the minion all succeeded. But they don't seem to use the
|
That may be because in some places in Salt we actually just eat stdout. That might be one of those places. Rather than a simple pretty print, try:
If it is actually hitting that point, that command will get the output written to the file for sure. |
I tried logging to file and i tried something into the blue instead:
and then it works. And someting like
BUT by using "use_vt=False" i can't use arguments like "test=True" anymore e.g. I'm not sure what use_vt does exactly yet, except executing the commands differently via |
@waynew do you have any more ideas? |
That's very weird that |
Looking at the code I don't see how that would prevent It's a bit promising that
Might be something different as the preamble, but I'm almost positive that's what's happening and why we aren't seeing it across the board. Basically, |
I think this is definitely a bug within Salt, but I'm not 100% sure where the bug is. Assuming my theory is correct (if you can provide the logging output, that would be 👍 - another place would be in
) If my theory is correct and we're getting extra output, one of the following should be true:
In either case, this is probably a more broad issue than the title indicates, we'll probably create another more broad issue and close this in favor of that 🤔 |
Hi, concerning the cmd_ret["stdout"], this is just empty, as i wrote earlier. |
We have the same issue with Salt Minion 3004 running on Ubuntu 18.04 (AWS). |
I'm seeing the same issue with Salt Minion and Master 3003.2 on REHL 8.4. In my setup I'm running the minion as a non-privileged user using the |
We're seeing the same issue with Salt 3002.6+dfsg1-4+deb11u1 on Debian 11.5. Manually setting |
Description
Minions running with Saltstack 3002.6 or 3003 throw an exception when called from master also running on 3002.6 or 3003.
Setup
Fresh installation of saltstack 3003 on Ubuntu 20.04
Steps to Reproduce the behavior
Expected behavior
Versions Report
master:
minion:
Additional context
When i downgrade the minion on Ubuntu 20.04 to 3001.7 the call from master works just fine.
Also running
sudo salt-call state.highstate
on the minion itself with 3002.6 or 3003 works fine too.Also when using Ubuntu 20.04 on the master and using Ubuntu 18.04 as the minion it only works with 3001.7.
Using Ubuntu 18.04 with 3002.6 as the master and Ubuntu 18.04 with 3002.6 or 3003 as the minion works too.
Using Ubuntu 18.04 with 3002.6 as the master and Ubuntu 20.04 with 3002.6 on the minion doesn't work.
Using Ubuntu 18.04 with 3002.6 as the master and Ubuntu 18.04 with 3002.6 on the minion doesn't work.
I also have some Servers using the same combination without problems.
Is this a locale problem, because the servers it works on are
locale: ISO-8859-1
while the others arelocale: UTF-8
?Even though it doesn't make sense that old version of saltstack on the minion work just fine.
------ Update
Doesn't seem to be the locale.
Working minion:
Not working minion:
The text was updated successfully, but these errors were encountered: