Problem upgrading clients from 5.7.0 to 5.8.0; getting "Could not find platform dependent libraries <exec_prefix>"

[fseesink]

Basically been using Munkireport for some time. But with 5.8.0 and the addition of Python3 to the mix, it seems something is off. Of course, the idea is that you can update all your clients using Munki. But it's not quite that simple.

With the new release, for my own box when testing, I installed it all manually, and that worked. So figured it was time to package it all up and update all the other clients. So then added the Mac Admins Python as a package into the repo, added in the new Munkireport package setting it to depend on the Mac Admins Python, config'd the necessary bits and pushed it out. But wasn't getting anything from any of the clients, as in they were no longer checking in. As days turned to weeks, I realized I wasn't getting the correct info anymore.

I finally sat in front of one of the clients and tried to figure out what was going on. What I found was interesting. I opened Terminal and went digging. The client had the necessary packages installed. So Munki did its job.

Found the Munkireport files down under

/usr/local/munkireport/

notably

• munkireport-python2
• munkireport-python3

If I ran munkireport-python2, it dropped me into the REPL with no issue.
However, if I ran munkireport-python3, it shows the following at the top before dropping me into the REPL:

Could not find platform dependent libraries <exec_prefix>

So it seems the new version isn't running properly for some reason. When I look online for issues that trigger this message, there's some indication that it has to do with virtual environs (venvs). But I haven't found specifics to explain the "why" of it, let alone how to fix it.

I would rather not have to go to each client and have to manually install this. But something clearly didn't go right. I just can't figure out what it is.

If you have any thoughts/suggestions, please let me know. But at least for me, upgrading from 5.7.0 to 5.8.0 has been... sub-optimal.

[tuxudo]

What version of the Mac Admins Python are you using? MunkiReport only supports 3.9 and 3.10, it does not support 3.11+.

[fseesink]

Ah. That might explain it. Had used the dataJAR Munki recipe, and apparently that's pulling down 3.12.1 at this point.

Any particular reason Munkireport is only supporting 3.9 and 3.10? I mean the test machines that I manually installed Munkireport 5.8.0 on all work fine, checking in, etc. And they're all showing 3.12.1 at this point. Is it just that there's something off with installing? Seems a bit arbitrary.

[tuxudo]

There is a bug with the upstream Python 3.11+ packages that causes MunkiReport to not work with Python 3.11+ when it's symlinked as MunkiReport does.

Please verify the version of Python on the clients that are reporting in by running /usr/local/munkireport/munkireport-python3 --version and then running a manual MunkiReport run with sudo /usr/local/munkireport/munkireport-runner to make sure it is not erroring out.

[fseesink]

That is odd. Because as I mentioned, on my test systems where I manually installed, they're all working fine. They're checking in, etc.

However, using what you mention above and doing

sudo /usr/local/munkireport/munkireport-runner

while the output shows it working, I DO see that it does kick up a Python crash at one point. Looks like in managedinstalls.py there's a warning returned, and networkinfo.py triggers a popup. But it still runs to completion.

$ sudo /usr/local/munkireport/munkireport-runner
Password:
    ## Starting MunkiReport run
    # Executing scripts in scripts
    Running applications.py
    Running disk_info
    Running filevaultstatus
    Running firewall.py
    0
    Running managedinstalls.py
WARNING: managedinstalls.py return code: -11
    Running munkireport.py
    Running networkinfo.py
    networkinfo.py Error: /usr/local/munkireport/scripts/networkinfo.py:192: SyntaxWarning: invalid escape sequence '\('
  network["activemtu"] = re.sub('[^0-9]','', re.sub("[\(\[].*?[\)\]]", "", mtudata))
/usr/local/munkireport/scripts/networkinfo.py:251: SyntaxWarning: invalid escape sequence '\)'
  bond['activemedia'] = re.sub('\)','', re.sub('\(','left_para', bond_line).split('left_para')[1]) # tbase
/usr/local/munkireport/scripts/networkinfo.py:251: SyntaxWarning: invalid escape sequence '\('
  bond['activemedia'] = re.sub('\)','', re.sub('\(','left_para', bond_line).split('left_para')[1]) # tbase
    Running power
    power Error: /usr/local/munkireport/scripts/power:397: SyntaxWarning: invalid escape sequence '\('
  powerinfo['sleep_prevented_by'] = re.search('\(.*\)', item).group(0).replace("(sleep prevented by ", "").replace(")", "").strip()
    Running printer.py
    Running security.py
    Running supported_os
    Running timemachine
    Running wifi
    Requesting timemachine
    Requesting filevault_status
    Requesting managedinstalls
    Requesting munkireport
    Requesting firewall
    Requesting wifi
    Requesting power
    Requesting disk_report
    Requesting security
    Requesting supported_os
    Requesting network
    Requesting applications
    Requesting printer
    Authentication challenge for Host: <MYFQDN_HERE> Realm: None AuthMethod: NSURLAuthenticationMethodServerTrust
    Allowing OS to handle authentication request
    Need to update reportdata (466B)
    Need to update munkireport (7KB)
    Need to update power (2KB)
    Need to update disk_report (2KB)
    Need to update applications (269KB)
    Need to update printer (2KB)
    Sending items (281KB)
    Server info: starting: reportdata
    Server info: starting: munkireport
    Server info: starting: power
    Server info: starting: disk_report
    Server info: starting: applications
    Server info: starting: printer
    ## Finished run

My bigger issue is simply deploying the right version of Python3 just for MunkiReport I guess. The documentation here says to use Python v3.10.9.80716 specifically. However, that was released back in Jan 2023, and there have been several updates to Python 3.10 since. The latest in that repo, for example, is Python v3.10.11.80742 released in March 2024. At the very least, you shouldn't advocate folks use an outdated build. Updates within versions are often due to security issues if not bug fixes.

I have been using Munki/MunkiReport with no issues for probably close to a decade by now (I've lost track). So I had hoped to make the transition from 5.7.0 to 5.8.0 using the existing setup. You know, push out the Macadmins Python3 package to clients along with the 5.8.0 release. Now I leverage AutoPkgr, and within that, I searched/found this dataJAR Munki recipe which I overrode and setup, figuring this should work. But it is pulling the latest stable release, which is 3.12.x. (Didn't realize this was an issue 'til reporting this.)

So guess I have to find another way to setup a recipe to deploy this Macadmins Python 3.10.x business. This is, frankly, one thing I don't care for in this setup: the dependency hell. Considering that Munki itself is written in Python and requires Python3 these days (and "Starting with Munki 4, Munki includes its own Python interpreter with all required dependencies" - from here), I honestly do no understand why you feel the need to use yet another Python package, instead of simply using the one Munki itself uses? Seems counterproductive and overly complex. (The ironic part here being that if I can get this particular situation sorted, I can easily deploy it to existing Munki clients BECAUSE every Munki client already HAS Python installed on it. That's what Munki itself is using.)

At the very least, since the entire point of this project is for clients to report on a Munki client setup, it would make sense to provide a recipe or instructions on how to deploy to those clients the Python package this project seems to now rely upon. This is especially true for anyone who has been using MunkiReport for awhile. This shift to Python3, and this specific Python3 setup, only occurred with the latest version (5.8.0). So this transition has been anything but painless.

Mind you, first and foremost, I really appreciate all the work that's been done. I fully understand that this is all a labor of love. And that you all contribute your time, sweat, and tears and share this with others is very much appreciated. But the questions I ask are simply because it seems like you have taken on a more complicated design than maybe was necessary?

Anyway, I provide the above in case it is of any use. For now I'll have to go figure out how to write a recipe to get specifically what I guess is needed.

[tuxudo]

Please verify what version of Python 3 by running /usr/local/munkireport/munkireport-python3 --version. Having just tried MunkiReport with Mac Admins Python 3.12.1, it failed as expected.

You can use any 3.10 version of the Mac Admins Python release. You linked to the release notes of MunkiReport 5.8, which at the time v3.10.9.80716 was the latest version. In the documentation here, https://github.com/munkireport/munkireport-php/wiki/Client-setup, it lists v3.10.11.80742 as the version to use.

We do not use the version of Python included with Munki because we do not want to be dependent on a Munki installation and the version of Python included in Munki is built specifically for Munki and may not include all the needed things MunkiReport uses. It is also incompatible because of the symlink bug from upstream Python with versions newer than v3.10.

Overall, the transition from 5.7 to 5.8 is very similar to the 5.6 to 5.7 transition. You need to install the required version of Python, MunkiReport's Python 2.7 in the case of MunkiReport 5.7, on to the clients before upgrading MunkiReport. The level of dependencies has not changed from 5.7 to 5.8, if anything it has gone down as we no longer require Rosetta 2.

There was lots of discussion on how to best run MunkiReport with Python 3. The ultimate deciding factor was to have MunkiReport use the Mac Admins Python package and not support other Pythons so as to reduce troubleshooting, testing, and maintenance overhead. We are after all a small handful of unpaid volunteers running this project.

Not everything can be fully documented for an open source project run by unpaid volunteers, but we always welcome new and updated documentation to be submitted. For deploying the needed version of Python, I would suggest manually downloading and then importing into your Munki the required version of Python. Perhaps with a unique name so that it doesn't get trampled over by another AutoPKG recipe.

[fseesink]

Please verify what version of Python 3 by running /usr/local/munkireport/munkireport-python3 --version. Having just tried MunkiReport with Mac Admins Python 3.12.1, it failed as expected.

3.12.1

It should be noted that I do have Python v3.12.5 installed (using the Python.org installer) on the system as well, as I do a decent bit of work using Python. So my money is on that if there's a symlinking/path type issue, it's potentially finding its way to that. But doing the command you specified, it's 3.12.1 as expected. And the symlink references the Macadmins Python install path.

We do not use the version of Python included with Munki because we do not want to be dependent on a Munki installation and the version of Python included in Munki is built specifically for Munki and may not include all the needed things MunkiReport uses.

Sorry to be a bit direct here, but what's the point of Munkireport if Munki isn't installed? And what could Munkireport need in Python that Munki wouldn't include? Is it THAT unique?

It is also incompatible because of the symlink bug from upstream Python with versions newer than v3.10.

What exactly is this bug you are referring to? Do you have a bug reference # or anything? I can't seem to find specifics on what this is. What little I can find via Google is anecdotal at best, and points if anything to issues with how folks are using Python, not Python itself. Not being contrarian. Just truly trying to understand. I do know that bugs have crept into Python at times. In our dev group we've hit on a few things that required backing off 'til at some point later the issue wasn't there. But this one's new to me.

Overall, the transition from 5.7 to 5.8 is very similar to the 5.6 to 5.7 transition...

I would beg to differ with this. When going from 5.6 to 5.7, we simply needed to

1. add a package in Munki to push out Python v2 to each client
2. the steps recommended having the Munkireport recipe "depend on" that version of Python package. This way if not installed, it would be installed before updating to 5.7.

NOW this presents a problem. You can't simply do a managed uninstall of the Python v2 package (which honestly should've been gone a LONG time ago) without first uninstalling Munkireport itself, as the latter depends on the former, which prevents the former from being removed. That means having to do a managed uninstall of both the Python v2 and Munkireport packages, and only once all clients have done so then installing the new Python v3 package with the 5.8 Munkireport.

Only... how do you know when all the clients have done so? You know by checking MunkiRepo... oooooooh. See the dilemma?

...The ultimate deciding factor was to have MunkiReport use the Mac Admins Python package and not support other Pythons so as to reduce troubleshooting, testing, and maintenance overhead.

The only "other Pythons" to consider would be the one that Munki itself includes/uses. And frankly, I still don't see how what you've done "reduces[s] troubleshooting, testing, and maintenance overhead" if I'm being honest. Making your code depend on the Python that Munki itself already requires/uses would've made it far simpler than what you've done. But as you say you're unpaid volunteers and it's your project. And maybe there's some library/module that you require in MunkiReport that Munki doesn't. Would strike me as unlikely, though. But ¯_(ツ)_/¯

At this point, I've gone and configured a managed uninstall of MunkiReport, the Python v2 package (since it's no longer used anyway), and the Macadmins Python 3.12.1 package. But I'm blind to how many devices have checked in/done so and will just have to wait a day or two before then adding the Macadmins Python 3.10.x package along with Munkireport to the managed installs again.

It is not nearly as seamless a transition as I remember 5.6 to 5.7 being. But I'll get this done and hopefully that'll be the end of it.

(Side note: It does, however, simply reinforce something I have been saying for some time to folks regarding Python vs. Go. I truly love working in Go as there's no dealing with external dependencies like this. You build an app, you deploy it. And "it just runs." No need for "Did you install this interpreter with this version? Do you have these packages/versions installed?" etc. And this is one example where it would be a non-issue if written in Go. But that's much easier said than done, as it would be a total rewrite.)

...For deploying the needed version of Python, I would suggest manually downloading and then importing into your Munki the required version of Python.

Regarding this, I went ahead and overrode the download recipe that the Macadmins Python munki recipe relied on. I tweaked it to specifically pull down the latest 3.10.x installer (vs. simply the latest). And that's what I'll be deploying in a day or two once I think all the client devices have done managed uninstalls of the other stuff.

[flammable]

3.12.1

MunkiReport requires MacAdmins Python 3.10. If you roll that back, things should work as expected.

Here's the output you should receive:

% /usr/local/munkireport/munkireport-python3 --version
Python 3.10.9

If it's not that, you're running the wrong version.

Sorry to be a bit direct here, but what's the point of Munkireport if Munki isn't installed?

Despite the name, Munki is no longer necessary to use MunkiReport. Here's an example:

https://github.com/munkireport/munkireport-php/wiki/Using-MunkiReport-with-Jamf

You might have missed the conversation in the #munkireport and #munkireport-dev Slack channels. The decision to use MacAdmins Python stemmed from not forcing people to install Munki if they didn't need it. Additionally, if issues were to arise, we didn't want people complaining to Greg that MunkiReport no longer works. Munki's Python is solely guaranteed to work for Munki.

Munki's Python would be incompatible with MunkiReport anyway, since Greg has moved on to Python 3.12. If you're good with Python and willing to contribute fixes to allow MunkiReport to work with Python 3.12, please submit pull requests!

Also, note that there's a swift-cli branch that's actively being worked on. It's possible Munki won't include Python after some point.

You can't simply do a managed uninstall of the Python v2 package (which honestly should've been gone a LONG time ago) without first uninstalling Munkireport itself, as the latter depends on the former, which prevents the former from being removed.

There's a built-in command for removing Python 2:

/usr/local/munkireport/munkireport-runner --remove-python2

You can run that as a postinstall script for your new MunkiReport 5.8 client pkg, or via any other method you have for running scripts.

For downgrading MacAdmins Python from 3.12 to 3.10, just create an installs array for 3.10 and remove 3.12 from any production catalogs.

[tuxudo]

MunkiReport does not do any path finding for its Python 3. It uses a hard coded symlink to point from /usr/local/munkireport/munkireport-python3 to /Library/ManagedFrameworks/Python/Python3.framework/Versions/3.10/bin/python3.10. As above, this was discussed in the #munkireport and #munkireport-dev channels on the MacAdmins Slack in early 2022 and again in early 2023 and determined to be the best fit. If your MunkiReport installation does not point to the Mac Admins Python version 3.10, you are using an unsupported modification to MunkiReport. If your modified MunkiReport set up is indeed using Python 3.12 and working without issue, we ask that you look into how it came to be and once reproducible please document how it was done or create a pull request.

Of the roughly 97 modules in MunkiReport's Module Marketplace, only about 5 or so are for Munki. There are over 90 reasons in the form of modules to use MunkiReport without Munki. Greg Neagle, the developer of Munki, has stated that the included Python is built to minimally support only Munki and has recommended that other projects not use it. We further do not want to burden him with having to support MunkiReport's use case with Munki's Python.

The symlink bug with Python 3.11+ can be found here: gregneagle/relocatable-python#31

Overall, I consider this issue closed as the issue with MunkiReport not working on your clients is the incorrect version of Python.

[kevinmcox]

I honestly do no understand why you feel the need to use yet another Python package, instead of simply using the one Munki itself uses? Seems counterproductive and overly complex.

One reason this decision was made is because the developer of Munki has long advised not to use Munki Python for anything else.

Yesterday it was announced that Munki Python is deprecated and will be removed in a future release:
https://github.com/munki/munki/wiki/Deprecation-Notes

Bundled Python: a future release of Munki will remove all dependency on Python and will no longer include a bundled version of Python.