-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
System.DllNotFoundException: Unable to load shared library 'lmdb' or one of its dependencies #5174
Comments
We just received another bug report via our paid support channel about this issue. This appears to be a regression in our HOCON configuration that didn't get caught because we have the https://www.nuget.org/packages/Akka.DistributedData.LightningDB/ package installed in our test suites. You can work around this issue, in the short run, by just installing that package. However, we're working on an emergency patch to roll this back as it represents a breaking change. We're also going to add a regression test to block for this in the future. |
Actually, it looks like we are referencing it: https://www.nuget.org/packages/Akka.Cluster.Sharding/1.4.22 |
@fscavo is this on Linux or Windows? |
This is on Linux and the docker image is mcr.microsoft.com/dotnet/aspnet:5.0.7-focal |
Could be related to CoreyKaylor/Lightning.NET#141 |
I’ve always installed lmdb libraries via apt-get since I started using ddata. I always assumed that was a prerequisite |
Looks like this was even previously reported and fixed CoreyKaylor/Lightning.NET#133 Not sure why this is happening now with v1.4.22. |
Workaround@fscavo For a workaround, if you're not using durable ddata, you can set |
Looks like none of this is the case. Just spot checked all of our HOCON configuration changes, library updates, and settings class changes. All of them appear to be ok - wouldn't have effected this in any meaningful way. |
This should stop LMDB from loading automatically for users who don't want to use it intentionally #5175 |
@fscavo we'll publish a fix to resolve this issue for users upgrading, but I want to keep this issue open until CoreyKaylor/Lightning.NET#141 is resolved as that will be a footgun for users in perpetuity until a new version is pushed. |
I'm amazed you didn't have this issue on v1.4.21 though - the offending HOCON was added back then. |
The previous version was running with the base image mcr.microsoft.com/dotnet/aspnet:3.1-focal. Could it be possible that aspnet:3.1-focal had liblmdb-dev installed by default? |
@fscavo I was just able to reproduce your error using https://github.com/petabridge/akkadotnet-cluster-workshop, but I had to set The issue does not occur when I'm going to try with v1.4.21 and see if that comes up. |
Version of the SDK we're using: https://github.com/petabridge/akkadotnet-cluster-workshop/blob/dev/src/Akka.CQRS.Pricing.Service/Dockerfile#L1 |
Yeah I'm definitely hitting this problem on v1.4.21 as well. It's possible that you're only hitting it now because of the underlying changes in the base image. |
Decided on a different approach for fixing this issue and reverted our planned PR. Stay tuned. |
I found out that the previous version of Akka was 1.14.20 and not 1.14.21. The other services run Akka 1.14.21 with no issue, they use |
No problem, this change was introduced in v1.4.21: akka.net/src/contrib/cluster/Akka.Cluster.Sharding/reference.conf Lines 184 to 195 in 7997b58
This is what's causing the system to attempt to load LMDB |
only until CoreyKaylor/Lightning.NET#141 gets resolved close akkadotnet#5174
Have a solution pending here: #5180 We're going to put this up into a nightly package after it's merged. I just verified a few moments ago that this ran fine inside https://github.com/petabridge/akkadotnet-cluster-workshop without having to install any custom packages inside our This is called a "vendored" package approach; it's fairly common in the JVM ecosystem; and it's, IMHO, a bit of a hack so I don't want this to be a "forever" thing. I've made our LightningDB.Vendored.Akka package hidden on NuGet because I'd rather have everyone in the ecosystem use the original, but I always don't want to leave our own users in the lurch while we wait for an official release. I will be very happy to drop our vendored version of the package and switch back to the original as soon as it's possible. I'll post an update on this thread when there is a nightly available - please give that a try @fscavo and let us know if that runs in your environment. If it does, we'll push v1.4.23 on Monday. |
only until CoreyKaylor/Lightning.NET#141 gets resolved close #5174
I ran |
I've been able to validate in local Docker builds that #5180 resolved the issue, but in post-mortem analysis with @Arkatufus this morning we determined that this issue really slipped through our testing matrix because of #3573 and we will need to add MNTR support to Linux going forward to catch these types of issues. I've begun preliminary work on that here #5182 but it will probably take a while to get right. |
Was able to validate that Windows still works with the latest nightlies here as well: https://github.com/Aaronontheweb/Akka.Cluster.Sharding.DDataDemo Going to open a separate issue for this, but that code sample definitely exposed some bottlenecks with |
Ok, pushing Akka.NET v1.4.23 live with this fix included. Validated inside Docker, Ubuntu WSL, and Windows desktop. |
Version Information
Version of Akka.NET?
1.14.22
Which Akka.NET Modules?
Akka.Cluster
Akka.Cluster.Sharding
Describe the bug
The service used to work fine with the version 1.14.21. After having updated to 1.14.22 the service stopped working and kept throwing System.DllNotFoundException with the following details:
Unable to load shared library 'lmdb' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblmdb: cannot open shared object file: No such file or directory
The text was updated successfully, but these errors were encountered: