latest versions of armadillo (9.850.1_1) and hdf5 (1.12.0) available from Homebrew do not interact together correctly, result in runtime error #52003
Comments
mikeroberts3000
changed the title
|
Update: I think this issue is due to breaking changes in HDF5 in 1.10 vs 1.12. This excerpt is from the official “MIGRATING FROM HDF5 1.10 TO HDF5 1.12” documentation:
https://portal.hdfgroup.org/display/HDF5/Migrating+from+HDF5+1.10+to+HDF5+1.12 Looking at this documentation page in more detail, I found a simple workaround. I can fix the problem simply by adding -DH5_USE_110_API to the compiler flags in my makefile. Going forward, I'm not sure what the cleanest fix is to this problem. There are several good alternatives. First, I can fix it in my makefile (as described in this post), but this step is currently undocumented in the Armadillo documentation. Second, the Armadillo source code could define H5_USE_110_API before including any of the HDF5 headers, since Armadillo is indeed relying on the 1.10 API. Third, Homebrew could require that the latest version of Armadillo use the 1.10 version of HDF5 as an explicit dependency. Regardless, I'm adding this workaround here it helps anyone that comes across this problem in future. |
|
Could you make a pull request adding that to the Armadillo formula? |
|
I agree that armadillo should use [email protected]. It would be great if a test could be added for this too (in the armadillo formula). Also, this should be reported to the armadillo developers so that they update their code. |
|
@SMillerDev @iMichka : here it is https://gitlab.com/conradsnicta/armadillo-code/-/commit/0166bc036c5c6754402f60b0b2ffeb0914ea2939 Thanks @mikeroberts3000 , for the workaround 🎉 |
|
Hi friends, I don't know how to edit Homebrew formulae correctly, so I'd prefer to avoid doing that task. However, I'm happy to contribute the example code snippet in this thread as a unit test. I have also reached out to the Armadillo developers and referred them to my findings here. @conradsnicta is a lead developer on Armadillo, and his commit mentioned above is targeted specifically at this issue. |
We do rebuild all reverse dependencies from source. We guarantee (or at least we try) that at build time, the brew libraries can be built. Armadillo was rebuilt and tested when we updated hdf5. We also have a minimal "runtime" test: https://github.com/Homebrew/homebrew-core/blob/master/Formula/armadillo.rb#L28-L38 We can not test everything. We guarantee that armadillo itself compiles, we make a minimal runtime test. But we can't possibly test everything. Ideally armadillo has a test suite (I bet it does). In general we hope that upstream has some sort of CI with the whole test suite running, where they can test different hdf5 versions. Runtime behaviour is mostly upstream's responsibility, not the responsibility of the package manager (though we try to help on a best effort basis when something breaks). |
|
If armadillo had a |
We could; but we have tried to avoid to do this too much. These tests are often slow for most of packages, and often drag in another bunch of dependencies. I prefer to rely on upstream's CI capability to test stuff by themselves. |
|
I think there's even an audit to say that should only be done for cryptography software. |
|
Armadillo 9.860.1 has been merged. I strongly encourage someone who is familiar with armadillo to improve the current tests if they wish to help prevent this happening again. It is simply infeasible to suggest that all automated tests are ditched and every feature of every dependent is manually tested. Several hours of effort can go into updating certain formula and it's just not possible to extend that into days/weeks. If you feel strongly against this, please do monitor hdf5 releases and take on the task of creating the hdf5 pull request. |
|
Based on my reading of the automated test that @Bo98 posted, it appears as though my small example code snippet in this thread could be included. Are these automated allowed write to (and read from) the filesystem? Can they link against HDF5, in addition to linking against Armadillo (presumably yes because HDF5 is a dependency of Armadillo)? If so, I think my code snippet could be added to the current automated tests. Again, I'm not familiar with the syntax for authoring Homebrew formulae, so I don't trust myself to do this correctly. But I think testing the most basic HDF5-Armadillo compatibility would fit into the existing automated testing framework, and would not require any additional dependencies. |
|
Yeah, that looks like it might work. I’ll have a look. |
Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.
brew updateand can still reproduce the problem?brew doctor, fixed all issues and can still reproduce the problem?brew gist-logs <formula>(where<formula>is the name of the formula that failed) and included the output link?brew gist-logsdidn't work: ranbrew configandbrew doctorand included their output with your issue?Note that I have chosen not to fix the issues reported by
brew doctorbecause I don't know how, and they seem unrelated to the current issue.Background
Armadillo is a C++ linear algebra library. HDF5 is a C++ library for reading and writing multidimensional arrays. Armadillo optionally depends on HDF5. Armadillo and HDF5 are both available by default on Homebrew. Under normal circumstances, both libraries can be installed by typing
brew install armadilloandbrew install hdf5.What you were trying to do (and why)
I am trying to install Armadillo and HDF5 using Homebrew, so I can build and run a small snippet of example C++ code that uses both libraries. Here is the code I'm trying to run. This code creates a 2D array in memory, saves it to disk, then loads it from disk.
What happened (include command output)
The first step is to install Armadillo.
brew install armadillo
Note that this step installs Armadillo and HDF5, so I should be ready to build my example code.
make
My makefile executed successfully, so now I should be able to run my example code.
./armadillo_experiment_2
Note that I am getting a warning message, and the matrix B has 0 rows and 0 columns, indicating that it was not loaded correctly.
What you expected to happen
I expect that my example code does not print out a warning message. I further expect that matrices A and B report the same number of rows and columns. This would indicate that loading and saving HDF5 files with Armadillo is working correctly.
Step-by-step reproduction instructions (by running
brew installcommands)See "What Happened" above.
Interestingly, I can fix this problem by manually reverting to slightly earlier versions of Armadillo (9.850.1) and HDF5 (1.10.6) in Homebrew.
brew unlink armadillo
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/9620946e004a29744f9c5e4a27800cba6d6d6b9b/Formula/armadillo.rb
brew unlink hdf5
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/7342a4cadf5191d8ae4e83a83e4fbbc3e96d434e/Formula/hdf5.rb
Now that I am on slightly older versions of Armadillo and HDF5, building and running my example code works as expected.
make
./armadillo_experiment_2
Output of brew gist-logs
brew gist-logs armadillo
brew gist-logs hdf5
Output of brew config and brew doctor
Note that I have chosen not to fix the issues reported by
brew doctorbecause I don't know how, and they seem unrelated to the current issue.brew config
brew doctor
Why do I think this a Homebrew bug (or maybe an HDF5 bug), but not an Armadillo bug?
I think this is a Homebrew bug (or maybe an HDF5 bug) because the maintainers of Armadillo have not released a "9.850.1_1" version of their library. This version string comes from Homebrew. In my self-contained reproducible example above, both the non-working and working cases use the same version of the Armadillo source code. Therefore, I do not believe it is possible that this issue is caused by the latest version of the Armadillo source code.
It is possible that this issue is caused by the latest version of the HDF5 source code. But I couldn't test this specific hypothesis because I couldn't test the latest version of Armadillo available in Homebrew (9.850.1_1) with the older version of HDF5 available in Homebrew (1.10.6). Similarly, I couldn't test Armadillo 9.850.1 with HDF5 1.12.0. I got a
dyld: Library not loaded...image not founderror when I tried either of these combinations.Regardless, even if this is a problem with the latest version of the HDF5 source code, it would be preferable for the dependencies in Homebrew to be configured to work around the issue. In other words, it would be preferable for Homebrew to use the previous version of HDF5 with the latest version of Armadillo, if this bug is indeed caused by HDF5.
The text was updated successfully, but these errors were encountered: