libgcc dependency hierarchy.

[msarahan]

@groutr commented on Mon May 02 2016

Each library in libgcc has it's own packages that track the .so version of that library and the ABI version of that library.
The shared object package contains the .so file and depends on its ABI package.
The ABI packages depend on libgcc metapackage. This way hierarchy guarentees a minimum version of ABI.

---

@groutr commented on Mon May 02 2016

@insertinterestingnamehere @msarahan @mcg1969 @mingwandroid
Please look this over and verify that these dependency relations are correct.

---

@msarahan commented on Mon May 02 2016

Nice work. I think this will be a much better approach. Please document things a bit more, as in the line comments - just for posterity's sake.

---

@groutr commented on Wed May 04 2016

Good point. I'll put in the comments in.

---

@insertinterestingnamehere commented on Wed May 04 2016

If I'm understanding this correctly, to use libstdc++ (for example) a third party package would have to specify:

• libstdc++ = 6
• libgcc >= 5.2

The constraint that each of these packages needs libgcc >= 5.2 seems odd to me. I've never heard of libstdc++ or libgfortran being used without the libgcc it was shipped with. That might work, but I don't know what backwards compatibility constraints apply there.

That's interesting about the version numbers stored in the shared objects. I wonder where they come from and what they do. Given that these numbers come from the shared objects themselves, why aren't we using equality constraints to specify the packages' dependencies on them?

---

@insertinterestingnamehere commented on Wed May 04 2016

Small clarification on the last comment when I say "version numbers stored in the shared objects" I mean the ones read by patchelf and specified as version numbers for the ABI metapackages, not the shared object versions themselves.

---

@msarahan commented on Wed May 04 2016

equality is far too restrictive. The behavior is well defined as far as backwards and forwards compatibility, so that's what we need to be using.

you might want libstdc++ >=6,<7, but ==6 is too tight.

---

@insertinterestingnamehere commented on Wed May 04 2016

I'm fairly sure that the Linux dynamic loader refuses to load a given shared object if the version specified as a dependency for a library isn't available. That's why we were getting all those load failures complaining about libgfortran 1.0 not being present even though 3.0 was there.

---

@msarahan commented on Wed May 04 2016

so 1.0 and 3.0 are not compatible. but 3.0 with 1_6 symbols is backwards compatible with something that has, say, 1_4 symbols. This is what the people in #686 are complaining about. They have built things with a newer version of fortran (but still API version 3), and Ilan's ancient libgfortran is incompatible. This is similar to what we'll see with libstdc++ (though the version may still be version 6) if we don't keep on top of updating that with the compiler. There, they are better about using the bugfix number as the inner symbol version compatibility number, though.

---

@insertinterestingnamehere commented on Wed May 04 2016

Wow that's confusing. Okay, so a revised version spec would be:

• libstdc++ >= 6
• libstdc++-abi >= 3.4
• libgcc >= 5.2

In that case how would we be protecting against failed loads resulting from further ABI updates?

---

@msarahan commented on Wed May 04 2016

as long as the ABI/symbol version (not sure about exact term) is newer than what a library wants, you're OK. If you dump the symbols, you'll see the list of all the ones supported: http://stackoverflow.com/a/10355215/1170370

Expressed more clearly:

• glibc is forwards compatible. Build on the oldest one you can tolerate. Do not staticallly link, though.
• everything else (as far as I can tell): mostly backwards compatible. major versions excepted.

---

@insertinterestingnamehere commented on Thu May 05 2016

Right, I had hoped that we'd be able to find a way to use the extra version information to avoid having shared objects be rejected because their ABIs are too old. This has me wondering if that's even possible at this point. Fortunately, these packages all have fairly strong backcompat guarantees, so issues with incorrect ABIs will be rare.

---

@groutr commented on Tue May 10 2016

Any feedback, @ilanschnell? Does this look good to you?

---

@ilanschnell commented on Tue May 10 2016

So what is the difference between libgfortran and libgfotran-abi?

---

@groutr commented on Tue May 10 2016

libgfortran-abi is a meta package that tracks the abi version of libgfortran. libgfortran is a package that contains the shared library.

---

@ilanschnell commented on Tue May 10 2016

Ok, but why do we need this extra meta-package?

---

@msarahan commented on Tue May 10 2016

This is the best example I know of: #686

This person has some software (maybe scipy) that broke when you updated libgfortran. The reason it is broken is that although you added the right libgfortran API version (3), you did not have the right ABI version for whatever software this person is running. This can happen when something that uses a library is compiled with a newer compiler than the library on the end-user system - perhaps they downloaded a package from anaconda.org that was built with a newer version of gfortran. By putting our libgfortran first on the search path, issues like this can happen.

Tracking ABI version dependencies is a very good improvement for us to make. The other side to this PR is that we need to track ABI information when we build packages, and store it in the metadata, so that we understand the full requirements (not just version/API requirements) of packages.

---

@ilanschnell commented on Tue May 10 2016

I don't have time to look at this closer right now, but I have the feeling that this PR adds a lot of complexity, which we should try to avoid.

---

@mcg1969 commented on Tue May 10 2016

The complexity is there already. The question is how it is managed/handled.