LLVM supported versions

[jkthorne]

I thought it might be worth an issue to discuss the supported versions of LLVM.
This issue comes from the discussion in this #8538.

Why?

Support for older versions creates more maintenance in code, and we can miss out on taking advantage of new features in LLVM.

Why Not?

Not supporting older versions can make support of deployments and platforms harder.

Currently

LLDB support is being added and is using features that are outside our LLVM support window (3.x)

Proposal 1

Crystal could maintain a soft 3-year window LLVM support.

After some digging around on the llvm site, it seems like they have rough support of 3 years for their toolchain.
http://llvm.org/docs/DeveloperPolicy.html#updating-toolchain-requirements

Note: This is not LLVM supported versions, just their toolchain.

Proposal 2

We could follow the Ubuntu LTS LLVM package.
This is a widely deployed target and would make deployment easier.
This was in the above PR #8538 (comment).

Proposal 3

We could update LLVM whenever we have a feature or blocker that would benefit the project.
This is more like Rust, which is discussed here, which updates more based on need. This would enable us to be more liberal in implementing features like LLDB.

[jkthorne]

I personally lean to if it is not a limiting factor on deployment targets we should pick the version the best supports the development of Crystal. So if Ubuntu or other major targets can install X.Y version of LLVM it should be a candidate for our minimum version. In this way if we wanted to update our minimum version of LLVM to 7.x for LLDB support that would be fine.

[Sija]

Is using llvm-7 and llvm-8 packages (or alike) out of scope here?

[jkthorne]

I think we are mostly talking about LLVM versions and less specific packages. But on LTS I think those would be the packages we would be depending on.

[straight-shoota]

LLVM is only used for the compiler. The library is usually statically linked, so it doesn't matter which versions are available on user's platforms. It only matters for compiler developers and the compiler toolchains. The lower bound is currently the runner on travis CI which is based on Ubuntu 14.04 (ships with llvm 3.4, but up to 3.9 is available). I suppose the runner could probably be upgraded though. The Docker images (used by distribution-scripts for building the compiler) is based on Ubuntu 18.04 which ships with llvm-8.

[jkthorne]

In that case I would be good with going to llvm-8

[skuznetsov]

LLVM is only used for the compiler. The library is usually statically linked, so it doesn't matter which versions are available on user's platforms.

In this case we can remove previous quite outdated support for 3.x versions then, which will make code more neater..

[skuznetsov]

Also Ubuntu 19.10 already using llvm-9 packages. I am using it on my AWS test Linux instance to test compiler changes.

[ysbaddaden]

The availability of recent packages for LLVM ain't much of a problem, except for old linux distributions, such as Raspbian Stretch or Centos 6 (EOL 2020-11-30).

Distribution provided packages:

• Ubuntu 16.04: LLVM 6 (xenial-updates);
• Ubuntu 18.04: LLVM 9 (bionic-updates);
• Debian Jessie (oldoldstable): LLVM 6;
• Debian Stretch (oldstable): LLVM 7;
• Debian Buster (stable): LLVM 8 (buster-backports);
• Raspbian Stretch: LLVM 4;
• Raspbian Buster: LLVM 8;
• Centos 6 : LLVM 3.6;
• Centos 7 : LLVM 7.

The LLVM APT repository has updated packages for i386 and amd64:

• Ubuntu 14.04 (EOL): up to LLVM 8 ;
• Ubuntu 16.04, 18.04: up to LLVM 11 ;
• Debian Jessie: up to LLVM 8.
• Debian Stretch, Buster: up to LLVM 11.

[ysbaddaden]

I don't think sticking to a fixed rule such as "we support the last N releases of LLVM" is a good idea. It's IMO better to see which versions are widely available, and which LLVM versions are providing what we need, and the cost of maintaining support for older releases. The LLVM C API isn't breaking that much anymore, and most of our C++ extensions is now available in the official C API since LLVM 7 (DIBuilder, Atomics).

• From the availability of packages, there are zero reasons to still support LLVM 3.x.

• Keeping anything below LLVM 6 feels useless. Only Centos 6 would be affected and it's nearing EOL. Sadly, Raspbian jessie/stretch are lagging behind Debian, but Buster has LLVM 8, and older releases can try the Debian armhf packages (I did that once).

• LLVM 7 introduces an official C API for most of what our C++ extension provides (DIBuilder, Atomics).

IMO: I see little reason to support anything below LLVM 7.

[skuznetsov]

@ysbaddaden so shall we do the clean up then?

The only drawback is here that we also expose llvm module in our standard library so if someone will need 3.x support in their code they will be out of luck.

[bcardiff]

Currently, the build process stresses only uses the two latest LLVM versions. Usually darwin build are bumped before linux to ensure homebrew releases will work with updated llvm formula.

There are no strick guarantees on what LLVM versions are supported other than that. Even if there is code around that.

An overdue in the CI is to stress different explicit versions of binaries dependencies like LLVM and others (libz, libyaml, openssl, etc). Without something like that at some extent there is no hard guarantee.

Regarding a lower bound, I would not limit it by features like debug support. And newer llvm consume more resources while compiling crystal code. So that might be something to have in mind.

[skuznetsov]

@bcardiff When I compile crystal on my computer it takes about 1.2 GB so it is still possible to compile on Raspberry Pi 4 2GB and up. Then that code can be used on on all other RPi's.

[RX14]

We should provide support for what the distros can compile with. There's no sense at all in only supporting LLVM versions that are too new for debian and missing out on getting an official package.

Old centos is, however, a lost cause.

[refi64]

CentOS/RHEL 7 has a newer LLVM (8 I think?) in the software collections, and 8 has a newer one as part of the app streams.

…

On Wed, Feb 5, 2020, 6:59 PM Stephanie Hobbs ***@***.***> wrote: We should provide support for what the distros can compile with. There's no sense at all in only supporting LLVM versions that are too new for debian and missing out on getting an official package. Centos is, however, a lost cause. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#8725?email_source=notifications&email_token=AAM4YSJ3EKQVWPRBLQOYEKTRBNOGZA5CNFSM4KN4GVMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK5RIZA#issuecomment-582685796>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAM4YSM5LDCICQ3HWPEFWYTRBNOGZANCNFSM4KN4GVMA> .

[deepj]

I have a question. Why Crystal doesn't bundle own LLVM toolchain with possibilities to disable/remove own one, and use system LLVM toolchain by own choice if needed? My understating is, Crystal still needs LLVM for compilation of Crystal programs.

I have a problem with Installation from Homebrew on Mac when with recent version of Crystal is huge large LLVM 10 and it breaks compilation of some gems and Ruby compilation through ruby-build. That's maybe a problem on my settings (I use all in default state).

TruffleRuby contains own built-in LLVM Toolchain based on LLVM 9 in the recent version (https://github.com/oracle/truffleruby/releases). The toolchain has ~170 MB after decompression. Compressed build of TruffleRuby 20.0.0 for Mac has only 110 MB. TruffleRuby bypasses paths while a gem compilation and doesn't use anything else outside its own toolchain.

I guess built-in LLVM toolchain would solve problems if any given distribution does or doesn't support LLVM X.X

[RX14]

@deepj I think that's more of a homebrew problem, you might be able to fix it by installing multiple versions of LLVM.

Bundling our own LLVM toolchain would be a lot of work, if we wanted to do more than static linking (which wouldn't help macos anyway since it doesn't support it). If it becomes a larger problem I think we'll explore it, but I'll always err on the side of doing the normal thing: using the system libraries.

[deepj]

@RX14 What I know Sulong team patches their LLVM toolchain minimally. They uses regular LLVM releases and just apply these patches into that.

https://github.com/oracle/graal/tree/master/sulong/patches
https://github.com/oracle/graal/blob/master/sulong/docs/TOOLCHAIN.md#graalvm-deployment

In TruffleRuby there are some issues with static linking rather then with dynamic linking currently (but it's a problem of TruffleRuby itself). So I guess with dynamic linking would not be a problem much.

[bcardiff]

@deepj the targz and pkg packages for darwin in github releases have a built-in llvm 6.0.1. As long as you compile crystal programs, but not the compiler, it should be enough.

brew formula requires linking against llvm formula.

Eventually we will build another llvm package to bump from 6.0.1 to whatever recent release of llvm is present at that time.

[j8r]

CentOS 6 is supported until 2020-11-30 .
I suggest at this date, or even sooner (as people wish), to drop support anything below LLVM7, as @ysbaddaden said, and bump the CI accordingly (and the brew package too apparently).

• Debian 7 Jessie support ends in June 30, 2020
• Debian 8 Stretch has LLVM 7
• Ubuntu 16.04 has LLVM8 with the in updates repositories
• CentOS 7 has LLVM 7

I suppose, supporting the most recent LLVM versions available on the oldest supported CentOS, Debian, Ubuntu at the minimum is the policy? (if yes, can it be written somewhere?)

[straight-shoota]

I think by now we can safely remove support for LLVM < 7, or maybe < 8 as suggested in #11329.

[Xosmond]

Hello, guys. I have installed crystal 1.2.1 but LLVM still showing as 10.0.0 when I have LLVM 13.0.0 installed locally. is there any option to make crystal use my local LLVM?

[straight-shoota]

LLVM is statically linked into the compiler. It doesn't choose the version at runtime.

You need to rebuild the compiler in order to use a differen LLVM version (grab this repo and run make crystal).

[straight-shoota]

As @rdp mentioned in #11403, the oldest LLVM versions are actually not supported anymore. Since nobody reported that yet, I think it's safe to drop them.

llvm_ext fails to build with LLVM 3.8, probably since #9562 (can't find llvm/Support/CodeGenCWrappers.h).
Linking the compiler with LLVM 6.0.1 fails with a lot of undefined references. I didn't dig into it.
LLVM 7.0.0 still works.

#11343 adds a CI job to test with the latest LLVM release (13.0.0). We should add a similar one for the oldest one. We could consider adding test for all supported versions, but that would definitely only need to run for maintenance release workflows.

[HertzDevil]

If there is a CI matrix for the different supported LLVM versions it would also be a good chance to run the rests in spec/llvm-ir.

[rdp]

Thank you!

On Thursday, January 19, 2023, Johannes Müller ***@***.***> wrote: Closed #8725 as completed via #12906. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.<

https://ci3.googleusercontent.com/proxy/-P9T2UC73kDQNBN1mqy-OlduJ8UK8n7j0OieiZ1Kb3rh7OS9sNz_4S3b2SBwzdoHoyS2nAMLg27P9bITWGUyGOS7SgPz0A1w9lt2PF_1tUhegTcXSD-qLah9O3yG0AUKCnQlL2St_gImNh7G_l4j9KlBJwVnaOvGnLohefFu3RtIm66qKt0HBaVdGFoDQUgdNgchcyHONf1gQFAkgSiC2sB8QENrFD5cXDRUuIA1W1Iaoyh-qexbXNmsB99iTEAxSYT957M=s0-d-e1-ft#https://github.com/notifications/beacon/AAADBUFRDUP5FM3QXD67GJLWTEU6NA5CNFSM4KN4GVMKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGPAAAAAAPPMS65G.gif>Message ID: ***@***.***>