Ubuntu 18.04 ~end of life on aws ... for new accounts only? What?

[biblicabeebli]

Received this notification from AWS:

We are writing to inform you that, as of November 15, 2023, Canonical Group Limited will no longer offer "Ubuntu 18.04 LTS - Bionic" to new subscribers on AWS Marketplace. As a current subscriber, your use and subscription to Ubuntu 18.04 LTS - Bionic is unaffected; you can continue to create new instances, and any running instances will not be affected in any way.

It is possible this applies to just one of the multiple options available when you select 18.04 images on the web console, one of them has extra support or something and there are extra EULAs iirc. That seems unlikely tho.

I don't know who this is going to affect, please chime in if you are running beiwe servers and did or did not receive this notification.

(Frankly I find it incomprehensible that Canonicle can retire a product offering but aws can then selectively continue to offer it. Its a business decision for sure, but... ugh I can't even test this without making a new aws account...? Not good.)

[hackdna]

To be fair the standard support for 18.04 was over in May 2023 (https://ubuntu.com//blog/18-04-end-of-standard-support).

Also, Forest is tested on 22.04 and we might need to switch to 24.04 as soon as it becomes available to keep the ability to read mp3 files (onnela-lab/forest#211).

[biblicabeebli]

New related: the new server versions should also include an upgrade to the amount of storage used, and/or its time to implement data compression! :D

[biblicabeebli]

@hackdna thanks, I've reviewed onnela-lab/forest#211 and looked into the libsndfile version requirement, we need 1.1+.

We conveniently happen to have a 22.04 server I am shutting down today, apt search libsndfile contains

libsndfile1/jammy-updates,jammy-security 1.0.31-2ubuntu0.1 amd64
libsndfile1-dev/jammy-updates,jammy-security 1.0.31-2ubuntu0.1 amd64

So it looks like you are right.

And my laptop running 23.10 shows 1.2.2:

libsndfile1/mantic-updates,mantic-security,now 1.2.2-1ubuntu0.23.10.1 amd64 [installed,automatic]
ibsndfile1-dev/mantic-updates,mantic-security 1.2.2-1ubuntu0.23.10.1 amd64

It would be nice if we can push this into April, buuuuut I kinda think I'll have to get to it sooner.
https://ubuntu.com/server/docs/cloud-images/amazon-ec2 does list 23.10, so hopefully if I update the deployment script to work on 23.10, 24.04 will be entirely or almost entirely compatible.

Thanks for the heads-up.

[biblicabeebli]

@hackdna is there a non-python library binaries list anywhere for Forest? or are all the binaries supplied by their python package?

[hackdna]

Forest only needs ffmpeg and libsndfile1 on Linux: https://github.com/onnela-lab/forest/blob/develop/.github/workflows/build.yml

[biblicabeebli]

Worst case, I can throw old librosa binaries onto our servers if something becomes a blocker.

[biblicabeebli]

pulling this into current sprint

[biblicabeebli]

• rabbitmq has known issues, replace queue backend to resolve them #93
• General improvements to automation of data analyses, data processing, and other background tasks #305
• Update Celery [finally] #353
• Django 3.2 lts support ends on 1 Apr 2024 #352
• Ubuntu 18.04 ~end of life on aws ... for new accounts only? What? #331 - ubuntu platform upgrade - required before celery and rabbitmq.

[biblicabeebli]

Celery and django are updated, next up is this one (next week, expect staging server to be... problemy.)

[biblicabeebli]

Yesterday I completed the upgrade to Ubuntu 23.10. This is a short-term solution, there were several years of platform updates - mostly limited to RabbitMQ configuration and re-treading obscure details of AWS AMI version naming and other obscure properties.

This work is not 100% complete, I've run into a bug with the Harvard-specific custom on-deploy hook - news flash: Beiwe has a custom on-deploy hook for advanced deployments - and I will have to wait on checking in with IT on it. Otherwise, 23.10 is on our staging data processing server, I will be merging in the changes into the staging branch soon.

Ubuntu 24.04's release will be April 25th, 2024, I've made a new tracking issue for that.

• Ubuntu LTS release is on April 25th, 2024 #369

[biblicabeebli]

I have merged the 23.10 deployment code into main because the main branch has diverged just enough to require those changes anyway. Waiting on 24.04

[biblicabeebli]

closing in favor of #369