Skip to content
This repository has been archived by the owner on Aug 2, 2023. It is now read-only.

feature: add support for multi-arch image #541

Merged
merged 35 commits into from
Mar 14, 2022
Merged

Conversation

kyujin-cho
Copy link
Member

@kyujin-cho kyujin-cho commented Feb 19, 2022

This PR resolves lablup/backend.ai#374.

What's new?

  • move image metadata storage backend to postgreSQL
  • apply ORM pattern for new images and image_aliases table
  • add support for multi-architecture image handling
  • collect agent machine's CPU architecture
  • consider image architecture when scheduling kernel to prevent container starting with invalid architecture

Additional works needed

  • generalize ORM declaration and DB session lifecycle (this can be handled on seperate issue/PR)
  • alias etcd *-image command to newly added image * command, for backward compatibility
  • implement readonly txns and advisory lock (additional talks needed) for AsyncSession

@kyujin-cho kyujin-cho self-assigned this Feb 19, 2022
@kyujin-cho kyujin-cho added this to the 21.09 milestone Feb 19, 2022
@codecov
Copy link

codecov bot commented Feb 21, 2022

Codecov Report

Merging #541 (2fbfe6f) into main (3e444e0) will increase coverage by 0.29%.
The diff coverage is 41.59%.

❗ Current head 2fbfe6f differs from pull request most recent head 39198cb. Consider uploading reports for the commit 39198cb to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #541      +/-   ##
==========================================
+ Coverage   49.07%   49.36%   +0.29%     
==========================================
  Files          55       55              
  Lines        9166     8945     -221     
==========================================
- Hits         4498     4416      -82     
+ Misses       4668     4529     -139     
Impacted Files Coverage Δ
src/ai/backend/manager/api/manager.py 37.26% <0.00%> (-0.63%) ⬇️
src/ai/backend/manager/models/session_template.py 31.64% <ø> (ø)
src/ai/backend/manager/scheduler/dispatcher.py 24.93% <12.50%> (-0.47%) ⬇️
src/ai/backend/manager/registry.py 17.70% <16.66%> (-0.03%) ⬇️
src/ai/backend/manager/cli/image_impl.py 23.15% <23.15%> (ø)
src/ai/backend/manager/models/image.py 49.50% <43.39%> (-8.29%) ⬇️
src/ai/backend/manager/cli/etcd.py 62.43% <51.72%> (+12.64%) ⬆️
src/ai/backend/manager/models/utils.py 78.01% <52.38%> (-4.49%) ⬇️
src/ai/backend/manager/models/base.py 53.80% <91.66%> (+0.26%) ⬆️
src/ai/backend/manager/cli/__main__.py 53.76% <100.00%> (+1.01%) ⬆️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e444e0...39198cb. Read the comment docs.

@kyujin-cho kyujin-cho marked this pull request as ready for review February 21, 2022 12:17
@kyujin-cho kyujin-cho requested a review from achimnol February 21, 2022 12:31
Copy link
Member

@achimnol achimnol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First batch of reviews. Also please merge the main branch.

src/ai/backend/manager/models/image.py Outdated Show resolved Hide resolved
src/ai/backend/manager/models/image.py Show resolved Hide resolved
src/ai/backend/manager/models/image.py Outdated Show resolved Hide resolved
src/ai/backend/manager/models/base.py Show resolved Hide resolved
src/ai/backend/manager/models/image.py Outdated Show resolved Hide resolved
src/ai/backend/manager/models/utils.py Show resolved Hide resolved
src/ai/backend/manager/scheduler/types.py Outdated Show resolved Hide resolved
tests/test_image.py Outdated Show resolved Hide resolved
@kyujin-cho kyujin-cho requested a review from achimnol March 10, 2022 02:42
Copy link
Member

@achimnol achimnol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! 🎉
LGTM for now, but there may be more future improvements of course.
Let's handle them as separate issues and PRs.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extend and migrate container image metadata
2 participants