feat: SPM(Swift Package Manager) backend #2241

kattouf · 2024-06-02T07:52:13Z

Implementation of backend for install executables managed by Swift Package Manager

Description

Discussion
Compared to previous PR there are no dependencies on third parties other than Swift itself.

SPM does not support installing an executable, only the build command. This backend implementation itself covers this functionality and is described by a flow:

Clone the repository with the SPM package into a temporary directory.
Read Package.swift for a list of executables
Build each executable and copy the binary with associated resources to mise install directory

Demo screenshots:

P.S. I'm not full-time Rust developer, feel free to make comments if something is not done according to best practices in Rust community

Requires attention

Missed swiftdependency
I had some problems installing swift with the current asdf plugin so I guess we'll just have to wait for Add Swift plugin #1708
Now missed dependency leads to:
- Unable to run e2e test
- Users are responsible for installing swift
Support for build artifacts on other platforms
I have no experience using swift on other platforms (Linux, Windows) and can't be sure about finite list of resource and dynamic libraries formats we should support to copy together with binary.
I decide dive into it after we are sure that everyone is happy with the feature and we will discuss merging it.
Name of package to install
1. I don't sure about how flexible it should be, so for now I limited it to two options:
  - GitHub repo name: spm:nicklockwood/[email protected]
  - GitHub https url: spm:https://github.com/nicklockwood/[email protected]
    Maybe we should add other options by community requests
2. When we use different formats for the same package repo, they are perceived as different packages:
  
  I got some errors when trying to change the BackendArg name to generic format (shortened) in SPMBackend::new(name)
  Are there any other options to solve this problem?
Versioning
Current implementation looking for GitHub release tags for installation candidates. Installation by branch/commit can also be supported, but I'm not sure if this should be done now.

src/backend/spm.rs

pepicrft

Logic-wise it looks prefect. Nicely done @kattouf.

pepicrft · 2024-06-02T12:33:20Z

src/backend/spm.rs

+impl SwiftPackageRepo {
+    fn new(name: &str) -> Result<Self, eyre::Error> {
+        let shorthand_regex = regex!(r"^[a-zA-Z0-9_-]+/[a-zA-Z0-9_-]+$");
+        let shorthand_in_url_regex =


Would it make sense to generalize this such that it also supports other Git platforms, for example Gitlab?

Yes, it makes sense globally, I just decided not to spend too much time on unpopular cases right now.

I think we could cover almost everything else cases by implementing:

support for any URLs up to .git

support for various git revisions as versions: tag, commit, branch

and divide the versioning flow into:

github releases

specific git revision

But is it worth doing this right away, I'm not sure

I think this can be done in a follow-up PR. It seems a good candidate to be extracted into a separate function that can be reused across the backend. I'm sure @jdx has already put some thoughts there.

src/backend/spm.rs

pepicrft · 2024-06-02T12:42:21Z

src/backend/spm.rs

+struct PackageDescriptionProduct {
+    name: String,
+    #[serde(deserialize_with = "PackageDescriptionProductType::deserialize_product_type_field")]
+    r#type: PackageDescriptionProductType,


Out of curiosity, what's the meaning of this r#type annotation? (I'm not that versed at Rust)

It's raw identifier, used to define user data names, same as language keywords (like `type` in swift)
https://stackoverflow.com/a/69871965

Gotcha! Thanks for the reference to Swift, it helped :)

pepicrft · 2024-06-03T07:41:00Z

e2e/backend/test_spm_slow

+mise use swift
+
+assert "mise x spm:nicklockwood/[email protected] -- swiftformat --version" "0.53.10"
+assert "mise x spm:https://github.com/nicklockwood/[email protected] -- swiftformat --version" "0.53.10"


Would you mind adding https://github.com/tuist/tuist?

Ha, with pleasure! But the installation time seems too long for testing, don't you think?
Maybe we could to this with some install specific executable feature, but now it takes about 5m and looks not tests friendly.

5m is terrible! So feel free to skip it then :)

Co-authored-by: Pedro Piñera Buendía <[email protected]>

kattouf · 2024-06-03T09:18:33Z

@pepicrft Could you also check Requires attention section in PR description? None of this seems like a blocker to you right now?
We have already partially discussed points 3 and 4, so you can focus on 1, 2 and 3.ii

pepicrft · 2024-06-03T13:11:04Z

Missed swift dependency

I'd be consistent with what the other backends do. If they install the required toolchain absent, we can error out telling users that "Swift was not found in the environment" until we install it ourselves. This is a scarce scenario because most developers using this backend will likely be Xcode developers with the toolchain available in their systems.

Support for build artifacts on other platforms

I'm in a similar spot here. At least on Linux, if everything is statically linked, the binary should work, so the issues will likely arise when using dynamic dependencies, which I'm not entirely sure are supported on Linux. Similarly, this is a bit uncommon, so I'd ship this as it is and iterate on it as feedback comes our way.

When we use different formats for the same package repo, they are perceived as different packages

This is a bit confusing, especially considering that they are referring to the same package. @jdx might be able to help with this one.

kattouf · 2024-06-03T13:50:11Z

I'd be consistent with what the other backends do.

Currently mise doesn't automatically install backend dependencies, instead the documentation suggests installing them using mise or any other convenient method and gives an error message:

It seems that the backend is a feature that expects that it will be used by those people who already have the entire necessary tools, they just want to describe them in a single config

Similarly, this is a bit uncommon, so I'd ship this as it is and iterate on it as feedback comes our way.

Agree 🤝

pepicrft · 2024-06-03T17:22:43Z

Overall, fantastic work @kattouf 👏🏼. As one of the advocates of Mise in the Swift ecosystem, I couldn't be more excited of this work landing.
We have to wait for @jdx to do the merge.

* first simple implementation * get executables list and work with them only * rename SpmForge -> SPMBackend * Package description deserialization with serde * copy build artifacts * refactoring * spm e2e tests * use project git client * refactoring * fix repo url name usage; fix git checkout * update test assert * fix lint violations * fix package version in e2e test * add spm repo parsing unit tests * Better error message Co-authored-by: Pedro Piñera Buendía <[email protected]> * reformat --------- Co-authored-by: Pedro Piñera Buendía <[email protected]>

kattouf added 12 commits May 30, 2024 13:14

first simple implementation

c4b0deb

get executables list and work with them only

d424710

Merge branch 'main' of https://github.com/kattouf/mise into spm-backend

72f4bae

rename SpmForge -> SPMBackend

f338e49

Package description deserialization with serde

9074941

copy build artifacts

4819ffb

refactoring

760c983

spm e2e tests

a8f993f

use project git client

bb50e3d

refactoring

bca5b4c

fix repo url name usage; fix git checkout

b3716ef

update test assert

ff3326e

kattouf commented Jun 2, 2024

View reviewed changes

src/backend/spm.rs Outdated Show resolved Hide resolved

fix lint violations

f293cc4

jdx approved these changes Jun 2, 2024

View reviewed changes

kattouf added 2 commits June 3, 2024 11:22

fix package version in e2e test

0fecc0b

add spm repo parsing unit tests

d67999b

kattouf force-pushed the spm-backend branch from cf73dbc to d67999b Compare June 3, 2024 04:39

pepicrft approved these changes Jun 3, 2024

View reviewed changes

kattouf and others added 2 commits June 3, 2024 15:20

Better error message

9dd0b89

Co-authored-by: Pedro Piñera Buendía <[email protected]>

reformat

36ba1da

jdx merged commit 8d37aab into jdx:main Jun 3, 2024
10 checks passed

This was referenced Jun 3, 2024

chore: release 2024.6.1 #2239

Merged

mise 2024.6.1 Homebrew/homebrew-core#173619

Merged

kattouf mentioned this pull request Jun 4, 2024

feat: Mint(swift) backend #2199

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: SPM(Swift Package Manager) backend #2241

feat: SPM(Swift Package Manager) backend #2241

kattouf commented Jun 2, 2024 •

edited

Loading

pepicrft left a comment •

edited

Loading

pepicrft Jun 2, 2024

kattouf Jun 3, 2024

pepicrft Jun 3, 2024

pepicrft Jun 2, 2024

kattouf Jun 3, 2024

pepicrft Jun 3, 2024

pepicrft Jun 3, 2024

kattouf Jun 3, 2024

pepicrft Jun 3, 2024

kattouf commented Jun 3, 2024 •

edited

Loading

pepicrft commented Jun 3, 2024

kattouf commented Jun 3, 2024

pepicrft commented Jun 3, 2024

feat: SPM(Swift Package Manager) backend #2241

feat: SPM(Swift Package Manager) backend #2241

Conversation

kattouf commented Jun 2, 2024 • edited Loading

Description

Requires attention

pepicrft left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kattouf commented Jun 3, 2024 • edited Loading

pepicrft commented Jun 3, 2024

kattouf commented Jun 3, 2024

pepicrft commented Jun 3, 2024

kattouf commented Jun 2, 2024 •

edited

Loading

pepicrft left a comment •

edited

Loading

kattouf commented Jun 3, 2024 •

edited

Loading