github-driven toolkits

[Baccata]

The idea of toolkit is really nice, and the opening of the idea to third-party toolkits is awesome ... but the requirement for it to be a published library on maven central is an undeniable barrier of entry.

It'd be super nice if this requirement could be alleviated via some kind of github-integrated mechanism, which would drastically reduce the barrier of entry for users to describe their own preferred set of libraries for personal scripts. The end goal being for users to put

//> using toolkit baccata

Scala-cli would fetch some json-file in the root dir of the repo (say scala.toolkit.json) from the github.com/baccata/scala.toolkit repo, that would describe a list of dependencies, that scala-cli would pull for compilation.

You could add some additional marks in the directive to pull from specific repos/branch/tags :

// pulls from a specific repo 
//> using toolkit baccata/foo

// pulls the file using a specific tag 
//> using toolkit baccata/foo:tag 

Of course the toolkit name in this proposal would have to be changed to something else to avoid collision with the existing one.

[tgodzik]

I think that is an interesting idea, though I heard some people opting for not having too much toolkits, because that again causes problems for users to choose which one they want 🤔

Also, maybe we should first open up an issue in https://github.com/scala/toolkit to talk about it? Or it doesn't really make sense? CC @szymon-rd

[romanowski]

requirement for it to be a published library on maven central is an undeniable barrier of entry

Scala CLI supports publishing to Maven Central, so the entry point should be considerably lower while we will stabilize that feature. What is most important, Scala CLI helps with setting up keys etc.

[Baccata]

What is most important, Scala CLI helps with setting up keys etc.

That's a fair point actually. Though the main barrier of entry is to have a domain name you control (even it's theoretically a one-off). Of course you can always use io.github.handle, but still the process is not self-service.

That being said, I can appreciate that a json piece of config that would have special interpretation in scala-cli would not facilitate the jump from scala-cli to other build tools when the need arise.

I heard some people opting for not having too much toolkits

Yeah I figured that'd be a blocker, which is entirely fair enough.

[szymon-rd]

From what I know, it doesn't necessairly need to be maven central. If you have some internal artifact repo in your company, or one that you use, then you are able to create new toolkits without publishing to maven central. Then, you have to add this external repo, but I think that's a fair compromise.

@tgodzik In what order does the scala-cli search the repositories? E.g. if you have an artifact with same name on maven central & your internal repo. I am asking about this to avoid introducing a new vector for supply chain attacks.

[Baccata]

If you have some internal artifact repo in your company, or one that you use, then you are able to create new toolkits without publishing to maven central

Does this mean it works with ivy local ?

[szymon-rd]

I think you have to put //> using repository ivy2local in your program first. But then, it should.

[tgodzik]

In what order does the scala-cli search the repositories? E.g. if you have an artifact with same name on maven central & your internal repo. I am asking about this to avoid introducing a new vector for supply chain attacks.

We use coursier underneath, usually it's the local jar first, but I don't think there is any guaranteed order otherwise.

[JD557]

I heavily use locally published artifacts with scala-cli and never had to add //> using repository ivy2local, so I think the defaults should be fine.

[Baccata]

Closing this as other avenues have been pointed out.