Module versioning, dependency management, and more ... #4574
Comments
|
First of all, NPM is pretty much a repository as well. The idea behind Deno is to simply not limit yourself to one single repository (as Node does) for your entire project, in pros of not being dependant of a private company like NPM is. Package Managers should be really a configurable option (again not really the case in Node since it all gets pulled from NPM) and Deno offers that option. Deno not choosing one as their default but relying in the JavaScript native module system is a smart move. Doesn't mean you can't have a Package manager in Deno. Most likely a way to pull NPM modules directly will be written shortly after 1.0 release. Now on the question of how it goes for me using this approach, it works just fine. I commonly avoid using non-vital libraries anyway. One particular experience I had was using a dependency that got broken under the last version of Deno. I fixed it in my local fork of the lib and then pulled from there while waiting the PR to merge with the lib source. Not the best example, but I could effectively replace a whole lib with a newer version without much a hassle just by changing its URL. Cases where NPM just randomly drops packages may find its solution here. Also and I said this some time ago while talking on how I deployed my app with Deno. |
😄
Go's modules are different from ours - specifically they are not standard URLs. I can not just host a Go module on my http server easially. Go's packages are hosted in version control systems and the Go tool uses these version control systems directly ( Deno is completely different - it has exactly 0 opinions or reliance on anything other than HTTP. Deno requests its modules via HTTP. How the identifier (URL) looks is completely up to the implementer. Deno does not have the problem of 'not being able to lock to a certain commit' for example, that for Go, tools like godep solved before Go modules existed. Because Deno has no understanding of what Git is, it doesn't need to have any understanding of commits and versions either and problems like these are completely non existent. In Deno, the module identifiers include all information you want them to include (version, path, repository - you can decide because you structure how your server handles incoming requests from Deno). All information that for Go is spread across multiple places (the
This is where URL redirector services like deno.land/x, denolib.com, pika.dev, jspm.io, ect come in. They provide a layer of abstraction so you don't have to worry about not being able to restructure your repo.
Again see point above, redirector services like deno.land/x or pika.dev solve this. Also do you actually use NPM's built in search? If I need something I use Google to find it, not NPM's search. I really don't care if that sends me to Github, deno.land/x or pika.dev. About the repeatable install: In NPM's case you are just trusting NPM Inc. that they will pay for their server equipment, won't shut down in your country or territory, serve you non malicious packages, not delete the package you are using, ect. This is all about trust & reliance on a single party. If NPM goes bankrupt tomorrow and their registry shuts down all of your builds will fail. It's a single point of failure.
There are tools to make your life easier. For example, hayd's udd. One more thingDeno tries to be compatible with the web where it can. Using a different module format from the web does not work if you are trying to be compatible. URLs have worked fine for the browser for the last 20 years, and it's not a battle between many competing centralized repositories either right? Why does Deno need one, and why would this happen? Deno can work with whatever registry you want it to work with because it is not opinionated. |
Why don't you consider ESM a "proper module system"? Versioning is critical and IMO it's important that libraries and users reference immutable urls (rather than pointing to "master")... that's really a "problem" with urls in general but can be a guarantee of a repository (see also #200 / --lockfiles). Generally git tags or npm releases are what are being used atm. |
|
@Soremwar The question is not how it goes for you specifically but how you guys are seeing:
Now if you are not developing with a team, and do not rely on any versioned open source or internal module dependencies, the simpler the better but it's a very specific use case.
|
|
That will do @kitsonk |
However there are best practices (that you'll see in many deno libraries):
|
Yes it does - Deno's proposal for version & dependency management is the exact same as the browsers. The URL is a (preferably immutable) reference to the module you are importing, and that reference can include a version. Part of this is also Deno having built in support for lock files. https://deno.land/std/manual.md#lock-file |
|
And how I can answer those concerns:
|
Go doesn't have this either.
|
|
What security checks are in Node.js or npm? None... All the security auditing for modules is an add on to the eco-system. Having a package manager and a centralised registry have lulled people into a very false sense of security. |
Also your trusting a single party to give you correct information - not the best bet for security. The best security is the one that is distributed and where nobody is trusted (e.g blockchain). |
|
@kitsonk I don't get it. Yes, the security auditing is an add on and then? |
So why are you suggesting the Deno needs to deliver it before 1.0, or even have an opinion about it? Why isn't it acceptable to be addressable by the wider community? |
Deno is the core, add ons can be developed separately. Google doesn't develop all Chrome extensions either. Third parties do.
I prefer not trusting a single for profit org that holds all of the worlds node packages. Up to you. Should not be forced on the user, they should choose. |
|
Was wondering how reproducible builds are handled ? Will there be any kind of lock file somewhere regarding the imported libraries versions ? What I understand from the previous messages, we can't trust anyone, so how can we be sure the version we used yesterday hasn't changed in the mean time ? |
|
@hayd |
|
|
There is transparency in the dependency tree (deno info)... so you can at least enumerate.
Security concerns is a much harder problem... and not solved by anyone/any language (yet?). But crev seems promising... https://github.com/crev-dev/cargo-crev Your best bet: minimize dependencies and review them. 🤷♂
My bet is libraries and repositories that choose sensible version schemes will be the most popular! |
Versus having a single private company dictate it to you? Deno just doesn't have opinions about it. It moves the semantic meaning elsewhere. |
You are so negative :) |
|
Regarding the idea of "if Deno doesn't endorse one solution the community will create too many" -- "too many solutions" is only bad when you require that your dependencies are using the same solution, like a |
|
The analogy I have used before is Deno is like a browser for JavaScript code. The browser doesn't dictate to you how to structure your website to have good usability, and what good usability for a website is has evolved over the years. So dictating conventions too early, and having your "browser" dictate those conventions are likely bad things. Part of it is the concept of making decisions at the Last Responsible Moment. Why dictate a version scheme, until there is a real world problem where a good semantic versioning scheme would be a solution to it. I personally haven't seen that problem be a real problem yet, at least not to the point of having to bake that concept into the runtime. |
|
@kitsonk Well, I'm not really buying your example. But providing those tools out of the box has obvious great advantages:
Following the same approach Deno's team could come up with a non mandatory recommended approach and tools for handling in the easiest possible way versioning and dependencies. Yes, I know The life of Brian. We are indeed individuals but we have to follow common patterns when working together. That being said, I learned a couple of things and have to do my homework and read a couple of additional information. If you ask me, a chapter on versioning and dependency management the Deno way would be more than welcome. We are all coming from a package manager based environment. The mental shift is not obvious at first sight. |
|
As I stated before, Deno provides this tools cause JS doesn't have a native implementation for them. However it does have a native implementation for modules that's been around for quite a while now
These tools for Deno are aiming not to be opinionated but configurable by the community. A package manager would likely be opinionated on where to fetch the dependencies. Deno test library can be forked by another person to offer a different approach or different functionality, that's why it's an std/ library instead of built in Deno For linting and formatting you could dictate your own directives, so there's no point on argue about that Is all about giving people the freedom to do whatever they want. Don't lean on nothing but Deno and you are just as good as if you were leaning on external tools |
|
@Soremwar Come on. There are already plenty of tools for formatting, linting, testing in the JS/TS world. Once again, I'm not sure what you are trying to say. Except for formatting tool, my argument is not about being opinionated but having a set of tools that will certainly be used in most cases. Having an agreed set of tools (even configurables one) will make life easier for the reason I mentioned previously. And I have not heard that there were any concerns for versioning and integrity in the js module system. If you came up with locks file, udd and such it's for a reason. Js modules per se are not enough without at least some conventions and at least a couple of tools to handle those legitimate matters (except for you if you are the only developer and does not rely on any modules). |
|
Ok, thanks guys for the additional information. I was obviously missing some bits. |
|
I'm not sure why Deno having a built-in formatter is reasoning to suggest it should have a built-in dependency manager. They're not the same thing. With formatters etc. you can look at the existing ones, observe that they all do pretty much the same thing and, well... add a formatter. With centrailised dependency management (or specifically all of those little things you say it provides which should really be discussed individually 😬) you just have an existing bad solution. That's why we have one and not the other, even though we don't need either. |
|
@nayeemrmn I can't see where you get that I suggested that having a built-in formatter implies that it should have a dependency manager
I can't see any mention of a dependency manager and even less a centralized one.
Must feels good to get it right when all other languages and environment get it wrong :) |
I can, here and there in earlier comments. I maybe conflated things though.
Language is fixed, one other environment. |
|
I think it has been a good discussion and worth having.
👍 It does, and will, come up a lot so maybe it'd be good for an FAQ? We will want to be careful in not picking a winner and perhaps outlining strategies with pros/cons. That said, it feels like there is broad agreement RE always using versioned urls. (importmaps vs deps.ts vs inline imports are note worthy.) |
Here is what I'm saying in my initial post:
Then 2 hours ago answering Andy Hayde:
Honestly, you mixed it up. Doing a parallel with all the non mandatory tools provided, I was suggesting that it would be a good idea that Deno team comes up with a set of (even non mandatory) convention/approach/tools (and in a way ti's already the case with locks-file generation) toward versioning and dependencies concerns. |
|
@hayd Yes, I get that you don't want to come up with a definitive approach. At the same time there are developers that don't really want to think about those cross cutting concerns more than necessary. Not necessarily because they would not be smart enough but because they already have complex business models and services to deal with. They don't really care an approach or another as long as it solves the usual concerns with module versioning and dependencies. At the very least, proposing one way (combining a set of convention and tools like udd or whatever) and showing the workflow associated will help approaching Deno. |
|
Im just here to say that I agree with the decision made by the Deno team. A dependencies manager should be a separated project, and should not be rushed. Maybe something like Cargo(rust) should be more than enough. And it should be separate from the Deno binary. |
|
Just a small question. Why don't we use importMap JSON as the package.json for deno? It's web compatible. For example, |
|
I'm curious what @ry and others think of Go Modules ... and whether it is possible/desirable to adopt that style of dependency management (or some aspects of it) for Deno (either before v1 or even afterwards - I personally wouldn't mind either) The reason why Go team didn't initially have dependency baked in was they wanted to take time to research alot of common pitfalls you find in systems like npm, cargo, and even godep. The Minimal Version Selection algorithm is one of the key ingredients that sets Go module system apart from the rest in this respect. It's used to basically guarantee reproducible builds. Here are some great talks that describe these features and issues in some detail: |
|
I do plan to present Deno 1.0 to my team. But when that inevitable "how does dep management work" question comes up, it would help if we had a Deno way FAQ/tidbit I can read from. Currently my response will be give or take, dep management is handled the way Browsers handle it, that it doesn't care as long as it's via HTTP to a URL in the form of a package@ver. In other words no built-in tools That's a tough pill to swallow for a lot of people. |
|
Sensible requirements in order of implementation date: (my opinion)
I don't think the Deno binary/runtime or the Deno team themselves should be responsible for implementing package management, but if they did it would be a huge boon to the present and future community. I look to languages like Rust/ |
|
I agree with 1 & 2 and have an idea for a community tool... but it's kinda unclear to me what it's scope should/will be. Personally I find deps.ts quite elegant so helping make writing those easier is key (add foo and it populates). What do you think the criteria for "package manager" is for deno? |
|
I closed this issue because I was unsure if it wasn't going to go into a war between "Deno is perfectly fine as of now - no need to add anything" vs "Hey .. what's up with versioning and dependencies?". Now if this is going into discussing pragmatic concerns/features that can possibly help Deno team to shape something, I guess it could be reopened. |
|
Just to add a voice of a passerby with an eye on Deno. I agree whole-heatedly with @FredericLatour on this. The issue here is not about centralisation to a particular company/system but centralisation in terms of mind share around how package management is done. It's immaterial whether the artefacts or index is distributed (and that's all fine and good). There is just a huge bonus in reduced overhead by being somewhat opinionated about how versioning works and how a dependency graph is resolved that is consistent between dependencies. The Go example is useful, and having lived through the best part of a decade of hand-wringing from the Go team over what turned out to be an essential feature; losing countless hours to the brokenness and fragmentation of Go dependency management (and of Go devotees exalting the but-you-don't-get-it greatness of vendoring dependencies) it's not something I would inflict on myself again. from @Jack-Works:
I think that is what people would inevitably start to do. Having a format to represent dependencies is only a small part of it though. You would also what tooling to resolve transitive dependencies to appropriate versions, deal with conflicts, and the such. Something that is 80% optimal that 99% of the community use by convention is so much better than a 90% optimal community solution that 20% of the community use here such is the network effect with package management. It's also one of the things that most people coming to your ecosystem are likely to touch first and that that experience is likely to colour their opinion and adoption in general. |
|
I've been using Node for many years, I even remember watching Ryan on YouTube, I was inspired. It looks like Ryan has done it again, although in his recent YouTube videos he seems more nervous.. :) In his recent video were he talks about what mistakes he made with Node, I totally agree. Been able to share code between web & back-end is a real bonus. But the web standard has it's own mistakes, and dependency management is one of them. That's the reason we have things like Webpack etc, and even now using the latest browsers and ES modules anything big still requires a bundler. The thing is the the Web (front-end), is the final output. So any form of dependency is not required. But back-end solutions like Deno, this is not the case. I think the solution here is very simple, Deno dev's need to take control here, not so much in coding, but just creating a standard, like maybe the @Version in the URL etc. Ryan said in his video he regrets ignoring the advice he got from the community, so please don't ignore this one. Deno has the potential to be a replacement for Node in so many ways. So here is hoping the Dev's take control here.. |
First of all, kudos for what sounds like a great project.
Using exclusively typescript for years, I'm quite excited of being able to use Deno in the future.
Now, I just don't get how it is possible to develop apps at a meaningful scale without a proper module system. I had already shared my concerns once on discord but was not expecting that a 1.0 would be released without any meaningful work in this area.
Because it sounds like Deno approach seems to be influenced by Go on this matter, let me copy and paste an extract from the Go documentation:
What's the point of going into the exact same pain points. Why not learning from this experience and avoiding a sure mess to come?
At the very least a proper module system should make it possible to handle versioning and dependency management.
Now is a central repository important or not? Honestly, I'm not really fond of everybody hosts it's publicly available modules in its own git repository because, as a developer, either you are not able to refactor your repository organisation or you will break module installs for everybody relying on your repository. Alternatively, if you do not want to face this dilemma, you will come up with a specific github repository where you publish your modules in order to mimic a "kind-of" group/dev/company central repository.
I tend to believe that a central repository has its use for public modules discoverability and repeatable installation paths but I have no religion on the subject. In the same time, the ability to import modules outside of a central repository (especially when private modules are concerned) is a must have as well.
I'm really interested to read how you guys are seing the development workflow of a reasonably sized app that depend on publicly available modules + internally developed modules with the current Deno module approach.
The text was updated successfully, but these errors were encountered: