Fix the versioning

[danielchatfield]

1.7.0 introduced loads of breaking changes.

The number of dependant modules which are now broken as a result is huge, personally I think that 1.7.0 should be killed (removed from npm) and 2.0 released - the longer the delay the harder it will be to do this.

underscore.js is solely consumed via package managers that mandate the use of semver, you may personally not like semver but that is what is used by the installers to determine compatibility. Last time this was brought up you stated that if you used semver then we would be on underscore version 47 now - well that is much better than having broken code everywhere and lodash has managed to keep the version number below 4.0.0 without breaking everyone's code.

[jashkenas]

I think we should leave 1.7.0 as 1.7.0. There are a lot of changes, some of which change behavior for edge cases, which justifies bumping the minor version up a notch. If you're using npm's ~ operator, you won't be automatically upgraded to 1.7.0.

But it's not really the kind of complete overhaul or massive change to the public API that justifies a 2.0. Perhaps once things are really polished and locked down.

But that said, let's leave this ticket open for discussion...

[danielchatfield]

If you're using npm's ~ operator, you won't be automatically upgraded to 1.7.0.

By default npm uses ^

[toddself]

According to Semver.org bumping the minor release should:

MINOR version when you add functionality in a backwards-compatible manner, and

If the behaviors changed do not solely constitute urgent bug fixes which were causing the edge cases to change, this is not a minor release.

Coupled with the change to NPM moving to ^ which will accept any version within a major release AND the fact that underscore is the MOST depended upon package in NPM it is likely better to resolve this issue as @danielchatfield suggests.

From your changelog:

Underscore templates no longer accept an initial data object. _.template always returns a function now.

Changing a method signature and what it returns (without it previously being deprecated) in a minor version is a pretty big deal:

> 14:21:41 ~/src/test
↳ npm i underscore
npm http GET https://registry.npmjs.org/underscore
npm http 304 https://registry.npmjs.org/underscore
npm http GET https://registry.npmjs.org/underscore/-/underscore-1.7.0.tgz
npm http 200 https://registry.npmjs.org/underscore/-/underscore-1.7.0.tgz
[email protected] ../../node_modules/underscore
14:21:47 ~/src/test
↳ node
> var und = require('underscore');
undefined
> und.template('hello: <%= name %>', {name: 'moe'});
{ [Function]
  source: 'function(obj){\nvar __t,__p=\'\',__j=Array.prototype.join,print=function(){__p+=__j.call(arguments,\'\');};\nwith(obj||{}){\n__p+=\'hello: \'+\n((__t=( name ))==null?\'\':__t)+\n\'\';\n}\nreturn __p;\n}' }
>

Versus:

14:20:43 ~/src/test
↳ npm i [email protected]
npm http GET https://registry.npmjs.org/underscore
npm http 200 https://registry.npmjs.org/underscore
npm http GET https://registry.npmjs.org/underscore/-/underscore-1.6.0.tgz
npm http 200 https://registry.npmjs.org/underscore/-/underscore-1.6.0.tgz
[email protected] ../../node_modules/underscore
14:20:52 ~/src/test
↳ node
> var und = require('underscore');
undefined
> und.template("hello: <%= name %>", {name: 'moe'});
'hello: moe'

[jamiebuilds]

@jashkenas I understand and respect your stance regarding semver. However, when making breaking changes I believe there is a universal expectation (that predates semver) to bump a major version. At the very least it sets expectations properly, but it will also help those who do follow semver (which is the vast majority of developers these days).

Considering how many people who depend of underscore now have broken code because of this latest release, I think it would be a good decision to do what @megawac said:

releasing 1.7.1 or 1.8 as a revert of 1.6.0 and then working on 2.0.0 for some time next week

[scottnonnenberg]

+1 on removing and replacing with 2.0.0. Rigid semver specifies that no breaking changes can appear except as a major version (unless you're below version 1.0). This has introduced mysterious breaks for me, deep in node module hierarchies. :0(

[josephspens]

This definitely just caused major issues for me. That's what I get for not thoroughly reading the changelog, but I didn't imagine a minor version upgrade would be that dangerous.

Is the solution to add backwards compatibility back into 1.7?

[spadgos]

If you're using npm's ~ operator, you won't be automatically upgraded to 1.7.0.

This really isn't the point. Semver is meant for humans just as much as machines. The entire point of semver is to give confidence to consumers of the library that they can safely upgrade, whether that happens automatically with a package manager instruction or by hand.

it's not really the kind of complete overhaul or massive change to the public API that justifies a 2.0

That's not what the major number is for though. It's for any breaking change.

[jdalton]

I'm 👍 for revising how Underscore bumps. It seems like most releases in recent memory have had some kind of hiccup with the 1.7.0 bump being the worst (failed unit tests, incomplete features, missing documentation, missing breaking change notices, and versioning that caused many projects to break). The process as it stands today isn't working.

[mathiasbynens]

Underscore is among the most popular packages on npm. The whole npm ecosystem is based on semver. Underscore should respect semver.

[webRat]

Underscore templating changes in 1.7.0 broke our project/templates and we had to revert back to 1.6.0. Pretty big / major change to the templating system alone.

[jashkenas]

As much as I'd like to make everyone happy by using "semantic versioning" — I'm afraid I just can't swallow the stuff. This margin is too narrow to contain the argument against it, but suffice to say, Node doesn't do it, Rails doesn't do it, Python doesn't do it, jQuery doesn't (really) do it — and as a fun little open-source project, it's nice for us to provide real version numbers that are useful for humans, instead of mechanical version numbers that are of merely dubious use for computers.

But you can have your cake and eat it too. I've just published an "official" npm repository for underscore-semver, which will be released at the same time as Underscore is released, and will always contain a "semantic" derivation of the true Underscore version number. Feel free to depend on that, if you prefer: https://www.npmjs.org/package/underscore-semver

For the curious, or if you'd like to do something similar, here's the script I'll be using to keep things in sync: https://gist.github.com/jashkenas/c71021bba8ee580ded92

I hope that's an acceptable solution.

[megawac]

I'm sorry @jashkenas thats a ridiculous response. What is the human readability issue about v2.0 over v1.7. Underscore breaking releases don't have to be more than once a year. Moreover they don't all have to be breaking as the underscore-semver repo seems to indicate...

All people are asking is when you change the contract (not extending the contract) of the function, you let them know via a major bump.

[toddself]

Additionally the an issue here is nested deps -- things that rely on underscore that you in turn rely on. Now, each of those packages needs to be updated and published to point to a new repo as well. Things that may not be well maintained may be forever broken now.

[danielchatfield]

Tl;DR goodbye underscore, hello lodash

[braddunbar]

The most useful thing you can do with a version number is not break people's code.

Regardless of opinions on semantics, it was obvious before this release that using 1.7.0 as the new version number would break a significant amount of code, likely in a non-obvious manner. I've yet to hear any reasoning that could justify that loss in user trust and productivity.

[w-shackleton]

Having two identical repos with different versioning schemes is going to lead to even more headaches than having nonstandard versioning.

[davidchambers]

It's one thing to state, @jashkenas, the way in which you believe version numbers should be used. The underscore-semver package, though, is a straw man: none of us is suggesting that Underscore should be at v170.0.0 by now!

[twhaples]

@jashkenas -- Wait, so when people came to your project with real concerns about the health of the node.js ecosystem and the expenditure of thousands of man-hours fixing things and hunting bugs, your response was to write a script to sync something over to a version called "v170.0.0"? (which also does nothing to address the issues they identified with an existing install base)

seriously? don't you think that's kind of going to alienate parts of the community? it's a whole lot easier to read that as spite than as an attempt to be helpful... you're in a position of some responsibility and authority here, and this sort of thing just lets everybody down.

[jashkenas]

Cross-posted from https://gist.github.com/jashkenas/cbd2b088e20279ae2c8e:

Spurred by recent events (https://news.ycombinator.com/item?id=8244700), this is a quick set of jotted-down thoughts about the state of "Semantic" Versioning, and why we should be fighting the good fight against it.

For a long time in the history of software, version numbers indicated the relative progress and change in a given piece of software. A major release (1.x.x) was major, a minor release (x.1.x) was minor, and a patch release was just a small patch. You could evaluate a given piece of software by name + version, and get a feeling for how far away version 2.0.1 was from version 2.8.0.

But Semantic Versioning (henceforth, SemVer), as specified at http://semver.org/, changes this to prioritize a mechanistic understanding of a codebase over a human one. Any "breaking" change to the software must be accompanied with a new major version number. It's alright for robots, but bad for us.

SemVer tries to compress a huge amount of information — the nature of the change, the percentage of users that will be affected by the change, the severity of the change (Is it easy to fix my code? Or do I have to rewrite everything?) — into a single number. And unsurprisingly, it's impossible for that single number to contain enough meaningful information.

If your package has a minor change in behavior that will "break" for 1% of your users, is that a breaking change? Does that change if the number of affected users is 10%? or 20? How about if instead, it's only a small number of users that will have to change their code, but the change for them will be difficult? — a common event with deprecated unpopular features. Semantic versioning treats all of these scenarios in the same way, even though in a perfect world the consumers of your codebase should be reacting to them in quite different ways.

Ultimately, breaking changes are no fun, and we should strive to avoid them when possible. To the extent that SemVer encourages us to avoid changing our public API, it's all for the better. But to the extent that SemVer encourages us to pretend like minor changes in behavior aren't happening all the time; and that it's safe to blindly update packages — it needs to be re-evaluated.

Some pieces of software are like icebergs: a small surface area that's visible, and a mountain of private code hidden beneath. For those types of packages, something like SemVer can be helpful. But much of the code on the web, and in repositories like npm, isn't code like that at all — there's a lot of surface area, and minor changes happen frequently.

Ultimately, SemVer is a false promise that appeals to many developers — the promise of pain-free, don't-have-to-think-about-it, updates to dependencies. But it simply isn't true. Node doesn't follow SemVer, Rails doesn't do it, Python doesn't do it, Ruby doesn't do it, jQuery doesn't (really) do it, even npm doesn't follow SemVer. There's a distinction that can be drawn here between large packages and tiny ones — but that only goes to show how inappropriate it is for a single number to "define" the compatibility of any large body of code. If you've ever had trouble reconciling your npm dependencies, then you know that it's a false promise. If you've ever depended on a package that attempted to do SemVer, you've missed out on getting updates that probably would have been lovely to get, because of a minor change in behavior that almost certainly wouldn't have affected you.

If at this point you're hopping on one foot and saying — wait a minute, Node is 0.x.x — SemVer allows pre-1.0 packages to change anything at any time! You're right! And you're also missing the forest for the trees! Keeping a system that's in heavy production use at pre-1.0 levels for many years is effectively the same thing as not using SemVer in the first place.

The responsible way to upgrade isn't to blindly pull in dependencies and assume that all is well just because a version number says so — the responsible way is to set aside five or ten minutes, every once in a while, to go through and update your dependencies, and make any minor changes that need to be made at that time. If an important security fix happens in a version that also contains a breaking change for your app — you still need to adjust your app to get the fix, right?

SemVer is woefully inadequate as a scheme that determines compatibility between two pieces of code — even a textual changelog is better. Perhaps a better automated compatibility scheme is possible. One based on matching type signatures against a public API, or comparing the runs of a project's public test suite — imagine a package manager that ran the test suite of the version you're currently using against the code of the version you'd like to upgrade to, and told you exactly what wasn't going to work. But SemVer isn't that. SemVer is pretty close to the most reductive compatibility check you would be able to dream up if you tried.

If you pretend like SemVer is going to save you from ever having to deal with a breaking change — you're going to be disappointed. It's better to keep version numbers that reflect the real state and progress of a project, use descriptive changelogs to mark and annotate changes in behavior as they occur, avoid creating breaking changes in the first place whenever possible, and responsibly update your dependencies instead of blindly doing so.

Basically, Romantic Versioning, not Semantic Versioning.

All that said, okay, okay, fine — Underscore 1.7.0 can be Underscore 2.0.0. Uncle.

(typed in haste, excuse any grammar-os, will correct later)

[felixrabe]

Kudos Jeremy for making this happen. So will there be a 1.7.1 reverting the (breaking) changes? (Or reverting to pre-1.7.0?) That'd be double-awesome.

[lwansbrough]

+1 @jashkenas, thanks for listening.

[jashkenas]

1.7.1 — then 2.0.0 when @megawac is ready to roll with it.

[halfdan]

Node doesn't follow SemVer

Just to clarify, node is pre 1.0 which allows to break without bumping major version numbers. See point 4 of semver.

Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.

[alsotang]

just use lodash :)

[harrysingh]

when is the version 1.7.11 solving the problems of 1.7.0 expected to be released?

[jdalton]

@harrysingh The bump is being discussed over on #1855.

[acstll]

This is the most useless thread I've read, ever.
Intentionally ignoring semver at this point in time is just a very bad thing.

[linus-amg]

(ノಠ益ಠ)ノ彡┻━┻

[bobmagicii]

if i saw a 0.x.0 update i sure as hell would be reading changelogs...

https://www.npmjs.com/package/npm-check

[ViniciusSasso]

Underscore and Backbone are so dead now...lol

[acstll]

Dead or alive they deserve respect.