Use crates.io version of rustc-serialize

[nrc]

We must ensure that the crates.io version has all the functionality the internal one does. Apparently the only thing missing is https://github.com/rust-lang/rust/blob/master/src/libserialize/serialize.rs#L748-L815

Then we just need to change the deps in the Cargo.tomls and remove the crate.

Motivation is that I want save-analysis to depend on an external crate which in turn depends on rustc-serialize (the crates.io version). (We can't use Serde because of complex dependencies on the proc macro crate). That means we will pull in the crates.io version, so we may as well just have that, rather than two versions kicking around (and two sources of truth, etc.).

cc #40177

[alexcrichton]

There's an attempt to do this for liblog as well, but that has issues I'll need to address before it lands.

[alexcrichton]

(oops didn't mean to close)

[nrc]

So, thinking about this, we're somewhat getting into the question of the future of librustc-serialize. The internal and external variants have diverged somewhat, most notably in #36551. The internal version now uses unstable code, whereas the external version is stable.

So a few courses of possible action:

• we un-stabilise the external version and bring it up to date. The philosophy here is that most users should use Serde and rustc-serialize is just for use in rustc, its just hosted outside the repo for ease of development
• we land the changes as Cargo features, so you can opt in to the unstable stuff
• we re-implement the changes using macros (@eddyb suggests this is possible in the PR). The philosophy in these two cases is that rustc-serialize is a general purpose serialisation library that happens to be used by the compiler for some specific stuff that most people won't bother themselves with.
• we could abandon the external version and only use the internal version. This breaks some things I'm doing with save-analysis, but perhaps there are work-arounds.

@rust-lang/libs @eddyb thoughts?

[sfackler]

I'd feel a bit uncomfortable about declaring the external rustc-serialize "unstabilized" - it's still being downloaded 11,000 times a day!

The specialization stuff does seem like it'd be reasonable to have under a Cargo feature, though.

[sfackler]

It seems like the thing we should really be doing is switching over to serde, right?

[nrc]

@sfackler see discussion in #40177 tl;dr: rustc-serialize does some stuff that Serde doesn't and which it doesn't make sense to move into Serde. We might consider using Serde for JSON stuff and rustc-serialize (or something new) for metadata, etc.

[sfackler]

Cool, makes sense.

[alexcrichton]

I don't personally quite have a handle on the difference between the internal and external versions. @eddyb could you clarify what they are?

(that'd help guide my opinion at least on what to do here)

[eddyb]

The specialization stuff does seem like it'd be reasonable to have under a Cargo feature, though.

It's not, it's actually abusing specialization unsoundness. #40177 and #36588 are relevant.
@nrc Is it possible for save-analysis and effectively anything involving JSON to switch to serde?

If so, the only blocker is me figuring out how to get serde working with the compiler bootstrapping.
How high-priority is this, i.e. what's the timeline here? @nikomatsakis also wants me to investigate moving off of rustc-serialize, for a different reason (incremental/on-demand, mostly).

[nrc]

@eddyb we currently have problems with external crates using Serde - because it links against the proc_macro crate and thus there are build order issues. Also because we'd need to support proc macros with a stage 1 build in order to build the stage 2 compiler. So, basically moving to Serde for JSON stuff is a lot more effort than it sounds like :-(

[eddyb]

@nrc Right, that's what I mean, I'm supposed to solve those problems... somehow!

[steveklabnik]

Triage: still using an internal version.

[Xanewok]

@eddyb with #49219 merged, is it possible to do the switch to crates.io version now?

[Xanewok]

#60053 switches save-analysis to use serde, which is the only usage of crates.io version of rustc_serialize in the Rust tree IIRC.

Because of this, I believe we wouldn’t have to switch to crates.io version for other usages if we need the unstable/internal API in the compiler for rustc_serialize usage in-tree?

[jonas-schievink]

This feels like... something we should just close at this point, maybe? The unsound specialization abuse shouldn't really be on crates.io, and even without that the benefits of updating it are questionable as almost everyone has moved off of rustc-serialize by now.

[Mark-Simulacrum]

I would be in favor of closing this, yeah, it doesn't seem useful to track separately (we probably do want a bug for the soundness hole we're exploiting? Not sure what issue that is). Going to go ahead and do that and if someone disagrees feel free to reopen.