Arbitrary precision numbers don't work with serde(tag = ..) and serde(flatten)

[idubrov]

The code below fails if arbitrary_precision feature is turned on (works fine if it is turned off):

#[macro_use]
extern crate serde_derive;

#[derive(Deserialize)]
struct Data {
    value: i32,
}

#[derive(Deserialize)]
#[serde(tag = "type")]
enum Wrapper {
    Data(Data),
}

fn main() {
    let json = r#"{"type":"Data","value":123}"#;
    // Okay
    let _data1: Data = serde_json::from_str(json).unwrap();
    // Fails!
    let _data2: Wrapper = serde_json::from_str(json).unwrap();
}

Error backtrace:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: map, expected i32", line: 0, column: 0)', libcore/result.rs:1009:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227

Cargo.toml:

[package]
name = "serde-arbitrary-fail"
version = "0.1.0"
authors = ["Ivan Dubrov <[email protected]>"]

[dependencies]
serde = "1.0.80"
serde_derive = "1.0.80"
serde_json = { version = "1.0.33", features = ["arbitrary_precision"]}

[igorbernstein]

It also fails for #[serde(untagged)]

data did not match any variant of untagged enum Wrapper

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("data did not match any variant of untagged enum Wrapper", line: 0, column: 0)', src/libcore/result.rs:997:5
stack backtrace:

[idubrov]

Same for #[flatten]:

use serde_derive::{Serialize, Deserialize};

#[derive(Debug, Serialize, Deserialize)]
struct Outer {
    #[serde(flatten)]
    inner: Inner,
}

#[derive(Debug, Serialize, Deserialize)]
struct Inner {
    value: usize,
}

fn main() {
    let v = Outer {
        inner: Inner {
            value: 10,
        }
    };
    let json = serde_json::to_string(&v).unwrap();
    let v2: Outer = serde_json::from_str(&json).unwrap();
    println!("{:#?}", v2);
}

Output:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: map, expected usize", line: 1, column: 12)', src/libcore/result.rs:997:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:70
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:478
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:385
   6: std::panicking::try::do_call
             at src/libstd/panicking.rs:312
   7: core::char::methods::<impl char>::escape_debug
             at src/libcore/panicking.rs:85
   8: core::result::unwrap_failed
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/macros.rs:16
   9: <core::result::Result<T, E>>::unwrap
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/result.rs:798
  10: ttt::main
             at src/main.rs:21
  11: std::rt::lang_start::{{closure}}
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  12: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:297
  13: panic_unwind::dwarf::eh::read_encoded_pointer
             at src/libpanic_unwind/lib.rs:92
  14: <std::panicking::begin_panic::PanicPayload<A> as core::panic::BoxMeUp>::get
             at src/libstd/panicking.rs:276
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  15: std::rt::lang_start
             at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  16: <ttt::Inner as core::fmt::Debug>::fmt

Cargo.toml:

[package]
name = "ttt"
version = "0.1.0"
edition = "2018"

[dependencies]
serde_json = { version = "1.0", features = ["arbitrary_precision"]}
serde = "1.0"
serde_derive = "1.0"

[idubrov]

Well, the workaround is as simple as deserializing to serde_json::Value first:

#[macro_use]
extern crate serde_derive;

#[derive(Deserialize)]
struct Data {
    value: i32,
}

#[derive(Deserialize)]
#[serde(tag = "type")]
enum Wrapper {
    Data(Data),
}

fn main() {
    let json = r#"{"type":"Data","value":123}"#;
    // Okay
    let _data1: Data = serde_json::from_str(json).unwrap();
    // Fails!
    //let _data2: Wrapper = serde_json::from_str(json).unwrap();
    // Works!
    let data2: serde_json::Value = serde_json::from_str(json).unwrap();
    let _data2: Wrapper = serde_json::from_value(data2).unwrap();
}

Not great, but okay for our purposes.

I think, what happens is serde JSON parser is confused when it is asked to deserialize into this Content (used internally by serde for tag / flatten processing). It thinks it needs to do an "arbitrary number" and yield a map, but in reality, it is not (parts of serde outside of Serde JSON don't know how to process that "arbitrary precision number map").

[idubrov]

Demonstration of what happens. The following code:

#[macro_use]
extern crate serde_derive;

#[derive(Debug, Deserialize)]
struct SneakyData {
    value: std::collections::HashMap<String, String>,
}

#[derive(Debug, Deserialize)]
#[serde(tag = "type")]
enum SneakyWrapper {
    Data(SneakyData),
}

fn main() {
    let json = r#"{"type":"Data","value":123}"#;
    // Sneaky!
    let data: SneakyWrapper = serde_json::from_str(json).unwrap();
    println!("{:#?}", data);
}

prints:

Data(
    SneakyData {
        value: {
            "$serde_json::private::Number": "123"
        }
    }

[marcelbuesing]

Thanks for the workaround I also ran in to this issue.

[marcelbuesing]

When having a u128 type in the struct i ran into a combination of the previously reported and u128 is not supported (although I'm using the arbitrary_precision feature).

See here:

marcelbuesing@dab262c#diff-9a86b31b12f3b48a02818b844654ab0cR35

test regression::issue505::test_i32 ... FAILED
test regression::issue505::test_u128 ... FAILED

failures:

---- regression::issue505::test_i32 stdout ----
thread 'regression::issue505::test_i32' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: map, expected i32", line: 0, column: 0)', src/libcore/result.rs:999:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

---- regression::issue505::test_u128 stdout ----
thread 'regression::issue505::test_u128' panicked at 'called `Result::unwrap()` on an `Err` value: Error("u128 is not supported", line: 0, column: 0)', src/libcore/result.rs:999:5

[vmalloc]

Also hitting this issue. Any estimate on a possible fix?

[dtolnay]

Closing as a duplicate of serde-rs/serde#1183. This is not an issue with arbitrary precision, it is primarily an issue with the tag, untagged, and flatten derive attributes. Ideally you should avoid using those attributes until someone submits a PR fixing them.