#[serde(flatten)] does not work when deserializing twice nested struct with a capturing map

[dennsva]

This is probably related to the issues with combining flatten and deny_unknown_fields. However, a slightly different variant with only flatten and no enums:

use std::collections::BTreeMap;

use serde::Deserialize;

#[derive(Debug, Deserialize)]
struct Inner {
    inner_field: u32,
}

#[derive(Debug, Deserialize)]
struct Middle {
    middle_field: u32,
    #[serde(flatten)] // comment out these two
    inner: Inner, // lines and it works as expected
}

#[derive(Debug, Deserialize)]
struct Outer {
    outer_field: u32,
    #[serde(flatten)]
    middle: Middle,
    #[serde(flatten)] // or comment out these two
    map: BTreeMap<String, u32>, // lines and it works as expected
}

fn main() {
    let json = r#"
{
    "outer_field": 0,
    "middle_field": 1,
    "inner_field": 2,
    "foo": 3
}
    "#;

    let result: Outer = serde_json::from_str(json).unwrap();

    println!("{result:?}");
}

I would expect only "foo" to appear in the map:

// Expected output:
Outer { outer_field: 0, middle: Middle { middle_field: 1, inner: Inner { inner_field: 2 } }, map: {"foo": 3} }

Instead, I get this

// Actual output:
Outer { outer_field: 0, middle: Middle { middle_field: 1, inner: Inner { inner_field: 2 } }, map: {"foo": 3, "inner_field": 2, "middle_field": 1} }

Commenting out either the map or the struct Inner, it works as expected.

Versions:

serde = { version = "1.0.203", features = ["derive"] }
serde_json = "1.0.118"

[Mingun]

Yes, that is a known bug, that FlatMapDeserializer does not consume data in some circumstances and it is difficult to fix it with current flatten architecture. The problem specifically in this method:

serde/serde/src/private/de.rs

Lines 2759 to 2775 in 9e6158e

	fn next_key_seed<T>(&mut self, seed: T) -> Result<Option<T::Value>, Self::Error>
	where
	T: DeserializeSeed<'de>,
	{
	for item in &mut self.iter {
	// Items in the vector are nulled out when used by a struct.
	if let Some((ref key, ref content)) = *item {
	// Do not take(), instead borrow this entry. The internally tagged
	// enum does its own buffering so we can't tell whether this entry
	// is going to be consumed. Borrowing here leaves the entry
	// available for later flattened fields.
	self.pending_content = Some(content);
	return seed.deserialize(ContentRefDeserializer::new(key)).map(Some);
	}
	}
	Ok(None)
	}

To be precise, problems will be with any types that use deserialize_map for deserialization (any maps and any #[serde(flatten)] types), and will not be with types that use deserialize_struct. So your issue is duplicate of #2200 because of that.

[dennsva]

Thank you for the response!