Parsing bytes from reader can lead to expected a borrowed string error

[mdibaiee]

Hello!

We are using pbjson on our prost messages that look like this:

https://github.com/estuary/flow/blob/master/crates/proto-flow/src/capture.rs#L191-L192

When I try to deserialize a JSON string that looks like this (all other fields omitted for brevity)

{
  "open": {
    "driverCheckpoint": "e30=",
  }
}

I'm getting this error from the deserializer:

`Err` value: invalid type: string "e30=", expected a borrowed string at line 25 column 30'

My understanding is that this is because of this line: https://github.com/influxdata/pbjson/blob/main/pbjson/src/lib.rs#L67

After searching a bit, I found these:

• Deserializer expecting a &str can't handle owned strings? serde-rs/serde#1009
• https://stackoverflow.com/questions/60801133/how-do-i-use-serde-to-deserialize-structs-with-references-from-a-reader

My understanding is that I'm getting this error because I'm deserializing from a Reader, namely stdin:

let request: PullRequest = serde_json::from_reader(stdin)?;

This means that the underlying data for deserialization (the bytes from stdin) are destroyed as soon as they are read, therefore a reference to this data will not be valid once the data is dropped. In this case, we would have to deserialize to an owned String.

The workaround at the moment is to read the whole value from Reader, and then deserialize:

let mut buf = Vec::new();
stdin.read_to_end(&mut buf)?;
let request: PullRequest = serde_json::from_slice(&buf)?;

[tustvold]

Thank you for the report, I'm a little bit swamped at the moment with other projects so I'm not sure when I'll have a chance to get to this, but if you, or anyone else, felt able to propose a fix I would be more than happy to give it a review 😄

[vthib]

I have stumbled upon this issue as well, when deserializing from an opened file. I have pushed a PR to fix this, as well as other issues that arose when I added the serde_json::from_reader check on all the kitchen sink tests: #71