Occasional invalid JSON returned by Stripe

[craigandrews]

Very occasionally (once every few hundred thousands requests) I will receive an apparently nonsensical error from charge.New(). Errors such as:

invalid character 'o' in literal null (expecting 'u')

and

invalid character 'u' looking for beginning of value

After a bit of head-scratching I realised that these are actually JSON unmarshaling errors so I had a look in stripe.go and found this at line 437:

	if res.StatusCode >= 400 {
		return s.ResponseToError(res, resBody)
	}

	s.LeveledLogger.Debugf("Response: %s\n", string(resBody))

	if v != nil {
		return json.Unmarshal(resBody, v)
	}

	return nil

and in ResponseToError at line 451:

func (s *BackendImplementation) ResponseToError(res *http.Response, resBody []byte) error {
	var raw rawError
	if err := json.Unmarshal(resBody, &raw); err != nil {
		return err
	}

The problem here is that invalid JSON is being returned from the remote API but the client code is making no effort at exposing it. And if the response is a 400 or worse, it doesn't even log the actual body to the debug log.

It would be useful to be able to get the actual raw response from the request so it can be logged and we can understand why we're seeing failures, rather than just cryptic notes about unexpected vowels.

[remi-stripe]

@candrewsmtn Thank you for the report! In theory, our API should always return a JSON string for the error which is why the library expects this explicitly. In some really rare cases, it's possible for the request to fail in a way that a non-json error is returned. That error is unlikely to be helpful since it would certainly be some default HTML page error or similar or a truncated response but I do agree that logging it could be useful.

Tagging as future for @brandur-stripe to maybe investigate.

[brandur-stripe]

It would be useful to be able to get the actual raw response from the request so it can be logged and we can understand why we're seeing failures, rather than just cryptic notes about unexpected vowels.

That error message originates directly from Go core, but yeah, we should do a better job of at least logging more detail about the problem. I'll write a patch for that.

[craigandrews]

It would be good if the error message was replaced with something more useful. Just an indication of the HTTP status code and the fact it could not be parsed would help enormously to work out if the issue is with us, stripe, or the network in between.

[brandur-stripe]

We just released 60.4.0, which logs some more flavor in case of a failure like this.

@candrewsmtn Would you mind trying that and checking back in if you discover something concerning is revealed?

[craigandrews]

Sure, will do

…

On Mon, 15 Apr 2019, 22:01 Brandur, ***@***.***> wrote: We just released 60.4.0, which logs some more flavor in case of a failure like this. @candrewsmtn <https://github.com/candrewsmtn> Would you mind trying that and checking back in if you discover something concerning is revealed? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#830 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Aq2ixWa6Pejm8nvYz_UYAXdBQZDXnut0ks5vhOimgaJpZM4cqzmu> .

[layby42]

Hi Everybody

attempt to get customer resulted in Stripe error:

Couldn't deserialize JSON (response status: 503, body sample: 'upstream connect error or disconnect/reset before headers'): invalid character 'u' looking for beginning of value

happened only once so far, cannot reproduce.

stripe-go: 60.13.1