Cannot unmarshal ISO-8859-1 encoded XML

[mjeco]

Trying to use the automatic unmarshaling of XML with SetResult() is failing for me as the XML in the response has encoding="ISO-8859-1" set.

Fails with the following error:

xml: encoding "ISO-8859-1" declared but Decoder.CharsetReader is nilexit status 1

Response example:

<?xml version="1.0" encoding="ISO-8859-1"?>
<Response stat="200" code="ok">
  <Success>
    <Data>
      <commands>
        <cmd>get</cmd>
        <cmd>set</cmd>
      <commands>
    </Data>
  </Success>
</Response>

I am able to successfully unmarshal the above XML response with the below function.

import (
  "golang.org/x/net/html/charset"
)

func UnmarshalXML(data []byte, result interface{}) error {
	decoder := xml.NewDecoder(bytes.NewReader(data))
	decoder.CharsetReader = charset.NewReaderLabel
	err := decoder.Decode(&result)
	return err
}

Is it possible to set the CharsetReader to be used for the XML unmarshal?

[mjeco]

I got it to work by updating the utils.Unmarshalc function as follows.

import (
+    "golang.org/x/net/html/charset"
)

func Unmarshalc(c *Client, ct string, b []byte, d interface{}) (err error) {
	if IsJSONType(ct) {
		err = c.JSONUnmarshal(b, d)
	} else if IsXMLType(ct) {
+             decoder := xml.NewDecoder(bytes.NewReader(b))
+             if strings.Contains(string(b), "encoding=\"ISO-8859-1\"") {
+                   decoder.CharsetReader = charset.NewReaderLabel
+             }
+             err = decoder.Decode(d)
-              err = xml.Unmarshal(b, d)
	}

	return
}

Is this something that could be added to the code ?

[jeevatkm]

@mjeco I think, it's better to expose XML marshal & unmarshal for the override similar to JSON. So that resty user can inject the custom function.
FYI, doing bytes -> string, and string comparing may not be performant.

[jeevatkm]

@mjeco After v2.7.0 release, you can do as following. Also charset string condition may not be needed.

client := resty.New()

client.XMLUnmarshal = func(data []byte, v interface{}) error {
   decoder := xml.NewDecoder(bytes.NewReader(data))
   decoder.CharsetReader = charset.NewReaderLabel
   err = decoder.Decode(v)
}

[mjeco]

@jeevatkm Allowing to set a custom unmarshal function for XML is the best path forward. Thanks for this!

[VinGarcia]

I had this same issue, and the above solutions didn't work for me (probably because the "encoding" was not set explicitly on the XML I was working with), so I actually found a library that just converts from ISO-8859-1 to UTF-8, my code ended up looking something like this:

import "golang.org/x/text/encoding/charmap"

// ...
func someFunc(r io.Reader) (MyDTO, err) {
	decoder := xml.NewDecoder(charmap.ISO8859_1.NewDecoder().Reader(f))
	var myDTO MyDTO
	err := decoder.Decode(&myDTO)
	return myDTO, err
}

[Carmendelope]

@VinGarcia , your solution doesn't work for me, it always returns an EOF error. Do you know what could be happening?

[VinGarcia]

@Carmendelope the only reason this would happen is if you io.Reader has already been read, you should be able to find help for this on a Golang group on discord, telegram or slack. You could also replace the .Reader(f) thing with .Bytes(yourByteArray) if you have the bytes available.