encoding/xml: CDATA section allows invalid characters

[firodj]

What version of Go are you using (go version)?

go version go1.16.15 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

What did you do?

A simple main.go program:

package main

import (
	"encoding/xml"
	"os"
)

func main() {
	type Person struct {
		Data    string `xml:",cdata"`
	}

	v := &Person{}
	v.Data = "\x1b[1;33mWARN\x1b[0m I dont know what to do"

	output, _ := xml.MarshalIndent(v, "  ", "    ")
	os.Stdout.Write(output)
}

Run and store the output:

$ go run main.go > output.xml

Try validate the xml:

$ xmllint output.xml

What did you expect to see?

encode error, or
no error with invalid character being replaced

What did you see instead?

encode success, but xmllint report parse error

Reference:

• https://www.w3.org/TR/xml/#NT-Char

[heschi]

cc @rsc

[alexxed]

This is a general issue, not just with cdata:

package main

import (
	"encoding/xml"
	"fmt"
)

func main() {
	data := "\x1b[31m; some colour\x1b[0m;"
	fmt.Printf("original data: %s\n", data)
	out, _ := xml.MarshalIndent(data, "", "  ")
	fmt.Printf("data after Marshal: %s\n", string(out))
	xml.Unmarshal(out, &data)
	fmt.Printf("data after Unmarshal:%v\n", data)

}

produces a broken string losing the original value

[alexxed]

looks like this old commit did the breakage f74eb6d#diff-0526795219b1a2003e16773ef66844f384410205af46abac3edd309b22b88aeeR704-R705 as a fix to #4235

[alexxed]

my expectation would be that \x1b gets encoded to  in xml, then on unmarshal it gets back to \x1b maintaining the original value