encoding/json: encode panic

[xtxy]

What version of Go are you using (go version)?

$ go version
go version go1.19 linux/amd64

Does this issue reproduce with the latest release?

I think so

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/clark/.cache/go-build"
GOENV="/home/clark/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/clark/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/clark/go"
GOPRIVATE=""
GOPROXY="https://goproxy.io,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build858534376=/tmp/go-build -gno-record-gcc-switches"

What did you do?

json encode map[string]interface{}

data, err = json.Marshal(dataMap)
if err != nil {
logger.Error("msg.encode.error:", err)
return nil
}

What did you expect to see?

return data or error

What did you see instead?

occasionally, it panic:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x50f704]

goroutine 1 [running]:
encoding/json.(*encodeState).marshal.func1()
/usr/local/go/src/encoding/json/encode.go:327 +0x6e
panic({0x81e580, 0xba1d20})
/usr/local/go/src/runtime/panic.go:884 +0x212
encoding/json.(*encodeState).string(0xc07876e080, {0x0, 0x20f}, 0x1)
/usr/local/go/src/encoding/json/encode.go:1034 +0x64
encoding/json.stringEncoder(0xc07876e080, {0x8008c0?, 0xc07510d2b0?, 0x1?}, {0x0?, 0x0?})
/usr/local/go/src/encoding/json/encode.go:646 +0x20a
encoding/json.mapEncoder.encode({0xc071f41680?}, 0xc07876e080, {0x81c260?, 0xc084d3b650?, 0x81c260?}, {0x60?, 0x65?})
/usr/local/go/src/encoding/json/encode.go:813 +0x5f5
encoding/json.(*encodeState).reflectValue(0x812080?, {0x81c260?, 0xc084d3b650?, 0xc084d3b6b0?}, {0xc0?, 0x55?})
/usr/local/go/src/encoding/json/encode.go:359 +0x78
encoding/json.interfaceEncoder(0xc07876e080, {0x812080?, 0xc074c4f2d0?, 0x1?}, {0x0?, 0x4?})
/usr/local/go/src/encoding/json/encode.go:715 +0xc8
encoding/json.mapEncoder.encode({0x20301d?}, 0xc07876e080, {0x81bf60?, 0xc084d3b680?, 0x81bf60?}, {0x21?, 0x30?})
/usr/local/go/src/encoding/json/encode.go:813 +0x5f5
encoding/json.(*encodeState).reflectValue(0xc0731f7958?, {0x81bf60?, 0xc084d3b680?, 0xc0731f79a8?}, {0xa0?, 0x2e?})
/usr/local/go/src/encoding/json/encode.go:359 +0x78
encoding/json.(*encodeState).marshal(0x8006c0?, {0x81bf60?, 0xc084d3b680?}, {0x7?, 0xd8?})
/usr/local/go/src/encoding/json/encode.go:331 +0xfa
encoding/json.Marshal({0x81bf60, 0xc084d3b680})
/usr/local/go/src/encoding/json/encode.go:160 +0x45

[mvdan]

Please provide a way to reproduce this issue. Ideally, a small standalone program with the command you used to run it, or a working example on https://go.dev/play/.

[xtxy]

Please provide a way to reproduce this issue. Ideally, a small standalone program with the command you used to run it, or a working example on https://go.dev/play/.

I can't. The code is a segment in a complex server, and it doesn't happen every time.
If you can not find the reason by above call stack info, I can only add more log and wait for next panic

[mvdan]

The stack trace makes little sense because it points to https://cs.opensource.google/go/go/+/refs/tags/go1.19:src/encoding/json/encode.go;l=1034, which does not use any pointers at all, so it shouldn't end up in a nil pointer dereference.

The stack does show {0x0, 0x20f} as the string argument, which perhaps means that you somehow ended up with a string value that has a nil pointer in it. Are you using unsafe or cgo to make strings by any chance?

Another option is a data race causing memory corruption. Please try running your program with -race: https://go.dev/doc/articles/race_detector

[mvdan]

Whatever the case, unless I'm missing something obvious here, this doesn't look like a bug in the json package. I think you just have a corrupted string value.

[xtxy]

Whatever the case, unless I'm missing something obvious here, this doesn't look like a bug in the json package. I think you just have a corrupted string value.

no race, dataMap is a local variable in function.

you mean wrong string format will cause json marshal panic?

more relative codes:

`
infos := make(map[string]string)
for k, v := range client.games {
if v.accountInfo == "" {
continue
}

infos[k] = v.accountInfo
v.accountInfo = ""

}
if len(infos) == 0 {
logger.Warning("dyn accounts info is empty")
return
}

logger.Debug("send dyn accounts info to center")

cmd := common.Cmd{
Op: common.OP_DYN_ACCOUNT, Data: infos,
}

pkg := common.CreatePkg(cmd, common.PKG_TARGET_CENTER)
...

// function common.CreatePkg:
func CreatePkg(pkgData interface{}, target uint8) []byte {
if nil == pkgData {
return nil
}

if slice, ok := pkgData.([]byte); ok {
    return CreatePkgBySlice(slice, target)
}

dataMap, ok := pkgData.(map[string]interface{})
if !ok {
    cmd, ok := pkgData.(cmdMapI)
    if !ok {
        return nil
    }
    dataMap = cmd.ToMap()
}
if len(dataMap) == 0 {
    return nil
}

format, ok := parser.ToInt(dataMap, "Fmt")
if !ok {
    format = PKG_FMT_JSON
}

data := EncodePkgData(dataMap, uint8(format))
...

}

func EncodePkgData(dataMap map[string]interface{}, format uint8) []byte {
if nil == dataMap {
return nil
}

var data []byte
var err error

switch format {
case PKG_FMT_JSON:
    data, err = json.Marshal(dataMap) // panic here
    if err != nil {
        logger.Error("msg.encode.error:", err)
        return nil
    }
    ...

}

// Cmd structure:
type Cmd struct {
Op string
Data interface{}
GameId string
Ip string
From string
Fmt int
}

func (cmd Cmd) ToMap() map[string]interface{} {
return map[string]interface{}{
"Op": cmd.Op, "Data": cmd.Data,
"GameId": cmd.GameId, "Ip": cmd.Ip,
"From": cmd.From, "Fmt": cmd.Fmt,
}
}

type cmdMapI interface {
ToMap() map[string]interface{}
}
`

[bcmills]

@xtxy

no race, dataMap is a local variable in function.

The dataMap variable itself isn't the concern. The question is, where do the strings for its keys come from? (How does the caller of CreatePkg construct the map, and/or how does cmd.ToMap() construct it?)

[xtxy]

@xtxy

no race, dataMap is a local variable in function.

The dataMap variable itself isn't the concern. The question is, where do the strings for its keys come from? (How does the caller of CreatePkg construct the map, and/or how does cmd.ToMap() construct it?)

I don't know why code tag (<Ctrl+e>) didn't work, so, I use picture instead, hope it be clear:

[bcmills]

@xtxy See https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks for instructions on how to format code blocks in GitHub issues.

[mvdan]

I think it's time to move this out of the issue tracker; so far there is little evidence that there is a bug in encoding/json. See https://golang.org/wiki/Questions.

[xtxy]

@xtxy See https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks for instructions on how to format code blocks in GitHub issues.

thanks a lot

[caiweiwenjs]

@xtxy same issue for me. did you find the root cause? if so, could you share with me, thank a lot.
here is call stack in my case.

net/http.(*conn).serve.func1()
.../go-env/go1-19-linux-amd64/src/net/http/server.go:1850 +0xbf
panic({0x1292c60, 0x204bba0})
.../go-env/go1-19-linux-amd64/src/runtime/panic.go:890 +0x262
encoding/json.(*encodeState).marshal.func1()
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:327 +0x6e
panic({0x1292c60, 0x204bba0})
.../go-env/go1-19-linux-amd64/src/runtime/panic.go:884 +0x212
encoding/json.(*encodeState).string(0xc004305e00, {0x0, 0x20}, 0x1)
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:1034 +0x64
encoding/json.stringEncoder(0xc004305e00, {0x122c400?, 0xc0013934a0?, 0x20?}, {0x10?, 0x0?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:646 +0x20a
encoding/json.(*encodeState).reflectValue(0x1264bc0?, {0x122c400?, 0xc0013934a0?, 0xc000f965d0?}, {0x48?, 0x15?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:359 +0x78
encoding/json.interfaceEncoder(0xc004305e00, {0x1264bc0?, 0xc001393c80?, 0x1?}, {0x0?, 0x0?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:715 +0xc8
encoding/json.mapEncoder.encode({0x13f?}, 0xc004305e00, {0x1285200?, 0xc000f963f0?, 0x1285200?}, {0xc0?, 0xc1?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:813 +0x5f5
encoding/json.(*encodeState).reflectValue(0xc00176d0a8?, {0x1285200?, 0xc000f963f0?, 0xdd9320955363c06c?}, {0x60?, 0xc6?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:359 +0x78
encoding/json.(*encodeState).marshal(0xf?, {0x1285200?, 0xc000f963f0?}, {0x9d?, 0x56?})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:331 +0xfa
encoding/json.Marshal({0x1285200, 0xc000f963f0})
.../go-env/go1-19-linux-amd64/src/encoding/json/encode.go:160 +0x45