map[interface{}]interface{} on a map[string]interface{}

[aanm]

Sorry to be persistent with this. I wasn't aware that yaml was able to have different types has keys than string. However, with the following example I see that newstruct is being considered has a map[interface{}]interface{}, is it a bug or is something I should expect?

type=string key=something type=string value=thing 
type=string key=newstruct type=map[interface {}]interface {} value=map[otherstruct:[map[foo-bar:12345]] bar:bar]

var data = `---
foo-map:
  something: "thing"
  newstruct:
    otherstruct:
      -
        foo-bar: 12345
    bar: "bar"`

type FooStruct struct {
    FooMap map[string]interface{} `yaml:"foo-map,omitempty"`
}

func main() {
    var foo FooStruct
    err := yaml.Unmarshal([]byte(data), &foo)
    if err != nil {
        fmt.Printf("error %+v", err)
    }
    for k, v := range foo.FooMap {
        fmt.Printf("type=%+v key=%+v type=%+v value=%+v\n", reflect.TypeOf(k), k, reflect.TypeOf(v), v)
    }
}

[danprince]

I'm having trouble with this too. Even if I unmarshal into a map[string]interface{} the nested maps are of type map[interface{}]interface{}, meaning I can't then marshal to JSON (which must have string keys).

Is it possible to analyse the type of the interface passed to unmarshal then use the same type for recursively unmarshalling all the way down?

[magnusbaeck]

Is it possible to analyse the type of the interface passed to unmarshal then use the same type for recursively unmarshalling all the way down?

I'm not sure that would work in all cases. I also had this problem and made a custom unmarshaller in github.com/elastic/beats:libbeat/common/mapstr.go that converts the keys of nested maps to strings so that the unmarshalled data can always be marshalled into JSON.

[sontags]

The approach of @magnusbaeck works like a charm! I changed unmarshalYAML a bit as well as cleanUpMapValue in order to fit my needs (eg. it has now the same signature as yaml.Unmarshal() and ints, bool et al are not quoted since this works fine with encoding/json)...

https://github.com/unprofession-al/gerty/blob/master/api/helpers.go

It would be nice to see this in go-yaml as "compatibility mode".

[maciejmrowiec]

+1

[ns-cweber]

+1

[ake-persson]

+1

[matiasinsaurralde]

+1

[ake-persson]

I modified the code from the suggestion above and it works nicely. Would be nicer not having to do it thought.

// Copyright (c) 2015-2016 Michael Persson
// Copyright (c) 2012–2015 Elasticsearch <http://www.elastic.co>
//
// Originally distributed as part of "beats" repository (https://github.com/elastic/beats).
// Modified specifically for "iodatafmt" package.
//
// Distributed underneath "Apache License, Version 2.0" which is compatible with the LICENSE for this package.

package yaml_mapstr

import (
    // Base packages.
    "fmt"

    // Third party packages.
    "gopkg.in/yaml.v2"
)

// Unmarshal YAML to map[string]interface{} instead of map[interface{}]interface{}.
func Unmarshal(in []byte, out interface{}) error {
    var res interface{}

    if err := yaml.Unmarshal(in, &res); err != nil {
        return err
    }
    *out.(*interface{}) = cleanupMapValue(res)

    return nil
}

// Marshal YAML wrapper function.
func Marshal(in interface{}) ([]byte, error) {
    return yaml.Marshal(in)
}

func cleanupInterfaceArray(in []interface{}) []interface{} {
    res := make([]interface{}, len(in))
    for i, v := range in {
        res[i] = cleanupMapValue(v)
    }
    return res
}

func cleanupInterfaceMap(in map[interface{}]interface{}) map[string]interface{} {
    res := make(map[string]interface{})
    for k, v := range in {
        res[fmt.Sprintf("%v", k)] = cleanupMapValue(v)
    }
    return res
}

func cleanupMapValue(v interface{}) interface{} {
    switch v := v.(type) {
    case []interface{}:
        return cleanupInterfaceArray(v)
    case map[interface{}]interface{}:
        return cleanupInterfaceMap(v)
    case string:
        return v
    default:
        return fmt.Sprintf("%v", v)
    }
}

[jveski]

+1

I'm currently maintaining a fork with a one line change to work around this issue.
Navops@85482c8

To be clear: I believe the library's design is correct given the way YAML handles boolean keys. However, I'd be willing to guess that a majority of implementations don't care about boolean keys and would rather have map keys default to strings.

Maybe exporting defaultMapType to allow implementations to mutate it (like my fork) is in order?

[rjeczalik]

Oh my, extremely sorry for this massive spam. Rebases ¯\(ツ)/¯

[toywei]

a.yaml

a.go

package main

import (
	"fmt"
	"io/ioutil"

	"gopkg.in/yaml.v3"
)

func main() {
	var err error
	m := make(map[string]interface{})
	var bs []byte
	bs, err = ioutil.ReadFile("a.yaml")
	if err != nil {
		panic(err)
	}
	err = yaml.Unmarshal(bs, &m)
	if err != nil {
		panic(err)
	}
	f1v := m["f1"].(string)
	var f3v string
	var f4v []int
	var f5v []string
	var f6v map[string]interface{}

	for k, v := range m["f2"].(map[string]interface{}) {
		switch k {
		case "f3":
			f3v = v.(string)
		case "f4", "f5":
			l := v.([]interface{})
			if k == "f4" {
				for _, u := range l {
					f4v = append(f4v, u.(int))
				}
			}
			if k == "f5" {
				for _, u := range l {
					f5v = append(f5v, u.(string))
				}
			}
		case "f6":
			f6v = v.(map[string]interface{})
		default:
		}
	}
	
	fmt.Println("---->", f1v, f3v, f4v, f5v, f6v)
}

go run *go
----> hi世界 value [42 1024] [a b] map[f7:77 f8:ok]

just use gopkg.in/yaml.v3 .

[giulianopz]

@aanm, @niemeyer, please close this issue since it was fixed with v3 (as noted by some comments above as well):

package main

import (
	"fmt"
	"reflect"

	"gopkg.in/yaml.v3"
)

var data = `---
foo-map:
  something: "thing"
  newstruct:
    otherstruct:
      -
        foo-bar: 12345
    bar: "bar"`

type FooStruct struct {
	FooMap map[string]interface{} `yaml:"foo-map,omitempty"`
}

func main() {
	var foo FooStruct
	err := yaml.Unmarshal([]byte(data), &foo)
	if err != nil {
		fmt.Printf("error %+v", err)
	}
	for k, v := range foo.FooMap {
		fmt.Printf("type=%+v key=%+v type=%+v value=%+v\n", reflect.TypeOf(k), k, reflect.TypeOf(v), v)
	}
        // output:
        // type=string key=newstruct type=map[string]interface {} value=map[bar:bar otherstruct:[map[foo-bar:12345]]]
        // type=string key=something type=string value=thing
}