Encode a struct with ",remain" back to a map?

I have a use case where I need to decode a JSON, where I care only about a few fields, change it here and there, and then serialize it back to JSON. However, the remainder values seem to get serialized into an empty "" JSON field. This is my code:

func TestDecodeEncode(t *testing.T) {
	var stuff struct {
		Field string                 `mapstructure:"field"`
		Other map[string]interface{} `mapstructure:",remain"`
	}
	err := Decode(strings.NewReader(`{
		"field": "hello",
		"anotherField": "world"
	}`), &stuff)
	if err != nil {
		panic(err)
	}
	bs, err := Encode(stuff)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%s\n", bs)
}

func Decode(r io.Reader, result interface{}) error {
	var m interface{}
	if err := json.NewDecoder(r).Decode(&m); err != nil {
		return err
	}
	return mapstructure.Decode(m, result)
}

func Encode(v interface{}) ([]byte, error) {
	var m map[string]interface{}
	if err := mapstructure.Decode(v, &m); err != nil {
		return nil, err
	}
	return json.Marshal(m)
}

And the output is:

{"":{"anotherField":"world"},"field":"hello"}

Am I missing some configuration option or is remain just not supported when encoding a struct into a map?

I'm using mapstructure version v1.4.3.

For now I'm using this simple workaround:

func Encode(v interface{}) ([]byte, error) {
	var m map[string]interface{}
	if err := mapstructure.Decode(v, &m); err != nil {
		return nil, err
	}
	expandRemainderValues(m)
	return json.Marshal(m)
}

func expandRemainderValues(m map[string]interface{}) {
	for k, v := range m {
		v, ok := v.(map[string]interface{})
		if !ok {
			continue
		}
		if k == "" {
			for remainderK, remainderV := range v {
				m[remainderK] = remainderV
			}
			delete(m, "")
		} else {
			expandRemainderValues(v)
		}
	}
}

Looks like the behavior has changed in 1.5.0. Now my Other fields are serialized not as "", but as "Other". (It doesn't solve the problem of course, but requires changing the workaround.)

Yeah, there was a bug fixed in 1.5.0 that caused some struct fields to get an empty string in the map. So you're seeing that fix, while still retaining your bug. We just need to turn your repro into a test case, and get a fix on top of it.

Fix Encoding Struct Back to Map Bug (mitchellh#279)

#307 looks like a nice fix for this issue.

@mitchellh

@mitchellh can we merge #307? it's a tested by prod fix :)

Merge pull request #8 from go-viper/decode-remaining-fields

Navigation Menu

Repository navigation

Encode a struct with ",remain" back to a map? #279

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Encode a struct with ",remain" back to a map? #279

Description

Activity

kszafran commented on Apr 7, 2022

kszafran commented on Apr 21, 2022

mitchellh commented on Apr 21, 2022

tschaub commented on Mar 7, 2023

Arsen204 commented on May 28, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions