Why Go’s yaml.v3 Fails to Convert YAML with Duplicate Keys to JSON (and How to Fix It)

While working on a feature that uploads YAML/JSON documents and converts them all to JSON for unified processing, I encountered a small issue where Go's yaml.v3 package raised an error during YAML‑to‑JSON conversion.

Using yaml.v3 to Process YAML

Assume the following YAML document:

object:
  a: 1
  1: 2
  "1": 3
  key: value
  array:
    - null_value:
    - boolean: true
    - integer: 1

The document contains an object with an array, and two keys that look similar: a numeric 1 and a string "1". In Go we can parse the YAML with yaml.v3 and then marshal it to JSON using the standard encoding/json package.

package main

import (
    "encoding/json"
    "fmt"
    "log"

    "github.com/icza/dyno" // recursive map[interface{}] → map[string]
    "gopkg.in/yaml.v3"
)

func main() {
    yamlExample := `
object:
  a: 1
  1: 2
  "1": 3
  key: value
  array:
  - null_value:
  - boolean: true
  - integer: 1
`
    var data interface{}
    if err := yaml.Unmarshal([]byte(yamlExample), &data); err != nil {
        log.Fatalf("YAML parse error: %v", err)
    }
    fmt.Printf("Type: %T
Value: %#v
", data, data)
    fmt.Println("--------------------------")
    convertedData := dyno.ConvertMapI2MapS(data)
    jsonData, err := json.Marshal(convertedData)
    if err != nil {
        log.Fatalf("JSON convert error: %v", err)
    }
    fmt.Printf("Type: %T
Value: %s
", jsonData, jsonData)
}

Running the code produces a parsing error because the keys 1 and "1" are considered duplicate:

$ go run main.go
2025/07/31 23:22:49 YAML parse error: yaml: unmarshal errors:
  line 5: mapping key "1" already defined at line 4
exit status 1

Commenting out the "1": 3 entry resolves the error, and the conversion succeeds.

$ go run main.go
Type: map[string]interface {}
Value: map[string]interface {}{"object":map[interface {}]interface {}{"a":1, "array":[]interface {}{map[string]interface {}{"null_value":interface {}(nil)}, map[string]interface {}{"boolean":true}, map[string]interface {}{"integer":1}}, "key":"value", 1:2}}
--------------------------
Type: []uint8
Value: {"object":{"1":2,"a":1,"array":[{"null_value":null},{"boolean":true},{"integer":1}],"key":"value"}}

The root cause is that yaml.Unmarshal returns maps with interface{} keys ( map[interface{}]interface{}), which json.Marshal cannot handle because JSON object keys must be strings. The helper function dyno.ConvertMapI2MapS recursively converts those maps to map[string]interface{}, enabling successful JSON marshaling.

Using go‑yaml (goccy/go‑yaml)

The go‑yaml library works similarly; swapping the import statement is enough. However, it also fails on duplicate keys unless the offending line is commented out.

$ go run goccy-go-yaml/main.go
2025/07/31 23:25:15 YAML parse error: [5:3] mapping key "1" already defined at [4:3]
...

After commenting out the duplicate key, the conversion succeeds just like with yaml.v3.

Python handling of YAML‑to‑JSON

Python’s yaml.safe_load parses the same document without error, and json.dumps produces a JSON string that contains both "1": 2 and "1": 3. When the JSON string is parsed back, the later key overwrites the former because JSON object keys must be unique strings.

Why the difference?

YAML allows arbitrary types as mapping keys, including numbers, while JSON requires keys to be strings. Go’s json.Marshal enforces the JSON rule, so the intermediate map[interface{}]interface{} must be converted to map[string]interface{}. The dyno.ConvertMapI2MapS function performs this conversion recursively.

YAML and JSON specifications

YAML is a superset of JSON; any valid JSON is valid YAML, but not vice‑versa. The JSON specification explicitly states that object keys must be strings, which explains the observed behavior.

Conclusion

When converting YAML to JSON in Go, always convert map[interface{}]interface{} to map[string]interface{} (e.g., with dyno.ConvertMapI2MapS) before calling json.Marshal. This avoids duplicate‑key errors and aligns with JSON’s string‑key requirement.