Why Converting Between Go string and []byte Costs More Than You Think

Why do conversions between string and []byte in Go have a cost?

What is a string?

The Go built‑in string type is defined as a struct containing a pointer to the underlying byte array and its length. The data is immutable: the pointer references read‑only memory, and the struct cannot be nil.

type stringStruct struct {
    str unsafe.Pointer
    len int
}

When a string is created, the runtime allocates memory for the byte array and stores the pointer and length in this struct.

What is []byte ?

In Go, byte is an alias for uint8. A slice is also a struct with a pointer to an array, a length, and a capacity.

type slice struct {
    array unsafe.Pointer
    len   int
    cap   int
}

The slice’s underlying array can be modified, which makes []byte more flexible than string.

Key differences

String values are immutable

Assigning a new literal to a string creates a new memory block and updates the struct’s pointer; the original bytes remain unchanged. By contrast, a []byte slice can modify the contents of its underlying array directly.

s := "A1" // memory for "A1" allocated, pointer set
s = "A2" // new memory allocated, pointer updated

s := []byte{1} // memory for array allocated, pointer set
s = []byte{2} // contents of the same array changed

Because a string’s bytes are read‑only, each modification forces a new allocation and a subsequent garbage‑collection, which is the main source of inefficiency.

Conversion between string and []byte

Converting a string to a byte slice uses the built‑in []byte(string) conversion, which allocates a new slice and copies the data.

func stringtoslicebyte(buf *tmpBuf, s string) []byte {
    var b []byte
    if buf != nil && len(s) <= len(buf) {
        *buf = tmpBuf{}
        b = buf[:len(s)]
    } else {
        b = rawbyteslice(len(s))
    }
    copy(b, s)
    return b
}

The runtime implements a special slicestringcopy function to copy the string’s bytes efficiently.

Converting a byte slice to a string ( string([]byte)) also allocates a new string and copies the slice’s contents.

func slicebytetostring(buf *tmpBuf, b []byte) string {
    l := len(b)
    if l == 0 {
        return ""
    }
    // race and msan checks omitted for brevity
    s, c := rawstringtmp(buf, l)
    copy(c, b)
    return s
}

Both directions involve memory allocation, which can be noticeable in tight loops.

Unsafe zero‑copy tricks

It is possible to reinterpret the underlying data without copying using the reflect and unsafe packages:

func stringtoslicebyte(s string) []byte {
    sh := (*reflect.StringHeader)(unsafe.Pointer(&s))
    bh := reflect.SliceHeader{Data: sh.Data, Len: sh.Len, Cap: sh.Len}
    return *(*[]byte)(unsafe.Pointer(&bh))
}

func slicebytetostring(b []byte) string {
    bh := (*reflect.SliceHeader)(unsafe.Pointer(&b))
    sh := reflect.StringHeader{Data: bh.Data, Len: bh.Len}
    return *(*string)(unsafe.Pointer(&sh))
}

These methods share the same memory between the string and the slice, so modifying the slice can corrupt the read‑only string memory and crash the program; they should be used only when you fully understand the risks.

How to choose?

Consider the following when deciding between string and []byte:

Strings can be compared directly and used as map keys; byte slices cannot.

If you need to modify individual characters, use []byte.

A string can never be nil; if you need a nilable value, use []byte.

Byte slices provide flexible slicing and capacity management.

For heavy text processing, []byte often offers better performance.

Ultimately, choose the type that best fits the specific use case and performance requirements.

Source: https://www.cnblogs.com/zhangboyu/p/7623712.html (© original author)