Boost Python Performance: Call Go Code via Shared Libraries

1. Introduction

Python is a highly productive language, but its Global Interpreter Lock (GIL) can become a bottleneck for CPU‑bound tasks. While Python excels at I/O‑bound workloads, developers often need a faster solution for heavy computations.

2. Calling Go from Python

Go (Golang) offers automatic memory management, built‑in concurrency, and easy compilation to C‑compatible shared objects. By compiling Go code into a shared library, Python can call performance‑critical functions without learning C/C++.

3. Test Environment

System: Windows

Python interpreter: Python 3.7.6 (64‑bit)

Go compiler: Go 1.14 (64‑bit)

4. Performance Comparison

We compare the execution time of summing numbers from 0 to 100,000,000 in pure Python and in Go.

Python code:

import time

def run(n):
    sum = 0
    for i in range(n):
        sum += i
    print(sum)

if __name__ == '__main__':
    startTime = time.time()
    run(100000000)
    endTime = time.time()
    print("耗时:", endTime - startTime)

Result: ~10 seconds.

Go code:

package main

import (
    "fmt"
    "time"
)

func run(n int) {
    sum := 0
    for i := 0; i < n; i++ {
        sum += i
    }
    fmt.Println(sum)
}

func main() {
    var startTime = time.Now()
    run(100000000)
    fmt.Println("耗时:", time.Since(startTime))
}

Result: ~200 ms.

5. Build Go as a .so File for Python

Go source (exported function):

package main

import "C"

//export run
func run(n int) int {
    sum := 0
    for i := 0; i < n; i++ {
        sum += i
    }
    fmt.Println("我是Go代码,我跑完了,我的结果是:", sum)
    return sum
}

func main() {}

Compile command: go build -buildmode=c-shared -o s1.so s1.go This generates s1.so and a header file. The shared object can be loaded from Python.

6. Python Code to Call the .so

from ctypes import *
import time

if __name__ == '__main__':
    startTime = time.time()
    s = CDLL("s1.so")  # load the shared library
    result = s.run(100000000)  # call the exported Go function
    print("result:", result)
    endTime = time.time()
    print("耗时:", endTime - startTime)

Result: ~0.11 seconds, but the returned value may be incorrect for very large inputs due to type‑size mismatches.

7. Observations and Issues

The Go implementation runs dramatically faster than pure Python, but when the result is returned to Python via ctypes, large integer values can overflow or be misinterpreted, leading to incorrect results for inputs larger than about one million.

Testing with smaller numbers (e.g., 10023) yields correct results, indicating the need for proper handling of integer sizes or using 64‑bit return types.

8. Summary

Compiling Go code into a shared library and calling it from Python provides a simple way to bypass Python's GIL and achieve substantial speedups for CPU‑bound tasks, while avoiding the steep learning curve of C/C++. However, developers must pay attention to data‑type compatibility when exchanging values between the languages.