Why Does Your Python Requests Call Throw Encoding Errors? Fix It Quickly

Problem Description

A fan in a Python group encountered an encoding error when using the requests library, as shown in the screenshots.

The code runs fine for others, but on his machine it fails.

Analysis

The core issue is a mismatch between the website's encoding (UTF‑8) and the local development environment's default encoding (often GBK on non‑Mac systems). On macOS the default is UTF‑8, so the problem is less common there.

If open() is used without specifying an encoding, the data may appear garbled on machines with a non‑UTF‑8 default.

Solution

Ensure the target website declares UTF‑8 (e.g., <meta charset='utf-8'/>) and set PyCharm's file encoding to UTF‑8. response.text.encode('utf-8').decode('utf-8') After changing the encoding setting in PyCharm to UTF‑8, the script runs without errors.

Extended Knowledge

If you encounter similar Unicode errors (e.g., '\u7535\u5546'), you can use the following snippet to ensure proper UTF‑8 handling:

text = '\u7535\u5546'
print(text.encode('utf-8').decode('utf-8'))

Conclusion

The article demonstrates how to diagnose and fix encoding bugs by checking the website’s charset and configuring the IDE’s encoding settings, helping Python developers avoid common Unicode pitfalls.