Why Do 502 Errors Appear Only on POST Requests After Migrating to PaaS?

Specific Phenomenon

After migrating the application to our PaaS platform, occasional 502 errors appear. The error screenshots are shown below.

The errors are relatively rare compared to request volume, but they continuously affect callers and need investigation.

Why Only POST Requests Are Visible

Readers may think only POST requests are logged because the ELK filter is set to POST, but GET requests can also produce 502 errors. Nginx retries GET requests (treated as idempotent) using the proxy_next_upstream mechanism, generating logs like the following.

Because GET is considered safe, Nginx retries it when an upstream returns 502, while POST is not retried. Therefore, the underlying cause is the same for both methods.

Network Topology

The request flow into the cluster is:

用户请求=>Nginx=>Ingress=>uwsgi

Do not ask why both Ingress and Nginx exist; it is due to historical reasons.

Statistical Investigation

Counting 502 errors from Nginx and Ingress shows they match, indicating the problem lies between Ingress and uwsgi.

Packet Capture

Statistical methods did not reveal a pattern, so a packet capture was performed.

Capture results show that the TCP connection is reused. Ingress uses HTTP/1.1, sending a second HTTP request on the same connection, but uwsgi does not support HTTP/1.1, so the second request is rejected, causing Ingress to report a 502. GET requests are retried, POST requests are not, which explains why only POST 502s appear in the statistics.

Ingress Configuration Learning

Ingress defaults to HTTP/1.1 for upstreams, while our uwsgi uses an http-socket (HTTP/1.0). This protocol mismatch leads to unexpected 502 errors.

Solution: explicitly set the HTTP version in the Ingress annotation.

{% if keepalive_enable is sameas true %}
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
{% else %}
nginx.ingress.kubernetes.io/proxy-http-version: "1.0"
{% endif %}

Conclusion

Capturing traffic once can quickly pinpoint the issue: the mismatch between Ingress (HTTP/1.1) and uwsgi (HTTP/1.0) caused the 502 errors. When troubleshooting a chain involving both Nginx and Ingress, compare their error counts; if they align, focus on the Ingress‑uwsgi link. Adjusting the HTTP version annotation resolves the problem.