Designing Secure and Robust APIs: Signatures, Encryption, Rate Limiting, and More

Introduction

In real work we often need to interact with third‑party platforms, either consuming their APIs or exposing our own. Designing an elegant API interface must satisfy security, repeatability, stability, and easy troubleshooting.

1. Signature

To prevent data tampering, the client concatenates request parameters, a timestamp, and a secret key, then hashes the string (e.g., using MD5) to generate a sign. The sign is sent in the request header or parameters. The gateway recomputes the signature with the same algorithm and compares the values; matching signatures indicate a valid request, otherwise a signature error is returned.

A timestamp is added to limit the request’s validity (e.g., 15 minutes) to prevent replay attacks.

Two key‑distribution methods are common: a pre‑agreed static privateKey, or an AK/SK pair where the client sends accessKey (AK) and the server retrieves the corresponding secretKey (SK) to compute the signature.

2. Encryption

Sensitive data such as bank card numbers or IDs must not be transmitted in plain text. A simple approach is Base64 encoding, optionally applied multiple times. The client sends a single data field containing the Base64 string; the gateway decodes it using the agreed key and algorithm.

【Data before encryption】www.baidu.com
【Data after encryption】d3d3LmJhaWR1LmNvbQ==

Note that Base64 is technically encoding, not encryption.

3. IP Whitelist

Restrict access to trusted IP addresses by configuring an IP whitelist at the gateway or firewall (e.g., ModSecurity). Only whitelisted IPs can call the API; others receive an “access denied” response.

4. Rate Limiting

Uncontrolled third‑party calls can overwhelm the service. Implement rate limiting by IP, by specific API, or by AK/SK user. Common tools include Nginx, Redis, or API gateways.

Limit total requests per IP (e.g., 10,000 per minute).

Limit requests to a specific endpoint per IP (e.g., 2,000 per minute).

Limit requests per AK/SK user (e.g., 10,000 per minute).

5. Parameter Validation

Validate required fields, types, lengths, and enum values to reject invalid requests early. In Java, the Hibernate Validator framework provides annotations such as @NotEmpty, @Size, @Max, @Min, etc. Custom annotations can handle dates and enums.

6. Unified Response Format

Define a single JSON response structure for all outcomes. Example of a successful response:

{
  "code": 0,
  "message": null,
  "data": [{"id":123,"name":"abc"}]
}

Examples of error responses (signature error, no permission) illustrate why inconsistent formats are problematic. The gateway should enforce a uniform error schema, e.g.:

{
  "code": 500,
  "message": "Internal Server Error",
  "data": null
}

7. Unified Exception Handling

Capture all exceptions at the gateway, hide stack traces, database details, and error codes from third parties, and return a generic error object (code 500, message “Server internal error”). Detailed logs remain internal.

8. Request Logging

Log URL, parameters, headers, method, response data, response time, and a traceId to correlate logs. Persist logs to MongoDB or Elasticsearch and provide a UI for authorized third‑party users.

9. Idempotency

Ensure repeated rapid calls with identical parameters do not create duplicate records. Implement unique database indexes or store a request identifier in Redis to guarantee idempotent behavior.

10. Record Count Limitation

Batch endpoints should cap the number of records per request (commonly 500). Exceeding the limit returns a clear error message; the limit should be configurable and agreed with partners.

11. Performance Testing

Before release, conduct load testing (e.g., with JMeter or ApacheBench) to measure QPS and verify that rate‑limit thresholds are realistic.

12. Asynchronous Processing

For long‑running or batch operations, push a message to a message queue and return immediately. The consumer processes the task asynchronously, and the client can receive the result via callback or polling.

13. Data Masking

Mask sensitive fields (e.g., phone numbers, bank cards) in responses using asterisks, such as 182****887, to reduce exposure risk.

14. Complete API Documentation

A thorough API spec should include endpoint URL, HTTP method, request parameters with descriptions, response schema, error codes, encryption/signature examples, full request demos, and extra notes (e.g., IP whitelist). Use consistent naming conventions (e.g., camelCase) and versioned URLs (e.g., v1/query/getCategory).