Using Postman and a Python Flask Service to Compare Old and New API Responses

Background: A project required replacing a data source and redesigning the database while keeping the external API contracts unchanged for downstream services.

Requirement: The input parameters, request paths, and output formats of hundreds of existing interfaces must remain identical, otherwise downstream impact must be manually verified.

Idea: Because manually scripting test cases for so many interfaces is inefficient, the author chose Postman to automate the comparison between old and new APIs.

Implementation Steps:

1. Import all microservice interfaces into Postman. Postman can import a Swagger‑UI JSON file, but if version incompatibility occurs, use Apifox to import the Swagger data first, export it, and then import the exported file into Postman.

2. In Postman's Pre‑request Script , write a script that clones the old request, modifies its host and port, sends it, and stores the old response in an environment variable.

var old_request = pm.request.clone();
const host = pm.environment.get("oldHost");
const port = pm.environment.get('oldPort');
old_request.update(old_request.url.host = host);
old_request.update(old_request.url.port = port);
pm.sendRequest(old_request, function (error, old_response) {
    if (error) {
        console.log(error);
    } else {
        if (old_response.responseSize == 0) {
            pm.environment.set("old_response", {});
        } else {
            console.log('old_response', old_response.json());
            pm.environment.set("old_response", old_response.json());
        }
    }
});

3. In the Tests tab, add an assertion script that compares the new response with the stored old response. For complex comparisons, a Python Flask service is used.

Python Flask Service (assertion backend):

from flask import Flask, request, redirect, url_for
import json
import deepdiff

app = Flask(__name__)

@app.route('/assert_json', methods=['POST'])
def assert_json():
    if request.method == 'POST':
        return_dict = {'status_code': '200', 'message': '处理成功', 'data': None}
        if len(request.get_data()) == 0:
            return_dict['status_code'] = '500'
            return_dict['message'] = '请求参数为空'
            return json.dumps(return_dict, ensure_ascii=False)
        old_response = request.json.get("old_response")
        new_response = request.json.get("new_response")
        compare_result = deepdiff.DeepDiff(old_response, new_response, ignore_order=True)
        return_dict['data'] = compare_result
        print('compare_result', compare_result)
    return json.dumps(return_dict, ensure_ascii=False)

@app.route('/assert_array', methods=['POST'])
def assert_array():
    if request.method == 'POST':
        return_dict = {'status_code': '200', 'message': '暂不支持该格式断言，后续扩展', 'data': None}
    return json.dumps(return_dict, ensure_ascii=False)

if __name__ == '__main__':
    app.run(debug=True)

4. The Postman Tests script calls the Flask service, sending the old and new responses for deepdiff comparison, and then asserts that the diff is empty.

var old_response = pm.environment.get("old_response");
var new_response = pm.response;
if (new_response.responseSize == 0) {
    new_response = {};
} else {
    new_response = new_response.json();
}
var url = '';
if (JSON.stringify(new_response).toString().startsWith('{')) {
    url = 'http://127.0.0.1:5000/assert_json';
} else if (JSON.stringify(new_response).toString().startsWith('[')) {
    url = 'http://127.0.0.1:5000/assert_array';
}
const compareRequest = {
    url: url,
    method: "POST",
    header: 'Content-Type: application/json',
    body: {
        mode: 'raw',
        raw: {"old_response": old_response, "new_response": new_response}
    }
};
pm.sendRequest(compareRequest, function (err, res) {
    var compare_result = res.json().data;
    pm.environment.set("compare_result", compare_result);
});
var compare_result = pm.environment.get("compare_result");
pm.test("Compare two API responses", function () {
    console.log("compare result:", compare_result);
    pm.expect(compare_result).to.eql({});
});

By placing these scripts at the collection level, they can be reused for all interfaces, and custom assertions can be added for specific endpoints, greatly extending Postman's native capabilities.