Master JSON Extraction in Python Tests with jsonpath-ng

Why not use dict.get()? – Limitations of manual indexing

Directly indexing nested JSON with Python dictionaries (e.g., resp.json()["data"]["user"]["profile"]["name"]) produces verbose code, raises KeyError when fields are missing, and requires explicit length checks for list elements. A declarative extraction method such as JSONPath avoids these issues.

jsonpath-ng overview

jsonpath-ng is a fully‑featured, Python‑3.7+ compatible implementation of the JSONPath query language. It supports standard JSONPath syntax, extraction of single or multiple values, conditional filters, and nested paths, and returns results without raising KeyError. It integrates well with pytest and requests.

Installation: pip install jsonpath-ng Do not install the deprecated jsonpath package.

Common JSONPath scenarios

Example response used in the snippets:

{
  "code": 200,
  "data": {
    "order_no": "ORD20251119001",
    "user": {"id": 1001, "name": "张三"},
    "items": [
      {"product_id": 101, "name": "iPhone", "price": 5999, "qty": 1},
      {"product_id": 102, "name": "AirPods", "price": 1299, "qty": 2}
    ],
    "total": 8597
  }
}

Root field

from jsonpath_ng import parse
expr = parse("$.code")
result = expr.find(json_data)
print(result[0].value)  # 200

Nested object field

expr = parse("$.data.user.name")
print(expr.find(json_data)[0].value)  # 张三

List element by index

# First product name
expr = parse("$.data.items[0].name")
print(expr.find(json_data)[0].value)  # iPhone

# Last product name without knowing length
names = [m.value for m in parse("$.data.items[*].name").find(json_data)]
print(names[-1])  # AirPods

Conditional filter

expr = parse("$.data.items[?(@.price > 2000)].name")
matches = expr.find(json_data)
print([m.value for m in matches])  # ['iPhone']

Extract multiple fields for batch validation

product_ids = [m.value for m in parse("$.data.items[*].product_id").find(json_data)]
qtys = [m.value for m in parse("$.data.items[*].qty").find(json_data)]
for pid, qty in zip(product_ids, qtys):
    print(f"商品 {pid} 数量: {qty}")

Reusable extraction function for pytest

# libs/utils.py
from jsonpath_ng import parse

def get_value_by_jsonpath(json_obj, expr_str):
    """Return the first match value or None if no match."""
    try:
        matches = parse(expr_str).find(json_obj)
        return matches[0].value if matches else None
    except Exception as e:
        print(f"JSONPath parsing failed: {expr_str}, error: {e}")
        return None

Example test case:

# tests/test_order.py
def test_order_total():
    resp = requests.get("/order/123")
    data = resp.json()
    total = get_value_by_jsonpath(data, "$.data.total")
    assert total == 8597
    first_product = get_value_by_jsonpath(data, "$.data.items[0].name")
    assert first_product == "iPhone"

Common pitfalls and debugging tips

find()

returns a list of Match objects; access .value via [0].value or iterate.

JSONPath is case‑sensitive; field names must match the response exactly.

When a list may be empty, check len(matches) > 0 before indexing.

Conditional expressions require the @ symbol, e.g., [?(@.field == value)].

Debug by printing all matches:

matches = parse("$.data.items[*].name").find(json_data)
print("Matches:", [m.value for m in matches])

JSONPath quick reference

Cheat‑sheet image: