How to Clean Malformed JSON Strings in Python: Practical Regex Solutions

Introduction

In a Python community chat a member posted a JSON string that cannot be parsed because some fields contain stray double‑quotes, e.g.

{
"taskType": 1,
"printPageHeight": 1459,
"exportTypeTemplate": "html",
"reportTitle": "信息科技"网络安全漏洞扫描系统 "安全评估报告-主机报表",
"companyName": "信息科技",
"createTime": "2024-08-09 10:03:48",
"curr_lang": "zh-CN"pt "漏洞"
}

The goal is to clean the string so that json.loads can read it.

Implementation

One suggestion uses regular expressions to remove the extra characters and ensure every value is surrounded by double quotes.

import re
import json

# original JSON string
json_str = '''
{
  "taskType": 1,
  "printPageHeight": 1459,
  "exportTypeTemplate": "html",
  "reportTitle": "信息科技"网络安全漏洞扫描系统 "安全评估报告-主机报表",
  "companyName": "信息科技",
  "createTime": "2024-08-09 10:03:48",
  "curr_lang": "zh-CN"pt "漏洞"
}
'''

# attempt to fix stray commas (example)
json_str = re.sub(r',\s*[^,}]*$', '', json_str)

# ensure each value is quoted
json_str = re.sub(r'("([^\"]+)"\s*:\s*)([^\"]+)(,?)', r'\1"\3"\4', json_str)

try:
    data = json.loads(json_str)
    print("JSON解析成功：", data)
except json.JSONDecodeError as e:
    print("JSON解析失败：", e)

This approach raised invalid group reference 5 at position 7, so an alternative method was proposed.

The second method extracts key‑value pairs with re.findall and rebuilds a dictionary, then optionally converts it back to a JSON string.

json_str = re.findall(r'"(.*?)":\s*(.*?)[,
]', json_str)
data = {i[0]: i[1].replace('"', ' ') for i in json_str}
# convert to JSON string if needed
json_str = json.dumps(data, ensure_ascii=False)

A refined version keeps integer values unchanged:

data = {i[0]: i[1][1:-1] if i[1][0] == '"' else int(i[1]) for i in json_str}

Summary

The discussion demonstrates several practical ways to clean malformed JSON strings in Python, from simple regex replacements to extracting pairs with re.findall and rebuilding the structure, enabling successful parsing with json.loads.