How to Build a Python Web Scraper for Eastmoney Data with Real Code

Hello, I'm PiPi. Recently a question about Python web crawling was asked in a group, and here's the share.

The code from ChatGPT provides a good idea, but details need verification.

Implementation

Later, Teacher YuLiang provided a complete script:

import requests
from pprint import pprint as print
import pandas as pd

url = 'https://datacenter-web.eastmoney.com/api/data/v1/get'
headers = {
    "User-Agent": "your UA",
    "Referer": "https://data.eastmoney.com/stock/tradedetail/2023-03-03.html",
    "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8"
}
cookies = {
    'Cookie': 'your Cookie'
}
params = {
    "callback": "",
    "sortColumns": "SECURITY_CODE,TRADE_DATE",
    "sortTypes": "1,-1",
    "pageSize": "50",
    "pageNumber": "1",
    "reportName": "RPT_DAILYBILLBOARD_DETAILSNEW",
    "columns": "SECURITY_CODE,SECUCODE,SECURITY_NAME_ABBR,TRADE_DATE,EXPLAIN,CLOSE_PRICE,CHANGE_RATE,BILLBOARD_NET_AMT,BILLBOARD_BUY_AMT,BILLBOARD_SELL_AMT,BILLBOARD_DEAL_AMT,ACCUM_AMOUNT,DEAL_NET_RATIO,DEAL_AMOUNT_RATIO,TURNOVERRATE,FREE_MARKET_CAP,EXPLANATION,D1_CLOSE_ADJCHRATE,D2_CLOSE_ADJCHRATE,D5_CLOSE_ADJCHRATE,D10_CLOSE_ADJCHRATE,SECURITY_TYPE_CODE",
    "source": "WEB",
    "client": "WEB",
    "filter": "(TRADE_DATE<='2023-03-03')(TRADE_DATE>='2023-03-03')"
}
response = requests.get(url, headers=headers, cookies=cookies, params=params)
data = response.json()['result']['data']
df = pd.DataFrame(data)
print(df)

Running it yields the expected result:

Note that the ChatGPT code may require you to verify the constructed URL manually.

The issue was solved successfully.

Conclusion

This article reviews a Python web‑crawling problem, provides detailed analysis and a complete code implementation, helping readers solve the issue effectively.