How to Build a Real‑Time People Flow Heatmap with Python, Pandas & Folium
Learn step‑by‑step how to scrape Tencent’s location data via its API, process the coordinates with pandas, deduplicate and filter them, and finally visualize regional foot‑traffic as an interactive heatmap using Folium, complete with code snippets and practical tips.
While browsing Tencent's big‑data map, I noticed it visualizes user locations from services like WeChat and QQ. Inspired, I created a Python script to generate custom regional foot‑traffic heatmaps.
1. Data Acquisition
Tencent provides a data API for commercial use, but we can capture the endpoint by monitoring network requests on https://xingyun.map.qq.com/ . Each request sends four POST parameters (count, rank, etc.). The response contains a locs field where the first two numbers represent latitude (e.g., 3295) and longitude (e.g., 11590) multiplied by 100, and the third number indicates the number of people at that point.
Below is the code to fetch the data:
import requests
import json
header = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0'}
url = 'https://xingyun.map.qq.com/api/getXingyunPoints'
for i in range(1, 5):
payload = {'count': i, 'rank': 0}
response = requests.post(url, data=json.dumps(payload))
datas = json.loads(response.text)['locs']
datas = datas.split(',')
datas = [int(x) for x in datas[:-1]]
all_data = []
a = []
for n, data in enumerate(datas):
a.append(data)
all_data.append(a)
if (n + 1) % 3 == 0:
a = []
all_data = [[i[0]/100, i[1]/100, i[2]] for i in all_data]Convert the list to a pandas DataFrame:
import pandas as pd
lat = [float(i[0]) for i in all_data]
lon = [i[1] for i in all_data]
weight = [i[2] for i in all_data]
dataframe = pd.DataFrame({'纬度': lat, '经度': lon, '人数': weight})Remove duplicate entries:
dataframe = dataframe.drop_duplicates(keep='first')Pandas tip
DataFrame.drop_duplicates(subset=None, keep='first', inplace=False)removes duplicate rows; keep='first' retains the first occurrence.
Select a specific region
data1 = dataframe[(dataframe.纬度.between(39.26, 41.03)) & (dataframe.经度.between(115.25, 117.30))]2. Draw the heatmap with Folium
import folium
from folium.plugins import HeatMap
map_data = data1[['纬度', '经度', '人数']].values.tolist()
hmap = folium.Map(location=[data1['纬度'].mean(), data1['经度'].mean()], control_scale=True, zoom_start=13)
hmap.add_child(HeatMap(map_data, radius=5, gradient={.1: 'blue', .3: 'lime', .5: 'yellow', .7: 'red'}))The resulting map visualizes the estimated foot‑traffic density for the chosen area.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Python Crawling & Data Mining
Life's short, I code in Python. This channel shares Python web crawling, data mining, analysis, processing, visualization, automated testing, DevOps, big data, AI, cloud computing, machine learning tools, resources, news, technical articles, tutorial videos and learning materials. Join us!
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
