Python Weather Data Scraping, CSV Storage, and Visualization with Matplotlib

The project uses Python's requests and BeautifulSoup libraries to fetch real‑time and 14‑day weather data from the China Weather website, parses the HTML and embedded JSON, and saves the extracted information into CSV files for further analysis.

Data acquisition is handled by the getHTMLtext(url) function, which performs an HTTP GET request with timeout handling and returns the page text, printing "成功访问" on success or "访问错误" on failure.

Useful information is extracted in get_content(html) by locating the div with id='7d' for the 7‑day forecast and parsing hourly data from a script tag that contains JSON. The function builds lists of date, weather, low/high temperature, wind direction, wind level, precipitation, humidity, and air quality, handling missing values and string cleaning.

Code snippet for data acquisition and extraction: def getHTMLtext(url): """请求获得网页内容""" try: r = requests.get(url, timeout=30) r.raise_for_status() r.encoding = r.apparent_encoding print("成功访问") return r.text except: print("访问错误") return " " def get_content(html): """处理得到有用信息保存数据文件""" final = [] bs = BeautifulSoup(html, "html.parser") body = bs.body data = body.find('div', {'id': '7d'}) # ... (parsing logic continues)

CSV files are written by write_to_csv(file_name, data, day) , which selects appropriate headers for hourly (day=1) or multi‑day (day=14) data and writes rows using Python's csv module.

Visualization scripts employ pandas to read the CSV files and matplotlib to generate several plots:

Temperature curve for a single day with average line and annotations for max/min values.

Relative humidity curve with similar styling.

AQI bar chart using color tiers to represent pollution levels.

Wind direction and level radar chart using polar coordinates.

Scatter plot of temperature vs. humidity with calculated correlation coefficient.

14‑day high/low temperature curves with average lines and annotations.

14‑day wind radar chart.

Weather condition distribution pie chart.

Each visualization function handles missing data, computes averages, identifies extreme values, and adds titles, axis labels, legends, and text annotations for clarity.

The concluding analysis notes a strong negative correlation between temperature and humidity, explains how AQI relates to meteorological factors such as temperature inversions, and describes wind patterns observed over the forecast period.

The codebase is organized into three scripts:

weather.py – fetches raw HTML, extracts daily and hourly data, and writes weather1.csv and weather14.csv .

data1_analysis.py – reads weather1.csv and produces the single‑day visualizations and correlation analysis.

data14_analysis.py – reads weather14.csv and creates the 14‑day temperature, wind radar, and weather‑type pie charts.

Full source code for these scripts is included in the article, preserving all original lines inside ... blocks.