Python Weather Data Scraping and Visualization Analysis

The article begins by explaining the motivation for weather data collection and introduces the use of requests and BeautifulSoup to fetch HTML content from the China Weather website for both the 7‑day and 15‑day forecasts.

Data acquisition : A function getHTMLtext(url) sends a GET request and returns the page text. Another function get_content(html) parses the HTML, extracts relevant fields such as date, temperature, wind direction, wind level, precipitation, humidity, and air quality, and stores them in lists. A similar function get_content2(html) processes the 15‑day forecast.

CSV storage : The extracted data are written to weather1.csv (hourly data for the current day) and weather14.csv (daily data for the next 14 days) using the csv module.

Visualization : Three analysis scripts ( data1_analysis.py and data14_analysis.py ) read the CSV files with pandas and generate plots with matplotlib:

Temperature curves for a single day and for the 14‑day period, highlighting maximum, minimum, and average values.

Relative humidity curves with similar annotations.

Air quality (AQI) bar charts colored by pollution level, plus average AQI line.

Wind direction and speed radar charts using polar coordinates.

Scatter plot of temperature versus humidity with calculated correlation coefficient.

Pie chart showing the distribution of weather conditions over the 14‑day horizon.

Statistical analysis : A custom calc_corr(a, b) function computes the Pearson correlation between temperature and humidity, revealing a strong negative relationship.

Conclusions : The analysis confirms daily temperature cycles, inverse humidity trends, moderate air quality in the studied region, and diverse wind patterns without a dominant direction, while also noting the impact of weather on temperature variations.

Code framework : The project is organized into three Python files— weather.py for data crawling, data1_analysis.py for single‑day visualizations, and data14_analysis.py for multi‑day visualizations—each containing functions for plotting and statistical calculations.