How to Scrape Global Electricity Capacity Data with Python and Visualize It

Hello, I'm Cai Ge.

China's electricity generation capacity has become the world's largest since the early 21st century, although recent regions face power curtailment.

In this article we explore how to collect global electricity capacity data from the 1990s to present using the International Energy Agency (IEA) API and visualize it.

1. Data Collection

The source is the IEA ( iea) website.

URL: https://www.iea.org/fuels-and-technologies/electricity

By opening the Data Browser and selecting an energy type and a country/region, you can obtain the data series.

To automate this we need to (1) retrieve all country/region English names and (2) fetch the electricity consumption data for each.

import requests
import pandas as pd

# Get country and region list
url = r'https://api.iea.org/stats/countries'
r = requests.get(url)
countries = pd.DataFrame(r.json())
countries = countries.dropna(subset=['region'])

Next we locate the indicator for total electricity consumption and request the data for each country.

dataList = []
for i, country in enumerate(countries['stats']):
    url = r'https://api.iea.org/stats/indicator/TotElecCons'
    params = {
        'countries': country,
        'startYear': 1990
    }
    r = requests.get(url, params=params)
    df = pd.DataFrame(r.json())
    try:
        data = df[['year']].copy()
        data[country] = df['value']
        data.set_index('year', inplace=True)
        dataList.append(data)
        print(f'{i}')
    except:
        continue

results = pd.concat(dataList, axis=1).fillna(0).loc[:'2018']

The resulting dataset looks like this:

2. Animated Visualization

We use the online tool Huahuo to create the animation. After adjusting the data to the required format, simply upload it and the tool generates the animated chart.

China's rapid capacity growth, driven by projects such as the Three Gorges Dam, has helped meet the nation's electricity demand.

This article is for learning and sharing code only; please avoid excessive crawling.