How to Scrape NetEase Cloud Music Hot Comments and Song List with Python

After a brief re‑introduction to Python web scraping, the author aims to collect the hot comments from NetEase Cloud Music’s Hot Songs chart.

By opening the web version of NetEase Cloud Music and navigating to the Hot Songs list, the author uses the browser’s developer tools (Network tab) to locate the request that returns comments. The relevant request URL follows the pattern

http://music.163.com/weapi/v1/resource/comments/R_SO_4_<song_id>?csrf_token=

and includes two encrypted POST parameters, params and encSecKey.

Because these parameters are generated per song but remain constant for a given page, the author reuses the params and encSecKey from any song’s first page to fetch its 15 hot comments, which are returned as JSON under the hotComments field.

To obtain the list of all songs on the Hot Songs chart, the author again inspects the Network panel and finds a GET request to toplist?id=3778678 that returns the full playlist. The response is parsed with two regular‑expression passes.

First, the author extracts the block containing song entries using:

<ul class="f-hide"><li><a href="/song?id=d*?">.*</a></li></ul>

Then, from each <li> element, the song name and id are captured with:

<li><a href="/song?id=d*?">(.*?)</a></li>

<li><a href="/song?id=(d*?)">.*?</a></li>

The extracted names and ids are stored, and the previously discovered comment‑fetching request is issued for each song to collect its hot comments.

Finally, the author runs the complete script, verifies that the saved comments match those displayed on the website, and confirms the correctness of the data.

Key screenshots illustrating the website, the Network panel, the comment request, and the playlist request are included throughout the tutorial.