How to Scrape NetEase Cloud Music with Python: A Step‑by‑Step Guide
This article walks through using Python's requests and lxml libraries to fetch, parse, and download songs from NetEase Cloud Music, detailing the implementation steps, code snippets, encountered obstacles, and tips for handling anti‑scraping measures.
Introduction
Many music sites require a VIP subscription to download songs, so I explored a free method using a Python web scraper to retrieve tracks from NetEase Cloud Music.
Implementation Details
The process consists of four main steps: locating the music on the server, sending HTTP requests, extracting the desired file links, and downloading the files.
import requests # send network request librarypip install requestsfrom lxml import etree # data parsing librarypip install lxmlurl = 'https://music.163.com/#/discover/toplist?id=3778678'response = requests.get(url=url) # request page datahtml = etree.HTML(response.text) # parse page dataid_list = html.xpath('//a[contains(@href,"song?")]') # all song id collectionbase_url = 'http://music.163.com/song/media/outer/url?id=' # download music URL prefixfor data in id_list:
href = data.xpath('./@href')[0]
music_id = href.split('=')[1] # music id
music_url = base_url + music_id # download music URL
music_name = data.xpath('./text()')[0] # music name
music = requests.get(url=music_url)
with open('./music/%s.mp3' % music_name, 'wb') as file:
file.write(music.content)
print('<%s>下载成功' % music_name)Pitfalls Encountered
The original video‑based method stopped working because the site added anti‑scraping defenses, causing missing music_name and music_id values and returning a "network congested" error (code -460) for some IDs.
base_url = 'http://music.163.com/song/media/outer/url?id='
music_id = '1804320463.mp3'
music_url = base_url + music_id
music = requests.get(url=music_url)
print(music.text)The response shows
{"msg":"网络太拥挤,请稍候再试!","code":-460,"message":"网络太拥挤,请稍候再试!"}, indicating the server blocked the request.
base_url = 'http://music.163.com/song/media/outer/url?id='
music_id = '1804320463.mp3'
music_url = base_url + music_id
print(music_url)Printing the URL still allows manual playback and download via a browser, suggesting the API endpoint works when accessed directly.
Conclusion
Websites rapidly adopt advanced anti‑scraping mechanisms, so scraping music files is no longer straightforward. This article shares my learning experience with Python web scraping and invites readers to suggest solutions for bypassing the current obstacles.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
