How to Scrape Quotes from quotes.toscrape.com with Python and Save to CSV

1. Input the URL

Open https://quotes.toscrape.com/ and observe that the page contains three main fields: the quote text, the author, and tags. These are the data we want to extract.

2. Determine requirements and analyze the page structure

Use the browser’s developer tools (Network tab) to see that the site uses simple GET requests without parameters. We can therefore simulate the request with requests.get(), adding appropriate User‑Agent headers to avoid being blocked as a bot.

3. Parse the page and extract data

After a successful request, parse the HTML with lxml.etree and use XPath to locate the list of quotes. For each quote, extract the text, author, and associated tags, and store them in a Python list.

4. Save the data to a CSV file

Write the collected information into a CSV file for later analysis.

import requests
from lxml import etree
import csv

url = "https://quotes.toscrape.com/"
headers = {
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36"
}

res = requests.get(url, headers=headers).text
html = etree.HTML(res)

quote_list = html.xpath('//div[@class="col-md-8"]')
lists = []

for quote in quote_list:
    # Quote text
    title = quote.xpath('./div[@class="quote"]/span[1]/text()')
    # Author
    author = quote.xpath('./div[@class="quote"]/span[2]/small/text()')
    # Tags
    tags = quote.xpath('./div[@class="quote"]/div[@class="tags"]/a[@class="tag"]/text()')
    lists.append(title)
    lists.append(author)
    lists.append(tags)

with open("./quotes.csv", "w", encoding="utf-8", newline="
") as f:
    writer = csv.writer(f)
    for item in lists:
        writer.writerow(item)

The above script demonstrates a complete workflow: sending a GET request with headers, parsing the response using XPath, extracting the desired fields, and persisting them to a CSV file.