Master CSS Selectors in Scrapy: A Practical Guide to Web Data Extraction

After a brief reminder of previous XPath tutorials, this guide focuses on CSS selectors, which serve the same purpose of locating elements in HTML but often use simpler syntax, especially for front‑end developers.

CSS Basics

Common CSS selector patterns are listed, such as * for all nodes, div span for descendant spans, div > p for direct child paragraphs, .entry-header for class selection, #container for ID selection, and attribute selectors like img[src] or img[src="http://baidu.com/"]. These selectors are easy to remember and useful for extracting web data.

Practical Application

Using a sample website, the article shows how to extract the article title, publish date, topic tags, like count, and other fields with CSS selectors in Scrapy. Images illustrate each step:

The title selector is demonstrated in Scrapy shell, noting that text extraction uses the ::text pseudo‑element (double colon) unlike XPath.

For the publish date, the unique class .entry-meta-hide-on-mobile is used:

Similar CSS expressions are written for topic tags, like count (using .vote-post-up), and the numeric conversion of the like count with int().

Conclusion

The article summarizes that with a solid understanding of CSS selector syntax, one can efficiently harvest web data using Scrapy, and encourages readers to explore further CSS‑based extraction techniques in upcoming posts.