How to Install Scrapy on Windows Without Errors: Step‑by‑Step Guide

Scrapy is a popular Python web‑crawling framework. This guide walks you through installing Scrapy on Windows and solving the frequent problems that arise.

Normally you would run pip install scrapy, but the installation often fails because the required libxml2 library is missing.

Download the appropriate lxml wheel that matches your Python version and system architecture (e.g., 64‑bit Python 3.4) from a trusted Windows‑binary repository, then install it with pip install lxml‑4.2.1‑cp34‑cp34m‑win_amd64.whl.

After lxml is installed, run pip install scrapy again; the installation should complete without errors.

If you encounter an error such as Microsoft Visual C++ 14.0 is required, download and install the Visual C++ Build Tools from Microsoft (http://landinghub.visualstudio.com/visual-cpp-build-tools) and retry the installation.

Another common issue is a wheel compatibility error, e.g.,

Twisted‑18.7.0‑cp34‑cp34m‑win_amd64.whl is not a supported wheel on this platform

. Verify that the wheel matches your Python version and OS bitness. If needed, rename the .whl file to .zip, extract its contents, and install the package manually.

Once all dependencies are satisfied, verify the installation with pip list, which should show scrapy among the installed packages.

Now you can create Scrapy projects and start crawling.