Life's short, I code in Python. This channel shares Python web crawling, data mining, analysis, processing, visualization, automated testing, DevOps, big data, AI, cloud computing, machine learning tools, resources, news, technical articles, tutorial videos and learning materials. Join us!
This article introduces five mobile apps — c4droid for C/C++, AIDE for Java, QPython3 for Python, w3cschool for web development, and Termux for Linux — with installation steps and basic usage guides for learning programming on smartphones.
A step-by-step Python tutorial demonstrating how to scrape book details (publisher, ISBN, ratings) from Douban using search suggestion API and XPath parsing, then merge with Excel data for statistical analysis and visualization with pandas and matplotlib.
This tutorial introduces XPath syntax for web scraping, explaining HTML node relationships (parent, child, sibling, ancestor, descendant) and demonstrating key expressions including path operators (/ vs //), attribute selection (@), predicates, wildcards, and union operators with practical examples for element targeting.
This guide walks through resolving the 'No module named win32api' error when running Scrapy on Windows, covering failed pip installs, downloading the correct wheel from UCI's Python libs, a .whl-to-.zip workaround, and the final simple fix using pip install pywin32.
This tutorial guides you through creating a Scrapy project on Windows, from activating a virtual environment and running scrapy startproject to examining the generated file structure and understanding the purpose of each core file like settings.py, pipelines.py, and spiders.
This tutorial walks through creating a Scrapy spider using the genspider command, verifying the project structure, importing the project into PyCharm, synchronizing the spiders folder, examining the generated spider file, and configuring the Python interpreter to use the project's virtual environment.
This article explains Scrapy's asynchronous architecture built on Twisted, details its five core components (Engine, Scheduler, Downloader, Spiders, Item Pipeline) and middleware, then walks through creating a project with scrapy startproject, the generated directory structure, and running a spider via scrapy crawl.
This article explains web crawler fundamentals, from the need for automated data collection amid information explosion to the URL-based queue crawling process, basic Python libraries, and the role of frameworks in boosting development efficiency.
This tutorial walks through installing Scrapy on Windows, resolving frequent errors like missing lxml by using unofficial Windows binaries, fixing Microsoft Visual C++ 14.0 requirement via Build Tools, and a workaround for unsupported Twisted wheels by renaming .whl to .zip and extracting.
This tutorial covers Python's built-in numeric types including integers, floats, complex numbers, booleans, and fractions, with detailed examples of division behavior differences between Python 2 and 3, floor division, modulo operations, and the fractions module for precise rational arithmetic.
