This guide shows how to create a custom Scrapy command to launch every spider at once, separate each spider's settings for better modularity, and resolve video download problems by adjusting request headers and handling file saving correctly.
The article surveys ten Python visualization libraries—from the classic matplotlib to newer tools like Plotly and Gleam—detailing each library’s main features, typical use cases, developer information, and where to find further documentation, helping readers choose the right tool for their data projects.
Learn how to build a Python web scraper that extracts iQIYI movie titles, actors, and scores, parses the data with regex or BeautifulSoup, displays results in a Tkinter GUI with a combobox, and saves the information to a file—all explained with code snippets and screenshots.
This article demonstrates how to crawl 6,025 Beijing Dank Apartment listings using Python, clean and enrich the data with Pandas, and visualize distribution, price, size, floor, and subway proximity through charts, revealing key market insights and correlation patterns.
This tutorial explains the fundamentals of Python logging, covering when to log, the standard log levels, simple console and file examples, multi‑module logging, variable interpolation, message formatting, date/time handling, and advanced concepts such as loggers, handlers, formatters, configuration methods, and best practices for library developers.
This article explains why identical integers and short strings may reference the same object in Python, how the small‑integer cache and string interning work, and how compilation units and constant pools affect object identity across interactive sessions and script files.
Learn how to use Python’s built-in urllib library for web requests, handling GET/POST, adding headers, managing proxies, processing cookies, handling errors, parsing URLs, and respecting robots.txt, with clear code examples and a practical case of scraping a novel site.
This article lists essential Python standard libraries and popular third‑party packages, explains their primary functions, and provides concise code examples for modules such as datetime, zlib, and sys to help developers quickly understand and apply them in their projects.
This guide shows how to create a Python 3.7 Socket.IO client script for testing socket interfaces, including installation of the python‑socketio package, handling authentication tokens with tools like Postman, and a complete example script with repository links for reference.
This article explains how to locate Weibo comment APIs, work around rate limits by using the mobile site, extract required parameters, and implement a Python script that handles cookies, pagination, emoji cleaning, deduplication, and scheduled execution to collect comments efficiently.
This guide demonstrates how to install PyMySQL, connect Python to a MySQL database, create tables, perform CRUD operations, manage transactions, and handle errors, providing complete code examples and explanations for each step.
A Tidelift analysis of over 6,000 developers reveals that JavaScript, Python, and Java dominate modern software projects, with detailed usage percentages, team size trends, and comparisons to other language rankings, offering valuable guidance for technology decisions.
This tutorial explains how to count word frequencies using Python's collections.Counter from static, random, or file‑based text sources and then visualize the results with the WordCloud library, covering code examples, parameter settings, and available colormaps.
FrankMocap, an open‑source AI algorithm from Facebook AI Research and HKU, delivers simultaneous 3D full‑body and hand pose estimation from a single monocular video, runs at about 9.5 FPS on a RTX 2080, and includes easy installation steps, code examples, and links to its GitHub repository and paper.
This article presents twenty practical Python snippets—including string reversal, title casing, unique element extraction, list repetition, comprehensions, variable swapping, splitting, joining, palindrome checking, Counter usage, anagram detection, exception handling, enumeration, memory sizing, dictionary merging, timing, flattening, random sampling, digit list conversion, and uniqueness verification—to help developers write cleaner, faster code.
This article provides a step‑by‑step guide on using Scrapy’s crawl template to automatically scrape product details such as ID, title, shop name, shop link, and price from JD.com, including source analysis, project setup, code snippets, and result verification.
This tutorial walks you through the fundamentals of sending emails with Python, covering SMTP/POP3/IMAP protocols, setting up authorization codes, establishing connections, and constructing plain‑text, HTML, image, and file attachments using the smtplib and email modules.
This article explains how to use Python's requests library and CSV handling to automate API testing of payment interfaces, focusing on correctly processing special characters in usernames and account names to ensure secure and fast transactions.
This tutorial walks you through reading Chinese GeoJSON files, converting projections, preparing scatter data, and creating a detailed map with an inset of the South China Sea using Python‑geopandas, Matplotlib, and custom legends, while also covering installation tips and known limitations.
This article traces Python’s history—from Guido van Rossum’s Dutch roots and the influence of ABC, through its 1991 birth, community‑driven growth, and design philosophy—to illustrate how open‑source collaboration shaped one of today’s most popular programming languages.
This tutorial demonstrates how to create a Python GUI image viewer and a simple computer‑vision application using PySimpleGUI, os, and OpenCV, covering layout design, element configuration, event handling, and camera integration with clear code examples and explanations.
This article provides a comprehensive guide to Neo4j, an open‑source graph database, covering its data model, key features, installation, and detailed Python usage with the Py2neo library, including node and relationship creation, property handling, subgraph operations, walkable traversals, and object‑graph mapping (OGM).
This article introduces the AI‑powered "Bringing-Old-Photos-Back-to-Life" project, explains its requirements, provides step‑by‑step commands for full‑pipeline restoration, scratch detection, global restoration, and face enhancement, and shares the Colab demo and GitHub repository for hands‑on experimentation.
This article walks through the complete process of designing a Python web scraper to collect Zhihu answer data, parse author, ID and excerpt fields, store the results in CSV files, and finally visualize the text with a word‑cloud, including all necessary code snippets and explanations.
The gRPC Core 1.34.0 release introduces numerous improvements and bug fixes across multiple language implementations, including new environment variable security support, Unix abstract socket URIs, enhanced xDS handling, updated platform compatibility, and added support for recent PHP, Python, and Ruby versions.
Learn how to use the Python Hypothesis library for property‑based testing, understand different test levels and methods, and see practical examples—including unit tests for increment/decrement functions and handling edge cases with custom examples—to write more robust, automated tests.
This article introduces Ke‑FUT, a home‑grown Flutter UI automation solution that leverages VMService and InspectorService to obtain element IDs, generates stable IDs via a custom IdGenerator, and drives view interactions through a Python‑based FUTClient, providing a low‑cost, cross‑platform testing approach for mobile apps.
This guide explains how to use Python to create custom brute‑force password dictionaries—covering simple numeric PIN lists, targeted web‑backend wordlists, and practical usage tips—for quickly cracking basic encrypted files.
musicpy is a Python library that lets developers write music using concise, theory‑based syntax, outputting MIDI files, and aims to enable AI composition through explicit music‑theory rules rather than deep‑learning, offering classes for notes, chords, scales and supporting mathematical operations on musical data.
This article presents a concise collection of thirteen practical Python snippets covering basic arithmetic, mathematical functions, random number generation, number classification, prime testing, factorials, multiplication tables, numeral system conversions, greatest common divisor, and calendar creation, each illustrated with example output images.
This article walks through using Python to crawl Tencent Video's "Offer" season 2 comments, merge and clean the CSV data, perform exploratory analysis, generate visualizations and word clouds, and apply Baidu's open‑source NLP model for sentiment scoring, providing complete code snippets for each step.
This article explains Python's two string‑formatting approaches—the C‑style % operator and the more powerful format() method—covering integer, float, and string placeholders, alignment, precision, and advanced usage with clear code examples.
This tutorial walks through Python's time and datetime modules, covering timestamps, sleep, local time, formatting, parsing, arithmetic, and the dateutil library for flexible month and year adjustments, with code examples and visual output to illustrate each concept.
This guide details the core PEP‑8 conventions for Python code, covering comments, indentation, line length, imports, whitespace, naming, and input/output practices, with clear examples and practical tips to improve readability and maintainability.
This guide walks you through setting up a Python environment, installing required libraries, constructing a FilmSky scraper class, handling pagination, parsing HTML with regex, and saving movie titles and download links, enabling you to browse and download movies from the FilmSky website efficiently.
Learn how to translate common mathematical symbols such as summation, product, factorial, conditional expressions, and matrix multiplication into clear Python code, revealing the underlying computations and helping data scientists and ML practitioners deepen their mathematical intuition through practical examples.
This article explains how Vaex, an open‑source DataFrame library, lets data scientists efficiently explore, visualize, and analyze massive datasets—such as the over‑billion‑row NYC taxi records—using memory‑mapping and virtual columns, all on a standard notebook without costly cloud resources.
This guide walks through generating word clouds from JD product reviews using Python, covering preprocessing, tokenization, frequency analysis, and visual creation with the stylecloud, wordcloud, and pyecharts libraries, including mask image selection, palette customization, and full code snippets for each method.
This guide walks you through installing, configuring, and using five Python text‑to‑speech libraries—gTTS, Baidu AIP, pyttsx3, pywin32, and speech—to generate personalized audio files, adjust voice properties, and automate playback.
Python 3.10’s second alpha introduces major enhancements such as expanded type annotations with delayed evaluation, the new ‘|’ union operator, explicit TypeAlias support, a strict mode for zip, int.bit_count() for population count, and mapping views for dictionaries, all aimed at improving code clarity and safety.
The article outlines a machine‑learning pipeline for automated captcha recognition, covering dataset generation, image preprocessing, segmentation via clustering or watershed methods, and classification using classic models and CNNs, achieving roughly 94% accuracy while noting the growing complexity of modern captchas and recommending developer collaboration when feasible.
This article explains how crawler engineers can fetch real‑time data such as sports scores, stock quotes, or cryptocurrency prices by comparing polling and WebSocket approaches, introducing the aiowebsocket library, and providing complete Python code to perform handshake, subscription, and continuous data streaming.
This tutorial walks you through setting up PyCharm to directly edit and synchronize code on a remote CentOS 7 server using SFTP, covering project path configuration, deployment settings, interpreter setup, and enabling automatic file syncing for efficient Python development.
This article explains how to set up a well‑configured CI/CD pipeline, create development‑friendly and production‑optimized Docker images for Python applications, and automate the entire workflow with Makefiles, GitHub Actions, and code‑quality tools such as CodeClimate and SonarCloud.
This article introduces Python’s sys and os modules, detailing their most useful functions for runtime configuration, file and path manipulation, process control, and environment handling, and provides clear code examples—including a simple progress bar and a demonstration of sys.argv versus __file__ behavior—to help readers master essential system programming techniques.
This article walks you through creating a Python Tkinter application that fetches a waste‑classification dataset, processes it with pandas, and provides interactive learning and testing modules so users can practice sorting recyclable, hazardous, wet, and dry waste.
Since its early 1990s debut, Python has surged ahead of many languages due to its readability and extensive libraries, yet it suffers from performance and memory drawbacks, GIL constraints, and limited mobile support, prompting a comparison with emerging languages like Julia, Rust, and Swift.
This article demonstrates how to use Python's os and shutil modules to enumerate system drives, list directory trees, delete files and folders, copy a script to the startup folder, and finally trigger a forced shutdown, illustrating a simple educational hacking tool.
This article presents ten innovative Python project ideas—from voice‑controlled GUIs and AI betting bots to trading algorithms, virtual assistants, web monitoring, SSL auto‑renewal, facial recognition, contact tracing, file management, and YouTube video aggregation—each designed to sharpen programming expertise and explore real‑world applications.
This tutorial explains how to use WeChat's template message API together with the iciba daily‑sentence service, set up a Python script on an Alibaba Cloud Linux server, configure the template in the public platform, and schedule automatic delivery via a daily cron job.
This article walks through a hands‑on experiment using Huawei Cloud RDS and Python to convert short videos into audio, extract voiceprints, and match them against a database, demonstrating how speech‑signal processing enables accurate identification of background songs in TikTok‑style clips.
This tutorial shows how to install the Python pynput library and use it to programmatically control the keyboard and mouse, enabling you to create a basic QQ message‑spamming script that can send repeated messages on both Windows and macOS platforms.
This article describes Qunar's DBA department automated migration platform, detailing its motivation, overall architecture, task pool and templates, customizable web‑based workflow, scheduling, execution queue, tooling, and the significant efficiency and reliability gains achieved across numerous database migration scenarios.
This guide shows how to package frequently used Python functions into importable modules, then demonstrates a complete script that merges multiple Excel files while preserving formatting, using pandas and standard libraries, and explains each step with screenshots and code examples.
This article walks you through creating a Python‑based voice synthesis tool using Baidu's AI platform, covering account setup, SDK installation, essential parameters, GUI design with Tkinter, code implementation, error handling, and generating audio files from custom text.
This guide demonstrates how to extract a macOS Chrome user's browsing URLs and timestamps from the History SQLite database, save them to a text file, and automatically email the file daily using Python scripts, a shell wrapper, and a crontab scheduled task.
This article introduces Python’s lambda anonymous functions, explains their syntax and differences from regular functions, showcases practical one‑line examples, explores functional programming tools like map, filter, and reduce, and highlights how lambdas can reduce code size while enhancing readability.
The article examines Python's enduring popularity, highlighting its readability, rich ecosystem, and corporate adoption while also discussing its performance drawbacks, memory usage, GIL limitation, and challenges in mobile and high‑performance computing, before comparing it to emerging languages.
This article explains how China's map is represented by three closed polygons, describes the text file format containing longitude‑latitude point groups, and provides a complete PySimpleGUI Python script to read the data and render the map as red polygons.
This article introduces eight widely used Python web scraping frameworks—including Scrapy, PySpider, Crawley, Portia, Newspaper, Beautiful Soup, Grab, and Cola—describing their main features, typical use cases, and providing links to their project repositories.
This article showcases seven intriguing code tricks—from a zero‑code deployment project and a concise shuffle algorithm to a sleep‑sort implementation, a one‑line AI chatbot, a simple next‑day timer, the legendary fast inverse square‑root constant, and the classic hello‑world example—each illustrated with brief explanations and runnable snippets.
This tutorial shows how to locate the "next page" link on a website, extract its URL using Scrapy selectors, add proper checks, and integrate the pagination logic into a Scrapy spider so that all article pages are crawled automatically.
Learn the core principles for writing effective unit tests—such as focusing on single functions, ensuring independence, and using descriptive names—alongside a concise overview of popular Python testing tools like unittest, pytest, nose, doctest, tox, and mock, to boost code reliability.
This article presents a curated list of the ten most popular Python‑related open‑source projects on GitHub in October 2023, summarizing each project's purpose, key features, and star count to help developers quickly discover valuable tools and libraries across security, web, automation, and more.
After retiring from Dropbox, Python’s creator Guido van Rossum announced he’s ending his retirement to join Microsoft’s developer division, aiming to improve Python’s performance across platforms, while the article also recounts his extensive career, awards, and the language’s origins from the ABC project.
This tutorial introduces Python fundamentals by demonstrating a Hello World program, explaining comments, covering numeric, string, tuple, list, set, and dictionary data types, and illustrating control flow constructs, loops, functions, command‑line arguments, and basic file read/write operations with clear code examples.
This article explains a Python puzzle where a list of lambda functions unexpectedly returns [6,6,6,6] due to late binding in closures, walks through the code, debugging screenshots, and clarifies how closures capture loop variables in Python.
The article recounts how Guido van Rossum, the creator of Python, retired, reflected on his next steps, and ultimately joined Microsoft’s developer team, highlighting the implications for Python’s evolution, cloud integration, and the broader open‑source ecosystem.
This tutorial explains RabbitMQ fundamentals, shows how to install the Pika client, provides complete producer and consumer Python examples, and covers advanced topics such as work queues, message acknowledgments, persistence, fair dispatch, exchanges, routing, and RPC usage.
This article presents thirty concise Python code snippets that demonstrate common programming tasks such as checking duplicates, counting vowels, merging dictionaries, measuring execution time, and performing list manipulations, each explained with clear examples for quick learning.
This tutorial explains how to use Scrapy's Request objects to feed article detail URLs into the crawler, configure callbacks for parsing, handle relative URLs with urljoin, and yield requests so Scrapy can download pages, completing the core data extraction workflow.
This article demonstrates how to scrape the Hurun Rich List from 2015 to 2020 using Python, clean the data, and create interactive visualizations of the top 20 wealth holders, wealth trends over six years, and industry shifts with Pyecharts.
This tutorial shows how to configure Scrapy to start from a list page, extract every article link, follow pagination automatically, and parse each article using XPath/CSS selectors, with practical shell commands and visual examples.
This article explains how to scrape detailed job description and address information from online job portals, use Python libraries such as requests, BeautifulSoup4, and pymongo for crawling, and then clean and normalize the collected data including publish dates, salaries, and work‑experience levels before storing it in MongoDB.
This guide shows how to install the Python Kubernetes client, initialize it, and perform comprehensive Job operations—including creation via SDK objects, YAML or dict definitions, monitoring with Watch, listing, reading, updating, and deleting Jobs—while providing concrete code snippets and best‑practice advice.
This guide explains what a message queue is, why it’s used in asynchronous architectures, walks through installing RabbitMQ on Linux and macOS, and demonstrates core patterns such as simple queues, work queues, publish/subscribe, routing, topics, and RPC with complete Python code examples.
This article recounts the author's three‑year journey building a Python‑based HR SaaS backend, detailing the architectural evolution from an initial Django MVC prototype through service splitting and microservices with Kong and custom RPC, to a domain‑driven design approach, and shares lessons learned and optimizations.
After two decades of Java and C dominating the TIOBE rankings, Python has surged to the second spot, prompting a discussion on why programming skills are becoming a universal necessity and how the latest index can guide developers in choosing the right language for future projects.
This article demonstrates how to locate and extract cover‑image URLs from a web page using Scrapy, explains handling absolute and relative URLs, shows the necessary XPath and meta‑passing code, and provides debugging tips to verify that the image URL is correctly transferred through the spider.
This article explains how to use Python's list index method to locate an element, details its full signature, performance considerations for large lists, ways to retrieve multiple matches, handling missing elements with exceptions, and useful tips like using the help function.
This tutorial explains how to retrieve an article’s cover image URL by starting from the list page, extracting the first image, and passing its URL through Scrapy’s Request meta dictionary to subsequent parsing callbacks, highlighting why list‑page extraction is more reliable than detail‑page scraping.
The newest TIOBE ranking shows Java's popularity slipping while Python climbs to the second spot, marking a historic shift in programming language trends and prompting developers to reassess language choices based on search‑engine‑driven popularity metrics.
This guide introduces NumPy, the core Python package for numerical computing, covering its ndarray structure, creation methods, attributes, reshaping, indexing, arithmetic and statistical functions, random number generation, and practical code examples to help you efficiently manipulate large data arrays.
This tutorial walks through checking the default Python version on a Linux system, locating its binaries, downloading Python 3.6.5 source, installing required build dependencies, compiling and installing Python, creating symlinks, and configuring environment variables, while also addressing common network and repository issues.
The article explains how automated testing—using Python‑based UI and API frameworks, configurable test‑data generation, Linux headless execution, and Jenkins continuous integration—addresses the growing complexity of software releases by reducing manual regression effort and improving defect detection speed.
This article walks through five common data‑cleaning scenarios—dropping/renaming columns, handling duplicates and nulls, string manipulation, merging tables, and window‑function ranking—showing practical SQL and Python code that can be applied to large‑scale data warehouses.
Discover 15 practical Pythonic techniques—from swapping variables and list comprehensions to context managers and chain comparisons—illustrated with clear code examples, while also learning how to claim free Python e-books through a simple WeChat interaction.
After three years of evolving a Python‑based HR SaaS backend from a simple MVC prototype to a microservices architecture with domain‑driven design, the author shares the architectural stages, challenges faced, and practical optimizations such as uWSGI‑gevent, Redis pooling, Kong API gateway, and custom RPC framework.
This guide walks you through extracting likes, comments, and article content from web pages using Scrapy’s CSS selectors, showing how to locate elements like bookmark buttons, parse numeric data with regular expressions, and integrate the resulting code into your Python spider for reliable data collection.
This guide walks through Python file operations—including opening, closing, reading, writing, path handling, file copying, CSV handling, in‑memory streams, sys.stdin/stdout/stderr redirection, and JSON vs. pickle serialization—providing code examples and practical tips for each topic.
This article demonstrates how to build a basic Tornado server, measures its performance under blocking operations, and then progressively applies multithreading, thread‑pool executors, and finally asyncio to achieve dramatically higher QPS and lower response times.
This guide walks you through using the Python httpx library to perform various HTTP methods—including GET, POST, PUT, DELETE, HEAD, and OPTIONS—handle query parameters, decode responses, work with JSON, custom headers, form data, file uploads, streaming, cookies, redirects, and authentication, all with clear code examples.
The author recounts a three‑year journey of scaling a Python‑based HR SaaS backend from a simple Django MVC prototype to a microservice architecture with domain‑driven design, detailing each architectural stage, the challenges faced, and the concrete optimizations applied.
This article introduces eight widely used Python web scraping tools—Scrapy, PySpider, Crawley, Portia, Newspaper, Beautiful Soup, Grab, and Cola—detailing their main features, typical use cases, and project links, helping developers choose the appropriate framework for data extraction tasks.
This article introduces the fundamentals of CSS selectors, compares them with XPath, and demonstrates step‑by‑step how to use CSS selectors in Scrapy to extract titles, dates, tags, likes, and other article data from web pages.
This article showcases a collection of practical Python snippets covering function chaining, mutable default argument traps, CSV reading and writing, number base conversion, JSON formatting, list flattening and merging, most‑common character counting, safe eval usage, matrix transposition, list comprehensions, permutations/combinations, defaultdict usage, and dictionary reversal techniques.
The author recounts a four‑stage evolution of a Python‑based SaaS backend—from a simple Django MVC prototype through service splitting, microservice adoption with Kong and custom RPC, to an exploratory domain‑driven design—highlighting practical problems, optimizations, and architectural trade‑offs.
This guide walks you through installing Scrapy-Distributed, setting up RabbitMQ and RedisBloom containers, creating a sitemap spider, configuring the distributed scheduler and dupefilter, and running the spider, while explaining why this non‑intrusive solution improves over existing Scrapy‑Redis and scrapy‑rabbitmq approaches.
This article presents a collection of practical Python snippets covering string reversal, title‑casing, set‑based deduplication, list multiplication, list comprehensions, variable swapping, splitting, joining, palindrome checking, Counter usage, dictionary merging, timing, flattening, random sampling, digit extraction, uniqueness testing and more, each illustrated with clear code examples.
This article details how Instagram migrated its massive Python 2/Django codebase to Python 3, describing the motivations, technical obstacles such as Unicode, pickling, iterator changes, the tools and strategies used, and the resulting CPU, memory, and latency improvements across billions of users.
