Skip to content
Tagged articles

web scraping

500 articles · Page 1 of 5
Geek Labs
Geek Labs
Nov 5, 2026 · Backend Development

Scrapling: Adaptive Web Scraping That Survives Site Redesigns

This analysis explores Scrapling, a Python scraping library whose adaptive element matching automatically relocates target elements after website redesigns, combining five fetcher tiers, anti-bot evasion, Scrapy-like crawling, and AI agent integration to turn site redesigns from disasters into minor maintenance.

BenchmarkCSS selectorsCloudflare
0 likes · 29 min read
Scrapling: Adaptive Web Scraping That Survives Site Redesigns
Data STUDIO
Data STUDIO
Oct 29, 2026 · Artificial Intelligence

20 Open-Source Agent Projects Analyzed by Capability: A Practical Evaluation Guide

This article categorizes 20 open-source AI agent projects across ten functional domains—orchestration, web scraping, browser automation, code execution, memory, tool management, model routing, observability, voice interaction, and payment—providing concrete evaluation criteria, integration caveats, and real-world examples for each.

AI agentsAgent orchestrationMemory Management
0 likes · 29 min read
20 Open-Source Agent Projects Analyzed by Capability: A Practical Evaluation Guide
Geek Labs
Geek Labs
Oct 28, 2026 · Artificial Intelligence

Agent Reach: Unlock the Entire Internet for Your AI Agent (100k Stars)

Agent Reach is a 100k-star Python CLI that equips AI agents with unified, read-only access to 14 previously unreachable platforms—including login-walled sites like Xiaohongshu and Bilibili—via a three-layer architecture with backend fallbacks, standardized JSON output, actionable error codes, and built-in browser automation to avoid bans.

AI agentsAgent ReachCLI tool
0 likes · 15 min read
Agent Reach: Unlock the Entire Internet for Your AI Agent (100k Stars)
AI Programming Lab
AI Programming Lab
Oct 19, 2026 · Artificial Intelligence

DeepSeek-V4.1-Flash-1017 Unifies Search & Scraping for AI-Native Agents

DeepSeek-V4.1-Flash-1017 introduces native Search and Fetch APIs that eliminate third-party glue code, offering advanced query syntax, quality-scored results with three-tier summaries, and Markdown extraction — all accessible via a single API key with sub-cent pricing and free server-side tool calling until November 2026.

AI agentsDeepSeekFetch API
0 likes · 11 min read
DeepSeek-V4.1-Flash-1017 Unifies Search & Scraping for AI-Native Agents
Python Crawling & Data Mining
Python Crawling & Data Mining
Sep 28, 2026 · Backend Development

Master XPath Syntax for Web Scraping: Node Relationships & Expression Guide

This tutorial introduces XPath syntax for web scraping, explaining HTML node relationships (parent, child, sibling, ancestor, descendant) and demonstrating key expressions including path operators (/ vs //), attribute selection (@), predicates, wildcards, and union operators with practical examples for element targeting.

XMLXPathdata-extraction
0 likes · 7 min read
Master XPath Syntax for Web Scraping: Node Relationships & Expression Guide
Python Crawling & Data Mining
Python Crawling & Data Mining
Sep 25, 2026 · Backend Development

Create Your First Scrapy Spider: genspider Command, PyCharm Import & Interpreter Setup

This tutorial walks through creating a Scrapy spider using the genspider command, verifying the project structure, importing the project into PyCharm, synchronizing the spiders folder, examining the generated spider file, and configuring the Python interpreter to use the project's virtual environment.

PyCharmPythonSpider Tutorial
0 likes · 5 min read
Create Your First Scrapy Spider: genspider Command, PyCharm Import & Interpreter Setup
Smart Sea Tide
Smart Sea Tide
Sep 21, 2026 · Backend Development

6 Top Open-Source Web Scrapers Compared: AI Extraction, Browser Automation & Distributed Frameworks

This article reviews six industrial-grade open-source web scraping tools across three categories—AI-focused extractors (Firecrawl, Crawl4AI), browser automation frameworks (Playwright, Selenium, Crawlee), and the distributed crawler Scrapy—providing a selection guide for building AI knowledge bases, handling dynamic pages, or scaling to millions of pages.

AI data extractionPlaywrightbrowser automation
0 likes · 5 min read
6 Top Open-Source Web Scrapers Compared: AI Extraction, Browser Automation & Distributed Frameworks
Python Crawling & Data Mining
Python Crawling & Data Mining
Sep 17, 2026 · Backend Development

Automate Daily Weather Emails with Python Web Scraping

This tutorial shows how to scrape real-time weather data from Sohu Weather using BeautifulSoup and automatically send it via email with Python's smtplib, including code walkthroughs for both Sohu and 163 email servers, a reference list of common SMTP servers, and PM2.5 air quality standards.

BeautifulSoupPM2.5Python
0 likes · 7 min read
Automate Daily Weather Emails with Python Web Scraping
AI Architecture Path
AI Architecture Path
Sep 9, 2026 · Artificial Intelligence

Browser-Use: 113K-Star AI Agent Automates Browsers for E2E Testing & Data Collection

Browser-Use is an open-source AI browser automation framework with 113K+ GitHub stars that uses accessibility trees and LLMs to replace brittle CSS selectors, enabling natural-language control for E2E testing, form filling, data scraping, and e-commerce tasks via local or cloud deployment with support for multiple LLMs including a specialized BU 2.0 model.

AI agentE2E testingLLM
0 likes · 16 min read
Browser-Use: 113K-Star AI Agent Automates Browsers for E2E Testing & Data Collection
Old Zhang's AI Learning
Old Zhang's AI Learning
Sep 1, 2026 · Artificial Intelligence

Doubao-Seed-Evolving Tested: Coding, Multimodal, and Long-Horizon Agent Skills

The author evaluates Doubao-Seed-Evolving's latest upgrades across coding, agent, multimodal, and long-horizon skill execution using real-world tasks like PPT-to-website conversion, personal project management, SVG generation, 3D Rubik's cube animation, and a complex 30-minute web scraping skill, finding strong autonomous task decomposition, testing, and error recovery.

AI model testingCoding AgentDoubao-Seed-Evolving
0 likes · 10 min read
Doubao-Seed-Evolving Tested: Coding, Multimodal, and Long-Horizon Agent Skills
SpringMeng
SpringMeng
Aug 21, 2026 · Artificial Intelligence

Tianqiong ZhiXiang Forecast System with Django, Vue3, LSTM, TensorFlow, AI Agents

This article details a full‑stack weather forecast platform that combines Django and Vue3 with LSTM‑based TensorFlow models and an OpenAI‑compatible agent, describing its asynchronous data‑collection pipeline, model training, real‑time visualization, and code architecture for both user and admin interfaces.

AI agentDjangoLSTM
0 likes · 7 min read
Tianqiong ZhiXiang Forecast System with Django, Vue3, LSTM, TensorFlow, AI Agents
Geek Labs
Geek Labs
Aug 20, 2026 · Backend Development

Camofox Browser: An AI‑Stealth Headless Browser with 7.8k Stars that Bypasses Cloudflare Anti‑Scraping

Camofox Browser is an open‑source, AI‑friendly headless browser that disguises its fingerprint, automatically solves Cloudflare JS challenges and Turnstile, offers plug‑and‑play compatibility with Puppeteer/Playwright, and provides a simple CLI for large‑scale data collection and AI agent web automation.

AI agentCamofoxCloudflare bypass
0 likes · 4 min read
Camofox Browser: An AI‑Stealth Headless Browser with 7.8k Stars that Bypasses Cloudflare Anti‑Scraping
Wuming AI
Wuming AI
Aug 9, 2026 · Artificial Intelligence

10 Essential AI Skills I Use Daily (Don’t Hoard Unneeded Skills)

The author curates ten practical AI Skills—ranging from web capture and article deep‑reading to local audio transcription, skill optimization, engineering method libraries, WeChat formatting, video creation, batch photo downloading, and skill discovery—explaining their problem scope, key features, usage steps, and real‑world observations such as token savings, star counts, and limitations.

AI skillsAudio TranscriptionTool Review
0 likes · 12 min read
10 Essential AI Skills I Use Daily (Don’t Hoard Unneeded Skills)
IoT Full-Stack Technology
IoT Full-Stack Technology
Jul 30, 2026 · Backend Development

Build a Super‑Simple Node.js Web Scraper for Douban Top 250

This tutorial walks through creating a minimal Node.js crawler that fetches the Douban Top 250 movie page using the built‑in https module, parses the HTML with cheerio, extracts titles, ratings and images, stores the data in an array, and writes the results to a JSON file.

DoubanHTTPSTutorial
0 likes · 5 min read
Build a Super‑Simple Node.js Web Scraper for Douban Top 250
Python Crawling & Data Mining
Python Crawling & Data Mining
Jul 25, 2026 · Backend Development

Comparing Python’s Four Web‑Scraping Selectors: Regex, BeautifulSoup, XPath, and CSS

This article reviews the four main Python selectors used for web scraping—regular expressions, BeautifulSoup, lxml/XPath, and CSS—detailing their installation, parsing behavior, advantages, drawbacks, and a performance comparison table to help developers choose the most suitable tool for their projects.

BeautifulSoupCSS selectorsPython
0 likes · 8 min read
Comparing Python’s Four Web‑Scraping Selectors: Regex, BeautifulSoup, XPath, and CSS
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 29, 2026 · Backend Development

How the New z‑skills Video Download Split Improves Knowledge‑Gathering Workflows

The author refactors the z‑skills suite by separating video link discovery (z‑web‑pack) from actual downloading (z‑video‑downloader), explains why the split solves performance and reliability issues, details four concrete code and documentation changes, and shows a simple usage pattern for the updated skills.

Pythontool architecturevideo download
0 likes · 7 min read
How the New z‑skills Video Download Split Improves Knowledge‑Gathering Workflows
Frontline Investigation
Frontline Investigation
Jun 28, 2026 · Information Security

Automated Data Collection: The Real Challenge Is Explainable Boundaries, Not Technical Feasibility

China's new draft national standard for automated network data collection tools shifts focus from technical feasibility to explainable compliance, requiring organizations to define collection boundaries, purpose, impact, and audit trails across the entire data lifecycle, especially for AI training data.

AI training dataComplianceTC260 standard
0 likes · 15 min read
Automated Data Collection: The Real Challenge Is Explainable Boundaries, Not Technical Feasibility
AI Architecture Path
AI Architecture Path
Jun 27, 2026 · Backend Development

Turn Your AI Agent into a Web‑Data Pro with Firecrawl’s 139K‑Star Open‑Source Scraper

Firecrawl is a 139K‑star open‑source web‑scraping API that handles dynamic JavaScript pages, full‑site crawling, search, and interactive browsing, offers built‑in proxy rotation and LLM‑ready Markdown/JSON output, and provides detailed code samples and deployment guides that outperform traditional tools like Scrapy and Selenium.

AI agentsFirecrawlLLM integration
0 likes · 12 min read
Turn Your AI Agent into a Web‑Data Pro with Firecrawl’s 139K‑Star Open‑Source Scraper
AI Engineering
AI Engineering
Jun 23, 2026 · Industry Insights

After Three Years of Building Scrapers, This One‑Line Tool Beats It

The article analyzes why acquiring public social‑media data is the biggest obstacle for AI‑driven content apps, compares DIY crawlers with commercial solutions, and demonstrates how RedFox’s unified API lets developers fetch multi‑platform data with a single key, minimal code, and lower maintenance costs.

AI dataPythonREST
0 likes · 10 min read
After Three Years of Building Scrapers, This One‑Line Tool Beats It
Ubiquitous Tech
Ubiquitous Tech
Jun 13, 2026 · Artificial Intelligence

How a Programmer Built an AI Agent Skill to Watch the 2026 World Cup

A developer created an open‑source AI Agent Skill that scrapes real‑time data from the 2026 North‑American World Cup, normalizes it into JSON, and lets anyone ask natural‑language questions about matches, teams, players, and statistics, all with a single command‑line install.

AI agentJSONNode.js
0 likes · 9 min read
How a Programmer Built an AI Agent Skill to Watch the 2026 World Cup
Old Zhang's AI Learning
Old Zhang's AI Learning
Jun 6, 2026 · Artificial Intelligence

How to Build a Personal Knowledge Base with My Custom web‑pack Skill

This article explains how to construct a personal knowledge base using the author’s open‑source web‑pack Skill, which automates raw material collection, image localization, link expansion, and structured output, addressing the limitations of Obsidian’s Web Clipper and aligning with Karpathy’s LLM Wiki three‑layer architecture.

AI agentsLLMObsidian
0 likes · 9 min read
How to Build a Personal Knowledge Base with My Custom web‑pack Skill
liandk
liandk
May 17, 2026 · Fundamentals

Advanced Python Regex & Error Debugging: Extract Complex Data in 5 Minutes

This article extends basic regex knowledge with three advanced techniques, demonstrates how to troubleshoot module‑mirror, URL‑format, and content‑limit errors across nine sample documents, and provides ready‑to‑copy Python code for precise data extraction while avoiding common pitfalls.

PythonURL validationerror handling
0 likes · 14 min read
Advanced Python Regex & Error Debugging: Extract Complex Data in 5 Minutes
liandk
liandk
May 13, 2026 · Backend Development

Python Web Scraping Basics: Extract Strings and Intro to Regex in 5 Minutes

This tutorial walks through fixing common requests‑module errors, selecting a reliable PyPI mirror, formatting URLs correctly, and using built‑in string methods and basic regex to reliably extract IP, keyword and request URL data from HTTP responses, with batch examples and a summary of four common pitfalls.

PythonString Extractionerror handling
0 likes · 12 min read
Python Web Scraping Basics: Extract Strings and Intro to Regex in 5 Minutes
liandk
liandk
May 13, 2026 · Backend Development

Advanced Requests and Simple Web Scraping in Python: Extract Data in 5 Minutes

This tutorial walks you through installing the requests library from reliable mirrors, setting browser-like headers, extracting JSON or raw text data, handling common errors, and performing batch requests to build a quick, functional web scraper in just five minutes.

HTTP headersPythonbatch-requests
0 likes · 11 min read
Advanced Requests and Simple Web Scraping in Python: Extract Data in 5 Minutes
liandk
liandk
May 13, 2026 · Fundamentals

Python Basics – Part 8: Master requests in 5 Minutes to Start Web Scraping

This tutorial walks you through installing the Python requests library, demonstrates basic GET requests, adding query parameters, handling errors with timeout and exceptions, and highlights four common pitfalls with solutions, enabling you to quickly build simple web‑scraping scripts in just five minutes.

HTTPPythonerror handling
0 likes · 8 min read
Python Basics – Part 8: Master requests in 5 Minutes to Start Web Scraping
liandk
liandk
May 10, 2026 · Fundamentals

Python Basics in 3 Minutes: What It Can Do and How to Get Started

This guide explains Python's main uses—automation, web scraping, and simple web apps—walks through Windows/macOS installation, shows how to run the first program, lists four common beginner pitfalls with solutions, and summarizes key takeaways for rapid onboarding.

First programPythonautomation
0 likes · 4 min read
Python Basics in 3 Minutes: What It Can Do and How to Get Started
AI Architecture Path
AI Architecture Path
May 6, 2026 · Backend Development

Scrapling: Self‑Healing Web Scraper That Bypasses Cloudflare and Is 784× Faster Than BS4

Scrapling is an open‑source, adaptive web‑scraping framework that automatically tracks element changes, bypasses Cloudflare and other anti‑scraping defenses, offers multiple fetchers (including stealth mode), and delivers extraction speeds up to 784× faster than BeautifulSoup (BS4) while supporting concurrency, AI integration, and easy CLI usage.

PythonScraplingadaptive crawler
0 likes · 16 min read
Scrapling: Self‑Healing Web Scraper That Bypasses Cloudflare and Is 784× Faster Than BS4
AI Engineering
AI Engineering
May 3, 2026 · Backend Development

Rust‑Based Headless Browser Uses 85% Less Resources Than Chrome

Obscura, an open‑source Rust headless browser designed for AI agents and large‑scale crawling, cuts memory usage by 85%, reduces binary size, speeds page loads to 85 ms, starts instantly, includes strong anti‑detection features, and works with the Chrome DevTools Protocol as a drop‑in replacement for Headless Chrome.

CDPObscuraRust
0 likes · 5 min read
Rust‑Based Headless Browser Uses 85% Less Resources Than Chrome
Wu Shixiong's Large Model Academy
Wu Shixiong's Large Model Academy
Apr 14, 2026 · Artificial Intelligence

Designing High‑Quality Tools for Deep Research Agents: From Search to Python Execution

This article explains how to turn simple API calls into robust, noise‑filtering tools—Search, Visit, Scholar, and Python—by adding domain blacklists, relevance scoring, query‑driven extraction, safety sandboxes, and a unified registry, ultimately boosting the success rate of LLM‑driven research agents.

AI agentsLLM safetyReact
0 likes · 32 min read
Designing High‑Quality Tools for Deep Research Agents: From Search to Python Execution
Three Knives
Three Knives
Mar 24, 2026 · Backend Development

Developer Diary: Building a Handy Web‑to‑Markdown Tool with Jina AI

To overcome large language models' knowledge staleness, the author created JinaReaderTool, a lightweight Java agent tool that fetches any webpage via Jina AI’s free API, converts it to clean Markdown, and can be seamlessly integrated into Feat AI agents for real‑time information retrieval.

Agent toolFeat AIJava
0 likes · 9 min read
Developer Diary: Building a Handy Web‑to‑Markdown Tool with Jina AI
AI Architecture Path
AI Architecture Path
Mar 22, 2026 · Backend Development

How Lightpanda Delivers 11× Faster Headless Browsing with 1/9 Memory Usage

Lightpanda, an open‑source headless browser written in Zig, claims up to eleven times faster execution and one‑ninth the memory of Chrome, offering a lightweight architecture for large‑scale web scraping, AI agent testing, and automation with detailed benchmarks, feature lists, usage commands, and known limitations.

Zigheadless-browserweb scraping
0 likes · 9 min read
How Lightpanda Delivers 11× Faster Headless Browsing with 1/9 Memory Usage
Code Wrench
Code Wrench
Mar 16, 2026 · Backend Development

Mastering Chromedp: Go‑Based Chrome Automation for Real‑World Use Cases

This article provides a comprehensive guide to using the Go library chromedp for Chrome DevTools Protocol‑driven browser automation, covering its core architecture, practical techniques for navigation, network interception, anti‑bot evasion, performance tuning, and production‑grade recommendations with full code examples.

Chrome DevTools ProtocolGobrowser automation
0 likes · 10 min read
Mastering Chromedp: Go‑Based Chrome Automation for Real‑World Use Cases
Su San Talks Tech
Su San Talks Tech
Mar 2, 2026 · Backend Development

How to Use OpenClaw’s Browser Relay for Seamless Web Data Scraping

This guide explains how OpenClaw’s built‑in browser feature and the optional Browser Relay extension enable effortless data extraction from both public and login‑required websites, providing step‑by‑step installation commands, Chrome extension loading instructions, and a practical example of scraping top posts from a social platform.

Chrome extensionOpenClawbrowser automation
0 likes · 6 min read
How to Use OpenClaw’s Browser Relay for Seamless Web Data Scraping
Old Meng AI Explorer
Old Meng AI Explorer
Jan 22, 2026 · Industry Insights

Three Must‑Try Open‑Source AI Tools for Scraping, PPT Creation, and Video Generation

This article reviews three recently popular open‑source AI tools—Spider_XHS for Xiaohongshu data scraping, PPTAgent for one‑click presentation generation, and Code2Video for code‑driven video creation—detailing their key features, quick‑start instructions, and GitHub repositories, all free and ready to deploy.

AI toolsPPT generationVideo Creation
0 likes · 7 min read
Three Must‑Try Open‑Source AI Tools for Scraping, PPT Creation, and Video Generation
AI Insight Log
AI Insight Log
Jan 4, 2026 · Artificial Intelligence

How Playwright + AI Powers a Fully Automated Xianyu Treasure Hunt

The article examines the open‑source ai‑goofish‑monitor project, which combines Playwright‑driven browsing with large‑language‑model analysis to continuously scan Xianyu listings, filter out junk, and highlight high‑quality items, while also discussing its AI‑generated code, benefits, limitations, and security risks.

AILLMPlaywright
0 likes · 7 min read
How Playwright + AI Powers a Fully Automated Xianyu Treasure Hunt
Data Party THU
Data Party THU
Dec 30, 2025 · Big Data

How Python Powers Financial Research: From Web Scraping to Large‑Scale Text Analysis

The "Academic Path" lecture series at Tsinghua University taught non‑CS students practical Python techniques—including Requests, AJAX, Selenium, PDF parsing, and multiprocessing—to extract, process, and analyze financial data, culminating in hands‑on case studies of ESG news, A‑share reports, and IFC databases.

Pythonfinancial datamultiprocessing
0 likes · 4 min read
How Python Powers Financial Research: From Web Scraping to Large‑Scale Text Analysis
Open Source Tech Hub
Open Source Tech Hub
Dec 28, 2025 · Backend Development

Automate Chrome with PHP: Install and Use ChromePHP for Screenshots, PDFs, and More

ChromePHP is a PHP library that integrates the Chrome DevTools protocol, allowing developers to control Chrome or Chromium directly from PHP code for tasks such as navigation, JavaScript execution, screenshots, PDF generation, and input simulation, with simple Composer installation and flexible configuration.

Chrome automationChromePHPDevTools
0 likes · 3 min read
Automate Chrome with PHP: Install and Use ChromePHP for Screenshots, PDFs, and More
Old Meng AI Explorer
Old Meng AI Explorer
Dec 10, 2025 · Operations

How Spider_XHS Turns Xiaohongshu Data Collection into a 10× Efficiency Boost

Spider_XHS is an open‑source Xiaohongshu crawler that automates note, user, comment, and message extraction, offers watermark‑free media downloads, exports structured Excel/JSON data, integrates with the creator platform, and includes proxy and anti‑ban features, enabling marketers and researchers to cut weeks of manual work into hours.

Xiaohongshuautomationdata collection
0 likes · 10 min read
How Spider_XHS Turns Xiaohongshu Data Collection into a 10× Efficiency Boost
Python Programming Learning Circle
Python Programming Learning Circle
Oct 16, 2025 · Backend Development

Why DrissionPage Is the Game‑Changer for Python Web Scraping and Automation

This article introduces DrissionPage, a Python library that merges Selenium and Requests, explains its three page objects, highlights seamless mode switching, built‑in utilities, API changes in version 4.0, and provides practical code examples for web automation, data crawling, and testing.

browser automationdata-extractiondrissionpage
0 likes · 14 min read
Why DrissionPage Is the Game‑Changer for Python Web Scraping and Automation
Alibaba Cloud Developer
Alibaba Cloud Developer
Sep 5, 2025 · Artificial Intelligence

How Browser-Use Leverages LLMs to Transform Browser Automation

This article explores Browser-Use, an AI‑driven browser automation framework that combines large language models, visual perception, and DOM analysis to enable intelligent, multi‑step web tasks such as registration, price comparison, form filling, and monitoring, while detailing its architecture, historical context, core modules, and future challenges.

AI agentsLLMLangChain
0 likes · 26 min read
How Browser-Use Leverages LLMs to Transform Browser Automation
37 Interactive Technology Team
37 Interactive Technology Team
Sep 3, 2025 · Artificial Intelligence

How AI is Revolutionizing Web Scraping: Tools, Techniques, and Best Practices

Discover how AI, especially large language models, transforms traditional web scraping by introducing semantic understanding, dynamic adaptability, and automated extraction, with in‑depth reviews of emerging tools like Crawl4AI and Browser‑use, practical code examples, best‑practice guidelines, and deployment tips for modern data collection.

AICrawl4AILLM
0 likes · 17 min read
How AI is Revolutionizing Web Scraping: Tools, Techniques, and Best Practices
21CTO
21CTO
Aug 19, 2025 · Artificial Intelligence

How Crawl4AI Transforms Web Scraping with AI‑Powered Automation

Crawl4AI is an open‑source AI agent that automates web crawling and data extraction, offering free usage, intelligent parsing, structured JSON/Markdown output, and versatile features like scrolling, multi‑URL scraping, media and metadata extraction, all demonstrated through step‑by‑step Python examples and integration with AI agents.

AI agentsCrawl4AILLM extraction
0 likes · 8 min read
How Crawl4AI Transforms Web Scraping with AI‑Powered Automation
Python Programming Learning Circle
Python Programming Learning Circle
Jul 24, 2025 · Backend Development

Boost Your Python Productivity: 8 Must‑Use Libraries for Faster Development

Discover eight essential Python libraries—Rich, Typer, Pendulum, Pydantic, Faker, tqdm, Requests‑HTML, and Loguru—that transform logging, CLI creation, date handling, data validation, mock data generation, progress tracking, web scraping, and logging, showing how leveraging existing tools can dramatically increase development efficiency.

LibrariesLoggingProductivity
0 likes · 8 min read
Boost Your Python Productivity: 8 Must‑Use Libraries for Faster Development
DevOps
DevOps
Jun 16, 2025 · Artificial Intelligence

Unlock AI’s Real‑World Power: 6 Must‑Have MCP Tools with Hands‑On Code

This article reviews six open‑source MCP servers—Bright Data, Graphiti, GitIngest, Terminal, Code Executor, and MindsDB—showing how each extends large language models with web scraping, long‑term memory, code navigation, command‑line control, sandboxed Python execution, and multi‑source data integration, complete with practical code examples.

AI toolsMCPcode execution
0 likes · 9 min read
Unlock AI’s Real‑World Power: 6 Must‑Have MCP Tools with Hands‑On Code
DataFunTalk
DataFunTalk
Jun 11, 2025 · Backend Development

Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration

This article provides a comprehensive overview of web‑scraping technologies, compares popular tools such as requests, BeautifulSoup and Selenium, introduces AI‑assisted crawling with DeepSeek, and walks through practical steps for using BrightData’s platform to collect industry data, complete with ready‑to‑run Python code.

BeautifulSoupBrightDataDeepSeek
0 likes · 13 min read
Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration
Python Programming Learning Circle
Python Programming Learning Circle
Jun 7, 2025 · Backend Development

Master Python Web Scraping: From Requests to Selenium and Scrapy

Learn how to efficiently scrape web pages using Python by exploring multiple approaches—including simple requests with BeautifulSoup, fast parsing with lxml, dynamic content extraction with Selenium, and large‑scale crawling with Scrapy—complete with installation steps, code snippets, and detailed explanations.

BeautifulSoupPythonrequests
0 likes · 10 min read
Master Python Web Scraping: From Requests to Selenium and Scrapy
php Courses
php Courses
May 14, 2025 · Backend Development

Python Advantages for Web Scraping and Core Library Guide

This article outlines Python's advantages for web crawling, introduces core libraries such as Requests, BeautifulSoup, and Scrapy, details a step-by-step development workflow, provides practical code examples for extracting news titles, and highlights important considerations and advanced techniques for robust scraper implementation.

BeautifulSoupPythondata-extraction
0 likes · 5 min read
Python Advantages for Web Scraping and Core Library Guide
Python Programming Learning Circle
Python Programming Learning Circle
Apr 30, 2025 · Backend Development

Python Weather Data Scraping, CSV Export, and Visualization Using Requests, BeautifulSoup, Pandas, and Matplotlib

This article demonstrates how to use Python's requests and BeautifulSoup libraries to scrape current and 14‑day weather data from China Weather, store the results in CSV files, and perform comprehensive visual analysis—including temperature, humidity, AQI, wind direction, and forecast charts—using pandas, numpy, and matplotlib.

Weather Analysisdata visualizationweb scraping
0 likes · 26 min read
Python Weather Data Scraping, CSV Export, and Visualization Using Requests, BeautifulSoup, Pandas, and Matplotlib
Subtle Storm
Subtle Storm
Apr 17, 2025 · Operations

Bypass Slider Captcha in Python Selenium Login Automation

This guide shows how to use Python Selenium to automate login with slider captcha protection by installing dependencies, initializing ChromeDriver, locating elements, generating a human‑like drag trajectory, performing the drag action, and verifying successful login, including advanced image‑processing tips.

Pythonautomationcaptcha-bypass
0 likes · 8 min read
Bypass Slider Captcha in Python Selenium Login Automation
DataFunSummit
DataFunSummit
Feb 25, 2025 · Artificial Intelligence

Collecting High-Quality LLM Training Data and Custom Model Training Guide

This article explains what constitutes high‑quality LLM training data, why large datasets are essential, outlines the step‑by‑step process for collecting, preprocessing, and fine‑tuning models, and highlights the best data sources—including web content, books, code repositories, and news—while noting available free datasets.

AILLMdata collection
0 likes · 9 min read
Collecting High-Quality LLM Training Data and Custom Model Training Guide
DataFunSummit
DataFunSummit
Feb 13, 2025 · Big Data

E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges

This tutorial explains e‑commerce web scraping fundamentals, covering definitions, tool types, data categories, step‑by‑step Python script creation with Requests, BeautifulSoup, and Selenium, provides sample code for Amazon, Walmart, and eBay, discusses challenges like dynamic pages and anti‑scraping measures, and recommends using specialized scraping APIs.

BeautifulSoupBright DataPython
0 likes · 15 min read
E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges
php Courses
php Courses
Dec 26, 2024 · Backend Development

Python vs PHP for Web Scraping: A Comparative Guide

This article compares Python and PHP for web scraping, outlining each language's strengths, ecosystem, performance, learning curve, and community support to help readers decide which tool best fits their project requirements and experience level.

data-extractionprogramming comparisonweb scraping
0 likes · 9 min read
Python vs PHP for Web Scraping: A Comparative Guide
Raymond Ops
Raymond Ops
Dec 23, 2024 · Backend Development

Master Python Proxy Techniques for Efficient Web Scraping

This guide explains the essential proxy concepts for Python web scraping, covering what proxies are, common types, how they protect crawlers, practical usage with the requests library, and the role of proxy pools in improving scraping efficiency.

backend developmentweb scraping
0 likes · 7 min read
Master Python Proxy Techniques for Efficient Web Scraping
Python Programming Learning Circle
Python Programming Learning Circle
Nov 25, 2024 · Backend Development

Python Web Scraping of Fund Holdings Data and Analysis Using Requests, Selenium, and MongoDB

This tutorial demonstrates how to analyze a fund ranking website, construct dynamic URLs, extract six‑digit fund codes, crawl fund holding pages with requests and Selenium, store the results in MongoDB, and finally process the data to identify the most frequently held stocks across thousands of funds.

MongoDBseleniumweb scraping
0 likes · 13 min read
Python Web Scraping of Fund Holdings Data and Analysis Using Requests, Selenium, and MongoDB
Test Development Learning Exchange
Test Development Learning Exchange
Nov 23, 2024 · Operations

Comprehensive Python Automation Scripts for Common Tasks

This article presents a collection of practical Python scripts covering file management, web scraping, email sending, Excel handling, data cleaning, image processing, system monitoring, PDF manipulation, OCR, database interaction, social media posting, testing, and cloud storage, each with clear descriptions and ready‑to‑run code examples.

Pythonautomationdata processing
0 likes · 12 min read
Comprehensive Python Automation Scripts for Common Tasks
Python Crawling & Data Mining
Python Crawling & Data Mining
Nov 8, 2024 · Fundamentals

Extracting Courier Addresses with Python: A Step‑by‑Step Guide

This article walks through a practical Python solution for parsing a list of courier records, extracting province information, grouping entries by location using dictionaries, and optionally applying regular expressions and pandas for further analysis, all illustrated with clear code examples and output screenshots.

DictionariesPandasPython
0 likes · 11 min read
Extracting Courier Addresses with Python: A Step‑by‑Step Guide