A step-by-step Python tutorial demonstrating how to scrape book details (publisher, ISBN, ratings) from Douban using search suggestion API and XPath parsing, then merge with Excel data for statistical analysis and visualization with pandas and matplotlib.
This comprehensive DuckDB tutorial explains why loading large CSV files into Pandas is inefficient, demonstrates how DuckDB can query CSV, Parquet, and JSON directly, walks through SQL basics, advanced features like window functions and CTEs, compares performance against Pandas on ten‑million‑row datasets, and provides practical tips for tuning, common pitfalls, and when to choose DuckDB for data analysis.
A head‑to‑head benchmark on a 2.3 GB CSV (~1.2 million rows) shows Pandas exhausting memory, Polars completing the pipeline in 8.7 seconds with modest RAM, and DuckDB answering the same query in just 12 milliseconds, highlighting distinct trade‑offs for Python data processing.
This article benchmarks seven common pandas tasks—arithmetic, conditional mapping, dictionary lookup, string cleaning, group aggregation, binning, and query/eval—showing how vectorized alternatives can speed up loops by up to 7,000× and offering practical guidance on when each technique delivers meaningful gains.
Polars outperforms Pandas on large‑scale ETL by using a multithreaded lazy execution model, columnar Arrow storage, and query optimizations, delivering up to 94× speedups on 10 GB workloads, while Pandas remains suitable for smaller datasets and tight ML ecosystem integration.
Even with 99 % accuracy, tens of thousands of errors remain in a 300 k‑row dataset, so the author builds a Python‑AI pipeline that preprocesses images, performs high‑precision OCR, merges data, applies custom validation rules, and automatically generates an error report, dramatically reducing manual effort.
This article demonstrates how to use Python and pandas to automatically structure, clean, and combine tens of thousands of OCR‑generated data rows from multiple files into a single Excel workbook, eliminating manual copy‑paste and ensuring uniform fields and formats.
The article shows that Polars, a query‑compiling DataFrame library, can accelerate ten‑million‑row GroupBy workloads by 6‑10× compared with Pandas, explains the underlying optimizer, Arrow columnar engine and Rust parallelism, provides a 20‑item syntax map, three real migration scenarios, streaming for out‑of‑memory data, and AI‑pipeline use cases, and offers a step‑by‑step migration guide.
This article walks through the entire feature‑engineering pipeline—data cleaning, missing‑value imputation, encoding, outlier handling, scaling, feature construction, and selection—using Pandas and Scikit‑learn, and shows how to wrap the steps into a reproducible Scikit‑learn Pipeline.
This article walks through a full feature‑engineering pipeline using Pandas and Scikit‑Learn, covering data inspection, missing‑value imputation, categorical encoding, outlier handling, scaling, feature construction, selection, and a final Pipeline that prepares clean, predictive features for a logistic‑regression model.
The article explains why good features matter more than fancy algorithms and walks through practical techniques—scaling, log transforms, binning, interaction, various encoding schemes, datetime extraction, text statistics, geospatial distances, aggregation, feature selection, and automated feature generation—illustrated with concrete pandas and scikit‑learn code examples.
The article presents ten built‑in Pandas vectorized operations—such as np.select, assign, cut/qcut, melt/pivot_table, describe, query, transform, to_datetime, explode, and string accessor methods—showing concise one‑liners, their verbose equivalents, and the typical speed gains they deliver on large DataFrames.
This article walks through a complete data‑science pipeline for autonomous driving, covering CSV/JSON preprocessing with pandas, rule‑based scene classification, a weighted complexity‑risk model, and visual analytics using Matplotlib and Seaborn to identify high‑risk driving scenarios.
The article explains why deeply nested np.where calls hurt readability, speed, and maintainability, and demonstrates how using pandas assign() with boolean masks and loc can replace them, delivering 2‑4× faster execution on large DataFrames while keeping the logic clear and extensible.
This article explains how Pyodide compiles CPython to WebAssembly, allowing developers to run complete Python—including libraries like Pandas, NumPy, and Matplotlib—directly in the browser without any server, installation, or build system, and demonstrates a practical CSV‑viewer example.
The article presents ten powerful Pandas query methods—such as .query(), .isin(), .between(), .str.contains(), .loc, .iloc, .nlargest/.nsmallest, .where/.mask, and .eval()—showing how each can replace verbose code, improve readability, and dramatically speed up data‑analysis pipelines.
The article explains why CSV becomes a bottleneck for large‑scale data, demonstrates how Parquet’s columnar, typed, and compressed format dramatically reduces storage, speeds up reads, and improves data safety, and provides step‑by‑step Python code for migrating and benchmarking the switch.
The article shows why a naïve Pandas groupby on a 25‑million‑row DataFrame can take 40 minutes, identifies common performance killers, and demonstrates that converting the grouping column to the categorical dtype with observed=True and sort=False reduces runtime to about 4 seconds while also cutting memory usage dramatically.
Learn how to create a modular Python script that batch processes Excel files—merging, cleaning, renaming, and more—using pandas and openpyxl, with a clear project structure and extensible design for real-world office automation.
This guide explains how to combine a master XLS file with a detailed XLSX file by matching the "Name" column, handling non‑standard headers, skipping index columns, inserting new fields at a specific position, and exporting the merged result using Python, pandas, xlrd, and openpyxl.
This guide introduces seven powerful Python libraries—Pandas, Selenium, Flask, Scrapy, Requests, Faker, and Pillow—explaining their core uses, installation commands, and example code snippets to help developers automate tasks, streamline workflows, and accelerate development.
This article explains how using pandas loc and iloc inside Python loops can be extremely slow, demonstrates the performance difference with concrete timing examples, and shows that replacing them with the scalar accessors at and iat can accelerate the operation by up to sixty times.
This article explains two practical techniques—histogram‑based per‑column synthesis and Gaussian‑Mixture‑Model generation—for creating large, privacy‑preserving synthetic datasets that retain the statistical distributions and inter‑column relationships of the original data, and shows how to evaluate their quality.
This article compares Polars with Pandas, Tidyverse, and Base R, showing how Polars’ Rust implementation, Arrow integration, lazy execution and expressive API deliver 5‑10× speed gains and lower memory usage, and explains why beginners may find Polars easier to learn.
This article demonstrates five practical techniques for loading Excel and CSV data with pandas in Python, including smarter DataFrame construction, parallel processing with Joblib, and using pickle for faster storage, achieving speed improvements of up to three orders of magnitude.
This article visually explores seven of Python’s most influential third‑party libraries—NumPy, Pandas, Matplotlib, Requests, Scikit‑learn, Flask, and Beautiful Soup—detailing their core features, typical use cases, and sample code to help developers quickly grasp how each tool can accelerate data processing, visualization, web development, and machine‑learning projects.
This article walks through a real‑world Pandas issue where British and American date formats are mixed, showing how to preprocess the data, handle empty rows, and apply pd.to_datetime with error coercion to produce clean datetime columns.
This article walks through practical Python data‑analysis steps for vehicle telemetry, covering data import, time‑based calculations, SOC efficiency metrics, speed binning, group aggregation, common coding pitfalls, and tips for clean, reproducible analysis using pandas.
This article presents ten concise, practical one‑line Python code snippets—ranging from loading CSV data with Pandas to building sophisticated Scikit‑learn pipelines—that streamline common machine‑learning tasks such as data cleaning, encoding, splitting, scaling, model training, evaluation, cross‑validation, and prediction.
This tutorial demonstrates how to import, clean, and explore 2013 weather observations from Toulouse Airport using Python libraries such as pandas and SciPy, perform consistency checks, visualize temperature trends, assess variable correlations, and fit probability distributions—including normal, log‑normal, and Weibull—to the data.
This article walks through using Pandas to directly read HTML pages, extract table data, handle AJAX‑loaded JSON and CSV formats, and save results, providing concise code examples and visual steps for effective Python web scraping and data mining.
This article explains why matching Python and Pandas versions is essential, provides a compatibility table, shows how to install the correct Pandas release for a given Python version, demonstrates checking current versions, and offers commands for upgrading or downgrading Pandas while recommending virtual environments and official documentation.
This article presents sixteen ready‑to‑use Python pandas examples that cover common Excel tasks such as reading, writing, sheet selection, cleaning, filtering, aggregation, styling, and exporting, providing a concise toolbox for data‑analysis automation.
This article provides a step‑by‑step guide on installing the openpyxl library, creating new Excel workbooks, reading existing files, applying common operations such as iterating cells, modifying values, styling, merging, freezing panes, adding formulas, adjusting dimensions, and demonstrates practical scenarios including bulk data writes and pandas integration.
This article introduces ten highly practical Python libraries for data analysis—from Pandas and NumPy for data manipulation to Matplotlib, Seaborn, Plotly, Bokeh for visualization, and Scikit‑learn, Prophet, Dask, and PySpark for machine learning and big‑data processing—each illustrated with concise code snippets.
This article explains how large language models combined with Python's data‑science ecosystem can automate metadata extraction, data cleaning, and analysis tasks—illustrated with a step‑by‑step Titanic passenger dataset case study, complete prompts, code snippets, and best‑practice recommendations.
This tutorial provides a detailed overview of Pandas, covering its core data structures, data import/export, selection, cleaning, aggregation, merging, and a practical sales analysis example, with complete code snippets for each operation.
This tutorial introduces Pandas as a core Python library for data processing, demonstrates environment setup, shows how to create and manipulate Series and DataFrames, performs data aggregation and grouping on the Iris dataset, and visualizes results using Matplotlib with extensive code examples.
This article walks through a common Python matplotlib visualization issue, showing how to read parameter data, generate multiple gamma distribution curves, add labels and legends, and produce polished plots, with complete code examples and explanations to help readers replicate the solution.
Python in Excel lets Microsoft 365 users write and run Python code directly within spreadsheets, offering built-in libraries for data analysis, visualization, and machine learning, with cloud execution, seamless cell integration, step‑by‑step activation, sample code, library support, data import, troubleshooting, and practical tips.
This tutorial introduces ten highly practical Python libraries—Pandas, NumPy, Matplotlib, Seaborn, Plotly, Scikit-learn, Dask, PySpark, Bokeh, and Prophet—providing code examples that guide readers through data cleaning, visualization, and predictive modeling to accelerate their data‑analysis expertise.
This article presents a detailed collection of built‑in pandas aggregation methods and numerous custom Python functions for time‑series feature engineering, offering beginners practical tools to enhance data preprocessing and model performance in machine‑learning projects.
The article explains why Python cannot fully replace Excel in finance, outlines spreadsheet pain points such as slowness, data‑size limits, and reproducibility issues, and introduces Python tools like Pandas, Mito, openpyxl, and Lux that can complement and extend Excel workflows.
This guide explains how to install required libraries and use a Python script that leverages pandas and excel2img to automatically capture high‑resolution screenshots of every worksheet in multiple Excel files, streamlining batch processing and boosting productivity.
CSV Trimming is a lightweight Python library that transforms irregular, poorly formatted CSV files into clean, well‑structured tables with a single line of code, supporting basic trimming as well as advanced row‑correlation handling for complex datasets.
This tutorial showcases a comprehensive set of pandas techniques—including reading data from the clipboard, random sampling, multi‑condition filtering, handling missing values, string splitting, list expansion, multi‑function aggregation, slicing, descriptive statistics, categorical conversion, DataFrame styling, and profiling—to efficiently explore and transform DataFrames in Python.
This guide walks you through creating a sample e‑commerce sales Excel file with Pandas and then demonstrates twenty practical data‑cleaning and transformation scripts—including handling missing values, renaming columns, filtering, grouping, and exporting—so you can efficiently prepare sales data for analysis.
Learn a collection of Python scripts that automate common Excel operations—including merging multiple files, find‑and‑replace, filtering rows, adding formulas, adjusting formats, extracting images, and encrypting workbooks—providing step‑by‑step code examples for efficient data processing.
This article walks through a Python community question by presenting two clear approaches—renaming columns to match and using NumPy addition—to sum corresponding columns from two tables, complete with visual code examples and a brief discussion on handling mismatched headers.
This article demonstrates how to replace common Excel operations such as VLOOKUP, data pivot tables, and charting with Python's pandas and plotly libraries, providing code examples, explanations of data import/export settings, and performance considerations for large‑scale data analysis.
This article presents a comprehensive collection of 25 practical pandas techniques, covering version inspection, DataFrame creation, column renaming, prefix/suffix addition, row and column reversal, dtype selection, type conversion, memory optimization, and efficient construction of DataFrames from multiple CSV files.
This tutorial introduces pandas' two core data objects—Series and DataFrame—demonstrates how to create, index, query, modify, add, delete, sort, merge, and copy them, and shows common parameters, functions, I/O operations, plotting, and a practical log‑analysis example using Python.
This guide demonstrates how to read and write common file types—including TXT, CSV, JSON, XML, Excel, YAML, Parquet, and HDF5—as well as how to connect to SQLite databases and perform web scraping using Python's standard libraries and popular third‑party packages.
This tutorial demonstrates how to convert a JSON lyric database into Excel, filter Jay Chou songs, perform Chinese word segmentation with Jieba, compute word frequencies, and create visualizations such as word clouds using Python code and online tools.
This article explains how to build animated line, bar, and pie charts in Python using Matplotlib's FuncAnimation library, demonstrating data preparation, key parameters, code examples, and saving the resulting animations as GIF files.
This article presents a complete data‑analysis workflow that scrapes complaint records from a Chinese platform, cleans and merges the data, performs time‑series analysis, generates visualizations and word‑clouds, and interprets the findings to reveal complaint trends and user demands.
This tutorial demonstrates how to use Python's pandas library to create, read, modify, and manage Excel files, covering basic DataFrame creation, sheet handling, column operations, filtering, merging, pivot tables, charting with matplotlib, performance tips, database integration, and advanced visualizations with seaborn and Plotly.
This article demonstrates how to use Python, pandas, and Plotly to download, clean, and visualize World Happiness Report data from 2015‑2019, revealing the strongest correlations between happiness scores and factors such as GDP per capita, healthy life expectancy, and social support.
This guide walks you through 35 core Pandas commands—from creating DataFrames and inspecting data to filtering, grouping, merging, pivoting, and iterating—providing concise examples that enable quick, effective data manipulation and analysis in Python.
This tutorial demonstrates how to perform essential data preprocessing techniques—including standardization, min‑max normalization, and various missing‑value imputation methods—using pandas and scikit‑learn in Python, providing code examples and explanations to help you prepare datasets for machine‑learning models.
This tutorial introduces Pandas, a powerful Python data‑analysis library, covering installation, importing, creating DataFrames from various sources, basic inspection, selection, filtering, sorting, grouping, handling missing values, and a practical stock‑price analysis example with code snippets.
This tutorial demonstrates how to import pandas, create a sample DataFrame, and apply the DataFrame.query method with SQL‑like expressions—including basic, multiple, OR, IN, NOT IN, variable, and isin conditions—to filter data efficiently.
This tutorial teaches how to handle time series data using pandas, covering datetime types, slicing by date ranges or months, various resampling frequencies, and rolling window calculations such as moving averages and standard deviations, followed by a practical stock price analysis example.
This tutorial guides you through a comprehensive sales data project that covers reading a CSV file, cleaning missing and duplicate entries, grouping by department to compute average sales, and visualizing the results with line and bar charts using pandas and matplotlib.
This tutorial teaches fundamental data cleaning with Pandas, covering how to handle missing values, rename columns, and remove duplicate rows through clear explanations and complete code examples.
This tutorial introduces Pandas fundamentals, demonstrating how to create Series and DataFrames, read and write CSV files, perform basic filtering and sorting, and includes practical code examples and a hands‑on exercise to display the first five rows of a CSV dataset.
This article explains the differences between pandas loc/iloc and at/iat, demonstrates why using loc/iloc inside loops can be extremely slow, and shows how replacing them with at/iat can speed up DataFrame updates by up to 60 times.
This article introduces the ten most commonly used Pandas functions—read_csv, head, tail, info, describe, dropna, fillna, groupby, merge, and to_csv—explaining their purposes and providing clear Python code examples for each in practical data analysis tasks.
This article provides a comprehensive guide to Pandas indexing in Python, covering basic loc and iloc selection, Boolean indexing, setting and resetting indices, multi‑level indexing, index alignment, sorting, dropping, and advanced methods such as at, iat, and query, with complete code examples.
Camelot is a Python library that enables users to extract tables from PDF files into pandas DataFrames, offering simple installation via conda or pip, code examples for reading PDFs, exporting to CSV/JSON, and handling merged cells, making PDF data extraction straightforward.
This article walks through a practical Python solution for parsing a list of courier records, extracting province information, grouping entries by location using dictionaries, and optionally applying regular expressions and pandas for further analysis, all illustrated with clear code examples and output screenshots.
This article compiles 40 practical Python pandas scripts covering Excel file reading, sheet selection, date parsing, column manipulation, data filtering, sorting, grouping, merging, pivot tables, visualization, cleaning, and advanced operations, providing clear examples and output for each step to help data analysts efficiently handle tabular data.
This article introduces Python's built‑in modules and popular libraries for parsing CSV, JSON, and XML files, demonstrates advanced data manipulation with pandas, and presents multiple strategies—including chunked reading, Dask, PySpark, HDF5, databases, Vaex, and NumPy memory‑mapping—for efficiently handling very large datasets.
This article provides a comprehensive guide to data preprocessing using Pandas, covering essential steps like data cleaning, feature engineering, and data transformation for machine learning projects.
This article provides a comprehensive guide to creating visualizations in Python using Matplotlib and related libraries, covering the three‑step workflow of problem definition, data transformation, and parameter tuning, along with detailed code examples for figures, subplots, legends, annotations, saving files, and Pandas‑based plotting.
This step‑by‑step guide shows how to generate a sample Excel file and then use pandas and openpyxl to read, write, update, filter, sort, group, format, chart, and finally save Excel workbooks, covering twenty essential operations for data automation.
This guide provides a comprehensive tutorial on using Python libraries such as pandas, openpyxl, and matplotlib for financial data processing, including reading/writing Excel files, data manipulation, and visualization techniques.
This article explains how to extract a variety of date‑time based features—including date, time, lag, rolling, expanding, and domain‑specific attributes—from a time‑series dataset using pandas, and discusses proper validation strategies for building reliable forecasting models.
This guide introduces Pandas, shows how to set up the environment, and provides detailed explanations and runnable code for common DataFrame operations such as reading CSV files, inspecting data, selecting columns, descriptive statistics, filtering, grouping, merging, and handling missing values.
This article explains how pandas loc and iloc work, demonstrates their slow performance when used inside loops, and shows that using the at and iat accessors provides a dramatically faster alternative for scalar assignments in DataFrames.
This article explains why pandas loc/iloc operations inside Python loops are extremely slow, demonstrates the performance difference with concrete examples, and shows that using the scalar‑access methods at and iat can speed up the same task by dozens of times.
The article reviews pandas 2.0’s major upgrades—including an Apache Arrow backend that speeds up CSV reads by over 30×, new Arrow dtypes, a nullable‑numpy dtype for missing values, a copy‑on‑write memory model, optional dependencies, and benchmark comparisons with ydata‑profiling—highlighting the library’s enhanced performance, flexibility, and interoperability for data‑intensive Python workflows.
This article introduces four Python packages—Pivottablejs, PyGWalker, Qgrid, and Itables—that transform Pandas DataFrames into interactive tables within Jupyter Notebook, providing code examples, visual demos, and guidance on when to choose each tool for data exploration and lightweight analysis.
This article presents ten practical Python scripts—including data analysis with Pandas, web scraping with BeautifulSoup, file renaming, image resizing, PDF generation, email sending, data backup, password generation, a simple HTTP server, and SQLite backup/restore—to help automate common programming tasks.
This article outlines nine common Python mistakes in data‑science projects—such as neglecting virtual environments, overusing notebooks, hard‑coding absolute paths, ignoring warnings, avoiding list comprehensions, missing type hints, writing unreadable pandas chains, disregarding PEP guidelines, and not using coding assistants—providing explanations and code examples to help developers improve code quality and productivity.
This article explains how to use Pandas' query() function to filter DataFrames with simple and complex conditions, covering single‑column filters, logical operators, text matching, arithmetic expressions, built‑in functions, datetime handling, and in‑place updates, all illustrated with clear code examples.
This article explains how to use Python's Matplotlib library and its FuncAnimation feature to build dynamic line, pie, and bar charts—demonstrated with COVID‑19 death data—while covering essential parameters, code examples, and methods for saving the animations.
This guide walks you through installing pandas and openpyxl, reading all sheets from two Excel workbooks, comparing matching sheets to identify differences, analyzing those differences, and saving the results to a new Excel file, with a complete code example provided.
This guide demonstrates how to use Python's pandas library to read Excel files, perform arithmetic, apply mathematical and statistical functions, handle missing values, create custom functions, and save the processed data back to Excel, providing a flexible and efficient workflow for data analysis.
This tutorial explains how to set up a Python environment, create a Tkinter graphical interface, and use pandas and openpyxl to let users select and compare two Excel files, displaying any differences directly in the application.
This article introduces pandas' fillna() method and demonstrates ten practical examples—including basic filling, column‑specific values, forward/backward filling, limiting fills, using other DataFrames, functions, conditional fills, dictionaries, and Series—to help developers effectively handle missing data in Python data analysis.
This tutorial demonstrates how to use Python libraries such as pandas, matplotlib, openpyxl, and python-docx to load, clean, analyze, visualize, and automatically generate Excel and Word reports from CSV data, complete with code examples for each step.
This article explains how Python decorators can enhance Excel data handling by providing use‑cases such as validation, error handling, logging, performance optimization, multithreading, and permission checks, and demonstrates each scenario with clear, runnable code examples.
This article explains how to use Python's parameterized approach to efficiently read, clean, transform, summarize, visualize, and export Excel data through reusable functions and code snippets, enabling flexible and automated data workflows.
This guide demonstrates how to use Python's openpyxl and pandas libraries to automatically sort Excel columns by sales amount, filter rows exceeding a threshold, and extend functionality with regex, providing complete code examples for efficient data processing.
This article introduces how to automate Excel chart generation using Python libraries like pandas and openpyxl, enabling efficient data visualization and analysis.
This article introduces the Pandas library for Python data manipulation, demonstrates how to install and use its basic functions, and shows how the Pandas Profiling package can generate comprehensive data analysis reports with just a single line of code.
This article presents a Python parking‑lot management application, detailing its directory layout, explaining each module such as button handling, OCR utilities, and time processing, and showcasing the main pygame‑based code that renders parking information and visualizes remaining spaces.
This tutorial shows how to use Python libraries such as pandas, python-docx, PyPDF2, and python-pptx to read, write, merge, and manipulate Excel, Word, PDF, and PowerPoint files, providing ready‑to‑run code snippets for each common automation scenario.
This guide presents ten practical Python scripts that cover installing required libraries, reading and writing Excel files, merging sheets, cleaning and filtering data, creating pivot tables, generating charts, styling cells, and automatically emailing Excel reports, providing a complete workflow for Excel automation.
