Unlock Powerful Data Analysis with Pandas: A Hands‑On Guide

Import Pandas

We begin by importing the library, typically using import pandas as pd. This short alias avoids naming conflicts while keeping code concise.

Pandas Data Types

Pandas is built around two core data structures: Series (a one‑dimensional labeled array) and DataFrame (a two‑dimensional table that can hold heterogeneous column types).

Loading Data into Pandas

Data can be read directly from CSV files using pd.read_csv('path/to/file.csv', header=0). If the file lacks column headers, set header=None. Pandas automatically infers column types.

Inspecting the DataFrame

Use df.head(n) to view the first *n* rows and df.tail(n) for the last *n* rows.

Rename columns to shorter, space‑free identifiers for easier attribute‑style access.

Determine the number of rows with len(df) or df.shape[0].

Obtain basic statistics via df.describe().

Filtering Data

Extract subsets using column labels, boolean masks, or string methods. For example, df['rain_octsep'] < 1000 creates a boolean Series.

Apply the mask to obtain the filtered DataFrame.

Combine multiple conditions with & and parentheses.

String filtering uses the .str. accessor, e.g., df[df['year'].str.contains('199')].

Indexing Rows

Numeric indexes are accessed with df.iloc[<em>position</em>], while label indexes use df.loc['label']. The ix indexer (now deprecated) mixed both behaviours.

Applying Functions

Use df['year'].apply(lambda x: ...) to transform a single column, or df.applymap(lambda x: ...) to apply a function element‑wise across the entire DataFrame.

Reshaping the DataFrame

Group data with df.groupby('column') and aggregate using mean(), max(), etc.

Multi‑column grouping works similarly.

Unstack pivots a level of the index into columns.

Further unstacking can reshape the DataFrame again.

Pivot combines set_index, sort_index, and unstack. Missing values can be filled with fillna('') or removed with dropna().

Merging DataFrames

Combine two related DataFrames with pd.merge(df1, df2, on='year'). If on is omitted, Pandas attempts to infer common columns.

Quick Plotting with Pandas

Pandas’ df.plot() leverages Matplotlib to produce fast visualizations of trends.

Saving the Processed Data

After cleaning and reshaping, export the DataFrame to CSV for later use.