Master Data Analysis with Python: From Excel/SQL to Pandas in 10 Steps

01 Import Data

Python can import data from many sources, including .sql databases, Excel files, CSV, HTML tables, and web pages. The pandas library provides read_* functions to load these formats directly into DataFrames.

02 Trust the Website Code

Use the requests library to fetch HTML content and BeautifulSoup to parse tables, such as the Wikipedia list of per‑capita GDP, then sort the data with pandas.

03 Quick Data Preview

Call df.head() to display the first rows of a DataFrame, similar to Excel’s row selection or SQL’s LIMIT clause.

04 Rename Columns

Rename columns with df.rename(columns={'US $':'gdp_per_capita'}) and verify the change using head().

05 Delete Columns

Remove unwanted columns using the del statement, e.g., del df['Rank'], then confirm with another head() call.

06 Convert Data Types

Clean string values using regular expressions ( re.sub) to strip commas, then convert columns to numeric types with pandas, enabling calculations like averaging.

07 Select/Filter Data

Apply boolean indexing to filter rows, e.g., df[df['gdp_per_capita'] > 50000], combine conditions with & (AND) and | (OR), and filter by string patterns.

08 Compute Results

Perform arithmetic operations on numeric columns, such as summing GDP values above a threshold, leveraging pandas’ integration with numpy.

09 Data Visualization

Create charts with seaborn or matplotlib; for example, a histogram ( df['gdp_per_capita'].hist()) shows the distribution of high‑GDP countries.

10 Group and Merge Data

Use df.merge() to join the GDP DataFrame with a World Development Indicators CSV, then group by region with df.groupby('Region'), pivot tables, and drop() to clean the final dataset.