Master Python’s Top Data Analysis & AI Libraries with Hands‑On Code

Python offers several built‑in features useful for data analysis and mining, such as mutable lists, immutable tuples, dictionary key‑value structures, sets, and functional programming tools (lambda, map, reduce, filter).

Common Python data‑analysis libraries

Below are the most frequently used libraries for data analysis and mining, together with brief introductions and code examples.

NumPy

Provides true n‑dimensional arrays with C‑level performance; it underpins SciPy, Matplotlib, Pandas, etc.

import numpy as np  # alias np
a = np.array([2, 0, 1, 5])
print(a)
print(a[:3])
print(a.min())
a.sort()
print(a)
b = np.array([[1, 2, 3], [4, 5, 6]])
print(b*b)

SciPy

Extends NumPy with advanced scientific functions such as optimization, linear algebra, integration, interpolation, FFT, signal and image processing, and ODE solving.

# Solve a nonlinear system
from scipy.optimize import fsolve
def f(x):
    x1, x2 = x
    return [2*x1 - x2**2 - 1, x1**2 - x2 - 2]
result = fsolve(f, [1, 1])
print(result)

# Numerical integration
from scipy import integrate
def g(x):
    return (1 - x**2)**0.5
pi_2, err = integrate.quad(g, -1, 1)
print(pi_2 * 2, err)

Matplotlib

A popular 2‑D (and simple 3‑D) plotting library.

import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 10, 10000)
y = np.sin(x) + 1
z = np.cos(x**2) + 1
plt.figure(figsize=(8, 4))
plt.plot(x, y, label='$\sin (x+1)$', color='red', linewidth=2)
plt.plot(x, z, 'b--', label='$\cos x^2+1$')
plt.xlim(0, 10)
plt.ylim(0, 2.5)
plt.xlabel("Time(s)")
plt.ylabel("Volt")
plt.title("Matplotlib Sample")
plt.legend()
plt.show()

Pandas

Powerful data‑analysis toolkit built on NumPy, offering Series and DataFrame structures, SQL‑like operations, time‑series handling, and missing‑data support.

import pandas as pd
s = pd.Series([1, 2, 3], index=['a', 'b', 'c'])
d = pd.DataFrame([[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15],[16,17,18]],
                 columns=['a','b','c'])
d2 = pd.DataFrame(s)
print(s)
print(d.head())
print(d.describe())
df = pd.read_csv("G:\\data.csv", encoding="utf-8")
print(df)

Scikit‑Learn

Machine‑learning library that depends on NumPy, SciPy, and Matplotlib, providing preprocessing, classification, regression, clustering, and model evaluation utilities.

from sklearn.linear_model import LinearRegression
model = LinearRegression()
print(model)

Common model methods: model.fit() , model.predict() , model.predict_proba() , model.transform() , model.fit_transform() .

Keras

High‑level deep‑learning API (originally on Theano) for building neural networks such as MLPs, CNNs, and RNNs.

from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.optimizers import SGD
model = Sequential()
model.add(Dense(20, 64))
model.add(Activation('tanh'))
model.add(Dropout(0.5))
model.add(Dense(64, 64))
model.add(Activation('tanh'))
model.add(Dense(64, 1))
model.add(Activation('sigmoid'))
sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='mean_squared_error', optimizer=sgd)
model.fit(x_train, y_train, epochs=20, batch_size=16)
score = model.evaluate(X_test, y_test, batch_size=16)

Gensim

Library for natural‑language processing tasks such as word2vec, LDA, and similarity calculations.

import logging
from gensim import models
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s',
                    level=logging.INFO)
sentences = [['first', 'sentence'], ['second', 'sentence']]
model = models.Word2Vec(sentences, min_count=1)
print(model['sentence'])

This note gives a concise overview of the most common Python tools for data analysis and mining; detailed usage will be covered in future articles.