Eight Essential Python Techniques for Efficient Data Analysis

Data analysis is a major application of Python; whether you are entering a Kaggle competition or developing a deep‑learning project, the first step is always data analysis.

This article introduces eight Python methods that can improve execution speed and make code more elegant.

One‑line list definition – Instead of a verbose for loop, a list comprehension creates a list in a single line.

x = [1, 2, 3, 4]
out = []
for item in x:
    out.append(item**2)
print(out)  # [1, 4, 9, 16]

# vs.
x = [1, 2, 3, 4]
out = [item**2 for item in x]
print(out)  # [1, 4, 9, 16]

Lambda expressions provide a concise way to define small, anonymous functions.

double = lambda x: x * 2
print(double(5))  # 10

Map and Filter work hand‑in‑hand with lambda to apply operations across iterables.

# Map example
seq = [1, 2, 3, 4, 5]
result = list(map(lambda var: var * 2, seq))
print(result)  # [2, 4, 6, 8, 10]

# Filter example
seq = [1, 2, 3, 4, 5]
result = list(filter(lambda x: x > 2, seq))
print(result)  # [3, 4, 5]

NumPy’s arange and linspace generate numeric sequences with different semantics.

# arange(start, stop, step)
np.arange(3, 7, 2)  # array([3, 5])

# linspace(start, stop, num)
np.linspace(2.0, 3.0, num=5)
# array([2.0, 2.25, 2.5, 2.75, 3.0])

Axis concept in pandas/NumPy – specifying axis=1 operates on columns, axis=0 on rows.

df.drop('Column A', axis=1)
df.drop('Row A', axis=0)

df.shape  # (number_of_rows, number_of_columns)

DataFrame concatenation, merging, and joining – analogous to SQL operations, they combine tables in different ways.

Concat appends DataFrames vertically or horizontally; Merge joins on key columns; Join aligns on index or column names.

pandas apply applies a function along a specified axis, avoiding explicit loops.

df = pd.DataFrame([[4, 9]] * 3, columns=['A', 'B'])
print(df)
#    A  B
# 0  4  9
# 1  4  9
# 2  4  9

print(df.apply(np.sqrt))
#    A    B
# 0 2.0 3.0
# 1 2.0 3.0
# 2 2.0 3.0

print(df.apply(np.sum, axis=0))
# A    12
# B    27

print(df.apply(np.sum, axis=1))
# 0    13
# 1    13
# 2    13

Pivot tables in pandas provide Excel‑like summarisation of data.

pd.pivot_table(df, index=['Manager', 'Rep'])
pd.pivot_table(df, index=['Manager', 'Rep'], values=['Price'])

These concepts and functions together give you a toolbox to write more concise, efficient, and readable Python code for data‑analysis projects.