5 Elegant NumPy Functions for Efficient Data Processing

Introduction

The author shares five elegant NumPy functions that enable efficient and concise data processing.

Reshape with -1

NumPy can reshape an array to a new shape compatible with the original. Using -1 as a dimension tells NumPy to infer that size automatically.

a = np.array([[1, 2, 3, 4],
               [5, 6, 7, 8]])
a.shape
# (2, 4)

Example: specifying rows=1, columns=-1 results in an array with shape (1, 8).

a.reshape(1, -1)
# array([[1, 2, 3, 4, 5, 6, 7, 8]])

Specifying rows=-1, columns=1 yields shape (8, 1).

a.reshape(-1, 1)
# array([[1], [2], [3], [4], [5], [6], [7], [8]])

Only one dimension may be set to -1. Incompatible shapes or multiple -1 values raise a ValueError.

a.reshape(-1, -1)
# ValueError: can only specify one unknown dimension.

Argpartition

np.argpartition

efficiently finds the indices of the N largest elements.

array = np.array([10, 7, 4, 3, 2, 2, 5, 9, 0, 4, 6, 0])
index = np.argpartition(array, -5)[-5:]
# array([6, 1, 10, 7, 0])
np.sort(array[index])
# array([5, 6, 7, 9, 10])

Clip

The np.clip function limits array values to a specified interval, truncating values outside the bounds.

array = np.array([10, 7, 4, 3, 2, 2, 5, 9, 0, 4, 6, 0])
print(np.clip(array, 2, 6))
# [6 6 4 3 2 2 5 6 2 4 6 2]

Extract

np.extract

returns elements that satisfy a given condition.

arr = np.arange(10)
condition = np.mod(arr, 3) == 0
np.extract(condition, arr)
# array([0, 3, 6, 9])

Complex conditions can be combined with logical operators:

np.extract(((arr > 2) & (arr < 8)), arr)
# array([3, 4, 5, 6, 7])

setdiff1d

np.setdiff1d

returns the unique elements in one array that are not present in another.

a = np.array([1,2,3,4,5,6,7,8,9])
b = np.array([3,4,7,6,7,8,11,12,14])
c = np.setdiff1d(a, b)
# array([1, 2, 5, 9])

Conclusion

These five NumPy functions are rarely highlighted in the community, yet they offer concise and elegant solutions for complex data manipulation, reducing code size and improving readability.