Using Open3D for 3D Data Processing: From Depth Maps to Point Clouds and Surface Reconstruction

1. Introduction

Open3D is an open‑source library that enables rapid development of 3D data‑processing software. Its front‑end offers a carefully chosen set of data structures and algorithms in C++ and Python, while the back‑end is highly optimized and parallelized.

3D data structures

3D data processing algorithms

Live reconstruction

Surface alignment

3D visualization

Physically‑based rendering (PBR)

3D machine‑learning support for PyTorch and TensorFlow

GPU‑accelerated core 3D operations

C++ and Python versions available

2. From Python: Depth Map to Point Cloud

2.1 Installation

Supported on Ubuntu, macOS and Windows 10.

<code>conda create -n open3d python=3.7
activate open3d -i https://pypi.tuna.tsinghua.edu.cn/simple
# install
pip install open3d
# verify
python -c "import open3d as o3d; print(o3d.__version__)"
</code>

Test visualization of a sphere (test3d.py):

<code>import open3d as o3d
mesh = o3d.geometry.TriangleMesh.create_sphere()
mesh.compute_vertex_normals()
o3d.visualization.draw(mesh, raw_mode=True)
</code>

2.2 Visualizing a Facial Point Cloud

Open3D supports various 3D file formats such as PCD and PLY.

<code>import pandas as pd
import numpy as np
import open3d as o3d
# pcd = o3d.io.read_point_cloud("./face.ply")
pcd = o3d.io.read_point_cloud("./face.ply")
print(pcd)
print(np.asarray(pcd.points))
o3d.visualization.draw_geometries([pcd],
    zoom=0.3412,
    front=[0.4257, -0.2125, -0.8795],
    lookat=[2.6172, 2.0475, 1.532],
    up=[-0.0694, -0.9768, 0.2024])
</code>

2.3 Converting a Depth Map to a Point Cloud

Depth images captured by TOF or other 3D cameras are first converted to 3‑D coordinates stored in a NumPy array, then visualized with Open3D.

Original CSV depth map:

<code>data_path = "./face.csv"
w = 320
h = 240
data = pd.read_csv(data_path, header=None)
points = np.zeros((w*h, 3), dtype=np.float32)
n = 0
for i in range(h):
    for j in range(w):
        deep = data.iloc[i, j]
        points[n][0] = j
        points[n][1] = i
        points[n][2] = deep
        n += 1
pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(points)
print("==========")
print(pcd)
print(np.asarray(pcd.points))
print("==========")
o3d.visualization.draw_geometries([pcd],
    zoom=0.3412,
    front=[0.4257, -0.2125, -0.8795],
    lookat=[2.6172, 2.0475, 1.532],
    up=[-0.0694, -0.9768, 0.2024])
</code>

Resulting point cloud (simple conversion, no camera intrinsics):

After applying camera intrinsics for proper scaling:

<code>camera_factor = 1
camera_cx = 180.8664
camera_cy = 179.088
camera_fx = 216.75
camera_fy = 214.62
# ... same loops as above but with:
points[n][2] = deep / camera_factor
points[n][0] = (j - camera_cx) * points[n][2] / camera_fx
points[n][1] = (i - camera_cy) * points[n][2] / camera_fy
</code>

Side view:

2.4 Point‑Cloud Segmentation and Clustering

Foreground points are retained, then clustering or plane segmentation can be applied.

<code>import pandas as pd
import numpy as np
import open3d as o3d
import matplotlib.pyplot as plt
# Load CSV and convert to point cloud (same as above, with camera parameters)
# Filter points with depth < 100
points = points[points[:,2] < 100]
pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(points)
# Plane segmentation
plane_model, inliers = pcd.segment_plane(distance_threshold=0.01,
                                          ransac_n=3,
                                          num_iterations=1000)
[a, b, c, d] = plane_model
print(f"Plane equation: {a:.2f}x + {b:.2f}y + {c:.2f}z + {d:.2f} = 0")
inlier_cloud = pcd.select_by_index(inliers)
inlier_cloud.paint_uniform_color([1.0, 0, 0])
outlier_cloud = pcd.select_by_index(inliers, invert=True)
# Bounding boxes
aabb = pcd.get_axis_aligned_bounding_box()
aabb.color = (1, 0, 0)
obb = pcd.get_oriented_bounding_box()
obb.color = (0, 1, 0)
# Visualize outliers and bounding box
o3d.visualization.draw_geometries([outlier_cloud, aabb],
    zoom=0.3412,
    front=[0.4257, -0.2125, -0.8795],
    lookat=[2.6172, 2.0475, 1.532],
    up=[-0.0694, -0.9768, 0.2024])
</code>

3. Surface Reconstruction

3.1 Poisson Reconstruction

When only an unstructured point cloud is available, surface reconstruction creates a dense triangular mesh. Open3D implements Poisson reconstruction, which first requires normal estimation.

<code>pcd.normals = o3d.utility.Vector3dVector(np.zeros((1,3)))  # invalidate existing normals
pcd.estimate_normals()
</code>

Poisson reconstruction can generate triangles even in low‑density regions, potentially extrapolating geometry.

3.2 Alpha Shapes Reconstruction

Alpha shapes generalize the convex hull and can be used for surface reconstruction.

<code>tetra_mesh, pt_map = o3d.geometry.TetraMesh.create_from_point_cloud(pcd)
for alpha in np.logspace(np.log10(2.5), np.log10(0.1), num=2):
    print(f"alpha={alpha:.3f}")
    mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_alpha_shape(
        pcd, alpha, tetra_mesh, pt_map)
    mesh.compute_vertex_normals()
    o3d.visualization.draw_geometries([mesh], mesh_show_back_face=True)
</code>

3.3 Ball Pivoting Algorithm

The Ball Pivoting Algorithm (BPA) is another method related to alpha shapes.

<code>radii = [0.005, 0.01, 0.02, 0.04]
rec_mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(
    pcd, o3d.utility.DoubleVector(radii))
o3d.visualization.draw_geometries([pcd, rec_mesh])
</code>

*This article is compiled from online sources; all rights belong to the original authors. Please contact us for removal or licensing requests.*