Understanding Deep Learning with TensorFlow: Applications, Computational Graphs, and MLP Implementation

Yin Yajun, a 2018 master's graduate of Beijing University of Posts and Telecommunications, has been interning at Huajiao since the second half of 2017 and now works as an algorithm engineer in the Huajiao Live intelligent engineering team, focusing on personalized recommendation and image recognition algorithms.

1. Deep Learning Applications at Huajiao Live

Data cleaning is performed with Spark to construct user and item profiles, extract features, and store the resulting datasets in HDFS. TensorFlow serves as the deep‑learning computation framework, with jobs scheduled by Hbox and distributed training across a cluster. Trained models are deployed via TensorFlow Serving, wrapped as TF‑Web services for internal prediction APIs, and served online by a Go server.

2. TensorFlow and TensorSlow

TensorFlow, open‑sourced by Google in 2015, is the most widely used deep‑learning framework, comprising over a million lines of code split between front‑end and back‑end components. To aid understanding of its internals, the GitHub project TensorSlow re‑implements TensorFlow’s core in pure Python, sacrificing performance for clarity.

Deep Learning Overview

Deep learning is a branch of machine learning that studies deep neural network structures and effective training methods. A typical deep model is a feed‑forward neural network (multilayer perceptron) that maps inputs x to outputs y using parameters θ . Hidden layers consist of multiple composite functions, and the model’s loss (cost) function J(θ) measures the distance between predictions and the dataset.

Gradient descent minimizes the loss by iteratively updating parameters in the direction of the negative gradient of J(θ) .

Computational Graph in TensorFlow

A computational graph is a directed acyclic graph (DAG) where nodes represent variables (tensors) and edges represent operations. Nodes are of three types:

Placeholder nodes: input nodes without incoming edges. class placeholder: def __init__(self): self.consumers = [] _default_graph.placeholders.append(self)

Variable nodes: model parameters with initial values. class Variable: def __init__(self, initial_value=None): self.value = initial_value self.consumers = [] _default_graph.variables.append(self)

Operation nodes: functions that take input nodes and produce output tensors. class Operation: def __init__(self, input_nodes=[]): self.input_nodes = input_nodes self.consumers = [] for input_node in input_nodes: input_node.consumers.append(self) _default_graph.operations.append(self) def compute(self): pass

A Graph object binds all nodes together. Calling Graph().as_default() sets the created graph as the global default.

Execution is performed via a Session . The Session.run method first topologically sorts the nodes and then computes each node’s output in order.

class Session: def run(self, operation, feed_dict={}): """Computes the output of an operation""" ...

Forward propagation passes inputs through the graph from placeholders to the output node, while back‑propagation applies the chain rule to compute gradients of the loss with respect to each variable.

TensorFlow registers gradient functions for each operation using RegisterGradient . For example, the sigmoid gradient is defined as:

@RegisterGradient("sigmoid") def _sigmoid_gradient(op, grad): sigmoid = op.output return grad * sigmoid * (1 - sigmoid)

Practical Example: Building an MLP

The following code builds a three‑hidden‑layer multilayer perceptron (MLP) for a classification task, using the TensorSlow‑style API:

# Create a new graph
ts.Graph().as_default()

# Placeholders for input and labels
X = ts.placeholder()
c = ts.placeholder()

# Hidden layer 1
W_hidden1 = ts.Variable(np.random.randn(2, 4))
b_hidden1 = ts.Variable(np.random.randn(4))
p_hidden1 = ts.sigmoid(ts.add(ts.matmul(X, W_hidden1), b_hidden1))

# Hidden layer 2
W_hidden2 = ts.Variable(np.random.randn(4, 8))
b_hidden2 = ts.Variable(np.random.randn(8))
p_hidden2 = ts.sigmoid(ts.add(ts.matmul(p_hidden1, W_hidden2), b_hidden2))

# Hidden layer 3
W_hidden3 = ts.Variable(np.random.randn(8, 2))
b_hidden3 = ts.Variable(np.random.randn(2))
p_hidden3 = ts.sigmoid(ts.add(ts.matmul(p_hidden2, W_hidden3), b_hidden3))

# Output layer
W_output = ts.Variable(np.random.randn(2, 2))
b_output = ts.Variable(np.random.randn(2))
p_output = ts.softmax(ts.add(ts.matmul(p_hidden3, W_output), b_output))

# Cross‑entropy loss
J = ts.negative(ts.reduce_sum(ts.reduce_sum(ts.multiply(c, ts.log(p_output)), axis=1)))

# Optimizer
minimization_op = ts.train.GradientDescentOptimizer(learning_rate=0.03).minimize(J)

# Training loop
feed_dict = {X: np.concatenate((blue_points, red_points)),
             c: [[1, 0]] * len(blue_points) + [[0, 1]] * len(red_points)}
session = ts.Session()
for step in range(2000):
    J_value = session.run(J, feed_dict)
    if step % 100 == 0:
        print("Step:", step, "Loss:", J_value)
    session.run(minimization_op, feed_dict)

Visualizing the decision boundary shows that the model learns complex non‑linear relationships.

Key Takeaways

TensorSlow provides a clear, pure‑Python view of TensorFlow’s core mechanisms.

TensorFlow’s backend is implemented in high‑performance C++ and includes optimizations such as common subexpression elimination and constant folding.

TensorFlow supports distributed execution and GPU acceleration.

A rich set of production tools is available for deploying TensorFlow models.

References

https://colah.github.io/posts/2015-08-Backprop/

www.cs.columbia.edu/~mcollins/ff2.pdf

http://www.cs.cornell.edu/courses/cs5740/2017sp/lectures/04-nn-compgraph.pdf

《TensorFlow 内核剖析》刘光聪

《深度学习》Ian Goodfellow, Yoshua Bengio