Step-by-Step TensorFlow Setup on Windows and Build MNIST CNN from Scratch

1. Environment Setup (Windows)

Install Anaconda3 (e.g., Anaconda3 4.2 ) which includes Python 3.5. Create an isolated TensorFlow environment:

conda create -n tensorflow python=3.5  # create env named tensorflow with Python 3.5
activate tensorflow                     # activate the env
# deactivate is not needed here

Set HTTP/HTTPS proxy if the machine is behind a corporate proxy to avoid connection‑refused errors during package installation.

CPU Version

pip install tensorflow                     # install via pip
pip install tensorflow‑cpu‑1.2.1‑cp35‑cp35m‑win_amd64.whl  # install specific wheel

GPU Version

Install matching CUDA and cuDNN versions (e.g., CUDA 8.1 and cuDNN 6). After extracting cuDNN, copy its bin, include and lib folders into the corresponding CUDA directories and add the bin path to PATH.

pip install tensorflow-gpu                     # install GPU package
pip install tensorflow_gpu‑1.2.1‑cp35‑cp35m‑win_amd64.whl

Install additional Python packages as needed (e.g., opencv-python, scipy, Pillow).

2. Understanding TensorFlow Execution

Key concepts:

Tensor : symbolic handle to an operation’s output; actual values are obtained by running a Session or calling eval().

Workflow: define a computation graph → launch a Session → fetch results.

import tensorflow as tf
hello_world = tf.constant('Hello World!', dtype=tf.string)
print(hello_world)  # prints a Tensor object
with tf.Session() as sess:
    result = sess.run(hello_world)
    print(result.decode())

3. MNIST Softmax Linear Regression

MNIST consists of 28×28 grayscale images (784 pixels). A softmax regression model computes logits y = Wx + b and normalizes them to probabilities.

x = tf.placeholder(tf.float32, [None, 784], name="input")
with tf.variable_scope("inference"):
    W = tf.Variable(tf.zeros([784, 10]))
    b = tf.Variable(tf.zeros([10]))
    y = tf.matmul(x, W) + b

Loss is computed with softmax_cross_entropy_with_logits. Accuracy is measured by comparing argmax of predictions and true labels.

loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))
accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.argmax(y,1), tf.argmax(y_,1)), tf.float32))

4. MNIST Deep Convolutional Neural Network (CNN)

The CNN adds layers: reshape → conv → pool → conv → pool → fully‑connected → softmax.

# reshape 1‑D vector to 4‑D image tensor
with tf.name_scope('reshape'):
    inputs = tf.reshape(inputs, [-1, 28, 28, 1])

# convolution helper
def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1,1,1,1], padding='SAME')

# pooling (max or average)
if is_max_pool:
    x = tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')
else:
    x = tf.nn.avg_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')

# dropout
x = tf.nn.dropout(x, keep_prob)

Typical architecture:

[1x28x28x1] → conv(32) → pool → conv(64) → pool → fc(1024) → fc(10)

. Training uses an optimizer such as AdamOptimizer or GradientDescentOptimizer.

train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)

5. Utility Tools

A custom tool.py wraps common TensorFlow operations (saving checkpoints, printing variable shapes, etc.). Alternatives include TensorLayer, Keras, and tflearn.

def print_all_variables(train_only=False):
    vars = tf.trainable_variables() if train_only else tf.global_variables()
    for idx, v in enumerate(vars):
        print("var {:3}: {:15}   {}".format(idx, str(v.get_shape()), v.name))

6. CPU & GPU Configuration

# limit CPU usage
sess_config = tf.ConfigProto(device_count={"CPU": 14}, allow_soft_placement=True)
sess_config.intra_op_parallelism_threads = 56
sess_config.inter_op_parallelism_threads = 56
sess = tf.InteractiveSession(config=sess_config)

# place ops on a specific GPU
with tf.device('/device:GPU:2'):
    a = tf.constant([...])
    b = tf.constant([...])
    c = tf.matmul(a, b)

# allow GPU memory growth
sess_config = tf.ConfigProto(allow_soft_placement=True)
sess_config.gpu_options.allow_growth = True
sess_config.gpu_options.per_process_gpu_memory_fraction = 0.8
sess = tf.InteractiveSession(config=sess_config)

Multi‑GPU training requires manual gradient aggregation; TensorFlow provides examples such as the CIFAR‑10 multi‑GPU script.

7. Learning Resources

Google Machine Learning Crash Course

Stanford Machine Learning (Andrew Ng)

Various Chinese blogs and GitHub repositories covering CNNs, VGG, ResNet, etc.

8. Using Tesla Platform

When running on the internal Tesla cluster, avoid OpenCV, use absolute paths, and follow the platform’s command‑line arguments (e.g., --data_dir, --out_model_dir, --batch_size).