Building a Traffic Sign Recognition System with Spring Boot and Deeplearning4j

In the era of rapid technological advancement, autonomous driving has become a hot research field, and traffic sign recognition is a crucial component that helps vehicles understand road conditions and obey traffic rules.

This article introduces how to use Spring Boot together with Java Deeplearning4j to build a traffic sign recognition system.

1. Technical Overview

1. Neural Network Choice

The system adopts a Convolutional Neural Network (CNN) because of its excellent performance in image recognition, leveraging local connections, weight sharing, and hierarchical structures to capture key features of traffic signs.

2. Dataset Format

The dataset typically contains image files (e.g., JPEG, PNG) and corresponding label files that indicate the class of each sign. An example directory structure is shown below:

traffic_sign_dataset/
├── images/
│   ├── sign1.jpg
│   ├── sign2.jpg
│   ├── ...
├── labels/
│   ├── sign1.txt
│   ├── sign2.txt
│   ├── ...

Label files may use numeric codes such as 0 for speed limit signs, 1 for prohibition signs, etc.

3. Technology Stack

Spring Boot: an open‑source framework for building enterprise applications with rapid development, auto‑configuration, and easy deployment.

Java Deeplearning4j: a Java‑based deep‑learning library supporting various neural‑network architectures, including CNNs and RNNs, providing efficient computation engines and rich tools.

2. Maven Dependencies

Add the following dependencies to pom.xml to include Deeplearning4j and Spring Boot libraries:

<dependency>
    <groupId>org.deeplearning4j</groupId>
    <artifactId>deeplearning4j-core</artifactId>
    <version>1.0.0-beta7</version>
</dependency>
<dependency>
    <groupId>org.deeplearning4j</groupId>
    <artifactId>deeplearning4j-nn</artifactId>
    <version>1.0.0-beta7</version>
</dependency>
<dependency>
    <groupId>org.deeplearning4j</groupId>
    <artifactId>deeplearning4j-ui</artifactId>
    <version>1.0.0-beta7</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>

3. Code Examples

1. Data Loading and Preprocessing

The following Java class loads images and labels, converts them to INDArray , normalizes the data, and returns a ListDataSetIterator :

import org.datavec.image.loader.NativeImageLoader;
import org.deeplearning4j.datasets.iterator.impl.ListDataSetIterator;
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.dataset.DataSet;
import org.nd4j.linalg.dataset.api.preprocessor.DataNormalization;
import org.nd4j.linalg.dataset.api.preprocessor.ImagePreProcessingScaler;
import java.io.File;
import java.util.ArrayList;
import java.util.List;

public class DataLoader {
    public static ListDataSetIterator loadData(String dataDirectory) {
        // Load image files
        File imageDirectory = new File(dataDirectory + "/images");
        NativeImageLoader imageLoader = new NativeImageLoader(32, 32, 3);
        List
images = new ArrayList<>();
        for (File imageFile : imageDirectory.listFiles()) {
            INDArray image = imageLoader.asMatrix(imageFile);
            images.add(image);
        }
        // Load label files
        File labelDirectory = new File(dataDirectory + "/labels");
        List
labels = new ArrayList<>();
        for (File labelFile : labelDirectory.listFiles()) {
            int label = Integer.parseInt(FileUtils.readFileToString(labelFile));
            labels.add(label);
        }
        // Create dataset
        DataSet dataSet = new DataSet(images.toArray(new INDArray[0]), labels.stream().mapToDouble(i -> i).toArray());
        // Normalize data
        DataNormalization scaler = new ImagePreProcessingScaler(0, 1);
        scaler.fit(dataSet);
        scaler.transform(dataSet);
        return new ListDataSetIterator(dataSet, 32);
    }
}

2. Model Construction and Training

The class below builds a CNN model with multiple convolution, pooling, dense, and output layers, then trains it for a number of epochs:

import org.deeplearning4j.nn.conf.ConvolutionMode;
import org.deeplearning4j.nn.conf.NeuralNetConfiguration;
import org.deeplearning4j.nn.conf.layers.ConvolutionLayer;
import org.deeplearning4j.nn.conf.layers.DenseLayer;
import org.deeplearning4j.nn.conf.layers.OutputLayer;
import org.deeplearning4j.nn.multilayer.MultiLayerNetwork;
import org.deeplearning4j.nn.weights.WeightInit;
import org.nd4j.linalg.activations.Activation;
import org.nd4j.linalg.lossfunctions.LossFunctions;

public class TrafficSignRecognitionModel {
    public static MultiLayerNetwork buildModel() {
        NeuralNetConfiguration.Builder builder = new NeuralNetConfiguration.Builder()
                .seed(12345)
                .weightInit(WeightInit.XAVIER)
                .updater(org.deeplearning4j.nn.weights.WeightInit.XAVIER)
                .l2(0.0005)
                .list();
        // Convolution layer
        builder.layer(0, new ConvolutionLayer.Builder(5, 5)
                .nIn(3)
                .stride(1, 1)
                .nOut(32)
                .activation(Activation.RELU)
                .convolutionMode(ConvolutionMode.Same)
                .build());
        // Pooling layer
        builder.layer(1, new org.deeplearning4j.nn.conf.layers.SubsamplingLayer.Builder(org.deeplearning4j.nn.conf.layers.PoolingType.MAX)
                .kernelSize(2, 2)
                .stride(2, 2)
                .build());
        // Additional conv & pooling layers
        builder.layer(2, new ConvolutionLayer.Builder(5, 5)
                .nOut(64)
                .activation(Activation.RELU)
                .convolutionMode(ConvolutionMode.Same)
                .build());
        builder.layer(3, new org.deeplearning4j.nn.conf.layers.SubsamplingLayer.Builder(org.deeplearning4j.nn.conf.layers.PoolingType.MAX)
                .kernelSize(2, 2)
                .stride(2, 2)
                .build());
        // Fully connected layer
        builder.layer(4, new DenseLayer.Builder()
                .nOut(1024)
                .activation(Activation.RELU)
                .build());
        // Output layer (10 classes assumed)
        builder.layer(5, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                .nOut(10)
                .activation(Activation.SOFTMAX)
                .build());
        return new MultiLayerNetwork(builder.build());
    }

    public static void trainModel(MultiLayerNetwork model, ListDataSetIterator iterator) {
        model.init();
        for (int epoch = 0; epoch < 10; epoch++) {
            model.fit(iterator);
            iterator.reset();
        }
    }
}

3. Prediction and Result Display

The following utility loads a new image, normalizes it, and uses the trained model to predict the sign class:

import org.deeplearning4j.nn.multilayer.MultiLayerNetwork;
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.dataset.api.preprocessor.DataNormalization;
import org.nd4j.linalg.dataset.api.preprocessor.ImagePreProcessingScaler;
import org.nd4j.linalg.factory.Nd4j;
import java.io.File;

public class Prediction {
    public static int predict(MultiLayerNetwork model, File imageFile) {
        // Load and preprocess image
        NativeImageLoader imageLoader = new NativeImageLoader(32, 32, 3);
        INDArray image = imageLoader.asMatrix(imageFile);
        DataNormalization scaler = new ImagePreProcessingScaler(0, 1);
        scaler.transform(image);
        // Perform prediction
        INDArray output = model.output(image);
        return Nd4j.argMax(output, 1).getInt(0);
    }
}

4. Unit Testing

JUnit tests verify data loading and model training functionality:

import org.deeplearning4j.datasets.iterator.impl.ListDataSetIterator;
import org.deeplearning4j.nn.multilayer.MultiLayerNetwork;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertNotNull;

public class TrafficSignRecognitionTest {
    private MultiLayerNetwork model;

    @BeforeEach
    public void setup() {
        model = TrafficSignRecognitionModel.buildModel();
    }

    @Test
    public void testLoadData() {
        String dataDirectory = "path/to/your/dataset";
        ListDataSetIterator iterator = DataLoader.loadData(dataDirectory);
        assertNotNull(iterator);
    }

    @Test
    public void testTrainModel() {
        String dataDirectory = "path/to/your/dataset";
        ListDataSetIterator iterator = DataLoader.loadData(dataDirectory);
        TrafficSignRecognitionModel.trainModel(model, iterator);
        assertNotNull(model);
    }
}

5. Expected Output

When the system runs, it should correctly classify input traffic‑sign images, e.g., outputting the label for a speed‑limit sign when such an image is provided.

6. References

Deeplearning4j official documentation

Spring Boot official documentation

"Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville