How GPULlama3.java Brings GPU‑Accelerated Llama 3 to Pure Java

0 Introduction

Manchester University's Beehive Lab released GPULlama3.java, the first Java‑native Llama 3 implementation that can automatically use GPU acceleration via TornadoVM, allowing developers to run LLM inference in Java without CUDA or native code.

1 Core of GPULlama3.java

TornadoVM is an heterogeneous programming framework that extends OpenJDK and GraalVM, enabling Java programs to run on GPUs, FPGAs, and multi‑core CPUs. It converts Java bytecode to GPU‑executable code at runtime, using annotations such as @Parallel to mark methods for parallel execution.

TaskGraph taskGraph = new TaskGraph("computation")
    .transferToDevice(DataTransferMode.FIRST_EXECUTION, data)
    .task("process", MyClass::compute, input, output)
    .transferToHost(DataTransferMode.EVERY_EXECUTION, output);

TornadoExecutionPlan executor = new TornadoExecutionPlan(taskGraph.snapshot());
executor.execute();

The runtime automatically handles device‑specific optimizations, memory management and data transfers, so the same Java code can run on different hardware platforms.

2 Supported Back‑ends

NVIDIA GPUs : OpenCL and PTX back‑ends.

Intel GPUs : Includes Arc and integrated HD Graphics, OpenCL support.

Apple Silicon : M1/M2/M3 can run via OpenCL (future Metal back‑end in development).

Command‑line options allow selecting the desired back‑end, for example:

# Run with GPU acceleration (example from README)
./llama-tornado --gpu --verbose-init --opencl --model beehive-llama-3.2-1b-instruct-fp16.gguf --prompt "Explain the benefits of GPU acceleration."

3 Project Requirements

Java 21 or newer (to use Vector API and Foreign Memory API).

Supports GGUF model format for easy packaging and deployment.

Supports quantization formats Q4_0 and Q8_0 to reduce memory usage.

4 Related Java LLM Projects

JLama – modern Java LLM inference engine with distributed deployment.

Llama3.java – CPU‑optimized pure Java implementation.

These projects illustrate the growing AI/ML capabilities of the Java ecosystem, enabling developers to build LLM‑driven applications without leaving the Java platform.

5 Current Status

GPULlama3.java is in a testing phase; the team is optimizing performance and collecting benchmark data. Because Apple has deprecated OpenCL, performance on Apple Silicon is limited, and a Metal back‑end is under development.

6 Conclusion

GPULlama3.java marks a significant step for Java in GPU‑accelerated LLM inference. With TornadoVM, Java developers can obtain high‑performance GPU computing while staying within a familiar language and ecosystem, making it attractive for enterprise scenarios that demand security, scalability, and maintainability. The project is open‑source on GitHub and includes documentation and examples for quick onboarding.