Java Architecture Diary

Dec 22, 2023 · Artificial Intelligence

Boosting LLM Inference on Java: Vector API, Project Panama & TornadoVM Performance

This article evaluates the performance of large language model (LLM) inference on the Java platform, examining llama2.java implementations that leverage Java Streams, the Vector API, Project Panama, and TornadoVM GPU acceleration, and compares them against native C versions across various model sizes.