Mesh LLM: Layer-Wise Splitting Runs 235B Models on Heterogeneous Hardware
Mesh LLM connects heterogeneous machines into a grid that splits large language models by layers, enabling distributed inference without tensor parallelism, using QUIC for encrypted transport and providing an OpenAI-compatible API, with a web console for monitoring and a marketplace vision for compute sharing.
