Simplify Multi‑LLM Integration in Rust with the genai Library

In the fast‑growing AI era, numerous large language models (LLMs) each expose different APIs, making multi‑model development cumbersome.

The Rust community created the open‑source genai library to provide a unified, lightweight API for popular LLMs such as OpenAI, Anthropic, Cohere, and Ollama.

Core Advantages of genai

Unified API, simplified development: A single interface abstracts the APIs of major LLM providers, allowing developers to call different models without learning each provider’s specifics.

Native implementation, lightweight and efficient: Implemented in pure Rust without heavy external SDK dependencies, keeping the library small and fast.

Chat‑focused, easy to use: Currently concentrates on text‑chat APIs and provides clear example code, enabling beginners to get started quickly.

Usage Example

The following Rust program demonstrates how to query two models (OpenAI and Anthropic) using genai , showing both single‑response and streaming response handling.

<code>use genai::chat::{ChatMessage, ChatRequest};
use genai::client::Client;
use genai::utils::{print_chat_stream, PrintChatStreamOptions};

const MODEL_OPENAI: &str = "gpt-3.5-turbo";
const MODEL_ANTHROPIC: &str = "claude-3-haiku-20240307";

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let question = "What is the meaning of life?";

    let chat_req = ChatRequest::new(vec![
        ChatMessage::user(question),
    ]);

    let client = Client::default();

    let print_options = PrintChatStreamOptions::from_stream_events(true);

    for model in [MODEL_OPENAI, MODEL_ANTHROPIC] {
        println!("\n===== Model: {} =====", model);
        println!("\n--- Question:\n{}", question);
        println!("\n--- Answer (single response)");
        let chat_res = client.exec_chat(model, chat_req.clone(), None).await?;
        println!("{}", chat_res.content.as_deref().unwrap_or("No answer"));

        println!("\n--- Answer (streaming response)");
        let chat_res = client.exec_chat_stream(model, chat_req.clone(), None).await?;
        print_chat_stream(chat_res, Some(&print_options)).await?;

        println!();
    }

    Ok(())
}
</code>

The example defines two model constants, builds a ChatRequest with the user question, and uses a Client to call exec_chat for a one‑shot response and exec_chat_stream for a streaming response, printing each result.

Future Directions

Support more models: Plans to add AWS Bedrock, Google VertexAI, and others.

Enhanced functionality: Adding multimodal (image, audio) support and function calling.

Performance optimization: Ongoing improvements to code structure and algorithms for better efficiency.

Conclusion

genai offers Rust developers a unified, concise, and efficient API for working with multiple LLMs, dramatically simplifying development and positioning itself as an essential tool in the Rust ecosystem.