How xAI’s Grok 1.5V Adds Multimodal Image Input for Developers

Readers: Users and developers will soon be able to feed images into Grok and receive text‑based answers.

The information is based on xAI’s publicly released developer portal documentation (https://developers.x.ai/python-sdk/sampler/).

Recent screenshots from the xAI developer platform illustrate the upcoming multimodal capabilities.

This marks a breakthrough for Elon Musk’s AI company xAI as it adds multimodal input to its Grok chatbot, allowing image uploads that generate textual responses.

In a blog post, xAI announced that Grok‑1.5V will provide a multimodal model across multiple domains, and the updated developer docs show progress toward this new language model.

The Python SDK example demonstrates how developers can read an image file, set a text prompt, and invoke the xAI SDK to generate a response. https://developers.x.ai/python-sdk/sampler/ This update represents a significant upgrade for the Grok model, which was first released in November 2023 for X Premium Plus subscribers, with the previous update in March (Grok 1.5) improving inference.

The model was trained on publicly available internet sources up to Q3 2023, including curated datasets reviewed by humans, unlike the earlier Grok‑1 which had limited training data.

According to xAI’s blog, Grok‑1.5V narrows the performance gap with GPT‑4 on various benchmarks, though such benchmarks can be gamed if included in training data.

Multimodal conversational chatbots are likely the next frontier in AI, as highlighted by recent announcements from Google I/O and OpenAI’s GPT‑4o, and Grok’s new capabilities aim to catch up with industry leaders.