Grok, Elon Musk’s chatbot, becomes multimodal: it can now process images

X (formerly Twitter) has shared its progress with the multimodal capabilities of its Artificial intelligence Grok generative (AI), which can process images and visual content, to answer questions related to said content.

The platform led by Elon Musk presented its chatbot in November of last year, which was launched in a trial version for Premium+ subscribers, with capabilities to offer responses to users in real time. This tool is powered by the language model of the same name, of which version Grok-1.5 was announced at the end of March, which includes improvements in its performance capacity and in tasks related to coding and mathematics.

Recently, X introduced a new update to its model with the Grok-1.5V, which added multi-modal capabilities. That is, it can process visual information, such as documents, tables, diagrams, screenshots, graphs and photographs.

Now, Elon Musk’s AI company, xAI, has shared Grok’s progress in processing visual information and has confirmed that the chatbot is capable of analyzing and processing images, as well as answering related questions, with the implementation of Grok-1.5V .

This is reflected in the updated documents of a software development kit (SDK), which shows a test of multimodal AI, which is now available to developers.

In concert, the lines of Python code specify how the xAI SDK library can be used for the chatbot to generate a response based on both text and images. Thus, in the shared example, the code details that the AI ​​receives the image and reads it to analyze it. After that, the chatbot receives the question “What is this?” and use the xAI SDK to resolve them.

As recently shown by nutritional information and calculate the calories of the food it belongs to, or even create a story from a drawing.

With all this, X continues to advance the AI ​​capabilities of its chatbot, so the multimodal capabilities of the Grok chatbot will reach users generally in the future.