Elon Musk’s xAI Announces Grok-1.5 With 128K Context Length

Afteropen-sourcing Grok-1two weeks ago, Elon Musk’s xAI has now announced an upgraded Grok-1.5 model. The new AI startup says Grok-1.5 comes with improved reasoning capabilities and acontext length of 128,000 tokens. The model is not available right away, instead, it will be available to early testers and existing Grok users on the X (formerly Twitter) platform in the coming days.

To showcase Grok-1.5’s problem-solving capability, xAI has benchmarked the model on popular tests. In theMMLU test, Grok-1.5 scored 81.3%(5-shot), higher than Mistral Large and Claude 3 Sonnet. In the MATH test, it scored 50.6% (4-shot), again beating Claude 3 Sonnet. In the next GSM8K test, it scored a whopping 90%, but with 8-shot prompting. Finally, on the HumanEval test, the Grok-1.5 model scored 74.1% with 0-shot.

xAI has also increased the context length from 8K tokens to 128K tokens on the Grok-1.5 model. To evaluate its retrieval capability, the company ran theNIAH test(Needle in a Haystack), and it achieved perfect results.

As this is an incremental model, xAI has not disclosed the parameter size. However, to give you an overview, Grok-1 is trained on314 billion parameters, one of the largest open-source models out there. It’s also based on the Mixture-of-Experts (MoE) architecture. xAI also released the model weights and the architecture under the Apache 2.0 license which is great.

Recently, Anthropic launched its family ofClaude 3 modelswhich have shown great promise and in many cases, the largest Opus model has already outranked OpenAI’s GPT-4 model. OpenAI is said to be working on an intermediateGPT-4.5 Turbomodel andGPT-5is also on the cards and may launch in the summer of 2024. Google’sGemini 1.5 Promodel has also demonstrated incredible multimodal capabilities over a long context window.

Among the powerful proprietary models, xAI’s Grok-1.5 sits somewhere in the middle, if we go by its benchmark numbers. We have to wait and see how well it does on reasoning tests. Anyway, what do you think about the Grok-1.5 model? Let us know in the comments below.

Arjun Sha

Passionate about Windows, ChromeOS, Android, security and privacy issues. Have a penchant to solve everyday computing problems.

Add new comment

Name

Email ID

Δ

01

02

03

04

05