Grok 4.1 tops Arena at 1483 Elo – wins 64.8% rollout tests

Grok 4.1 is now selectable on grok.com as a standalone Beta in the model picker, with both “thinking” and “non‑thinking” options live for many users Beta rollout, Web picker. xAI’s announcement frames it as more concise, higher intelligence per token and broadly available across grok.com, X, iOS and Android, with DeepSearch still toggling the previous model for some sessions xAI post.

Why it matters: Teams can A/B the new behavior in production chats today. If you rely on Grok’s X search, note that DeepSearch may still pin to the older model for now User note.

xAI’s Grok 4.1 vaulted to the top of the community‑run LMArena: the thinking mode landed 1483 Elo at #1 and the non‑thinking variant posted 1465 at #2, ahead of other models’ full reasoning configs Leaderboard update, xAI note. The Arena team also notes a 40+ point jump versus Grok 4 fast from two months ago.

• Expert board: Grok 4.1 (thinking) #1 (1510); non‑thinking #19 (1437) Leaderboard update.
• Occupational board: Grok 4.1 (thinking) shows broad strength across software, science, legal and business domains Occupational boards.

Why it matters: Arena win‑rates translate to fewer edge‑case stumbles in day‑to‑day chats and coding reviews. If you route by model quality, this is a new default to test against Gemini 2.5 Pro and Claude 4.5 Sonnet.

Internal slides show Grok 4.1’s hallucination rate on info‑seeking prompts down to 4.22% from 12.09% on Grok 4 fast; its FActScore falls to 2.97% from 9.89% (lower is better on both charts) Hallucination charts.

Why it matters: Lower ungrounded claims reduce clean‑up passes in retrieval workflows and lessen the need for strict tool‑forcing—especially useful when you don’t want to pay latency for web search on easy facts.

Shared EQ‑Bench charts place Grok 4.1 (thinking) and Grok 4.1 at the top with normalized Elo 1586 and 1585 respectively, ahead of Kimi K2 and Gemini 2.5 Pro EQ‑Bench chart, EQ and writing. Creative Writing v3 rankings likewise show Grok 4.1 variants pushing into the top tier, only trailing an early GPT‑5.1 Polaris checkpoint EQ‑Bench chart.

Why it matters: If your app needs empathetic, persona‑steady replies (support, sales coaching, tone rewrites), Grok 4.1’s EQ cluster is worth piloting against Claude.

xAI published a Grok 4.1 model card outlining abuse refusal results and propensity metrics; posts cite ~95–98% refusal on clearly violative requests and new input filters for restricted biology/chemistry with low false negatives Model card PDF, Model card sighting. Propensity tables shared by reviewers show sycophancy 0.19–0.23 and deception 0.46–0.49, slightly above Grok 4’s 0.43 deception baseline Propensity table. Meanwhile, a community role‑play “Library of Babel” jailbreak claims to elicit prohibited content from Grok 4.1; prompt and examples are public for repro attempts Jailbreak thread, Prompt GitHub.

Why it matters: Safety posture looks tighter, but red‑teamers are already probing. If you deploy Grok in tools‑enabled contexts, keep test suites current and gate dangerous tool calls behind human review.

A widely shared file purports to show Grok 4.1’s system prompt, including top‑level safety bullets (decline criminal help, keep declines short), product redirects, and a tool list spanning a stateful code interpreter, web search, X keyword/semantic searches, thread fetch, and image/video viewers Prompt leak, Prompt GitHub. Treat as unverified, but the structure matches the product’s observed capabilities.

Why it matters: For integrators, this hints at how Grok arbitrates tool calls and why it sometimes prefers X search over web search. If you wrap Grok, align your system prompts to avoid conflicting directives.

During a quiet two‑week prelaunch, xAI reportedly ran blind pairwise evals on live traffic and Grok 4.1 won 64.78% of comparisons against the incumbent model Rollout notes.

Why it matters: That’s a concrete routing signal. If you manage a meta‑router, weight Grok 4.1 more aggressively on general chat, writing, and ideation flows while you validate corner cases.

OpenAI’s GPT‑5.1 (Thinking, High) posted 72.83% on ARC‑AGI‑1 at ~$0.67/task and 17.64% on ARC‑AGI‑2 at ~$1.17/task, on the ARC Prize semi‑private evals Verified results, with full plots on the official board ARC Prize leaderboard. This follows Vals index where GPT‑5.1 moved up the rankings; today’s numbers show strong price‑performance at verified settings.

Artificial Analysis released AA‑Omniscience, a 6,000‑question, 42‑topic benchmark that rewards correct answers (+1), penalizes wrong answers (‑1), and gives 0 for abstentions; Claude 4.1 Opus tops the Omniscience Index, while Grok 4, GPT‑5, and Gemini 2.5 Pro lead on raw accuracy Benchmark thread, with the paper and public subset available for replication ArXiv paper. • Key takeaways: Hallucination is punished, domain leaders differ (e.g., Business vs Law), and only a few frontier models score slightly above 0 on the Index.

An AA‑Omniscience author clarifies the goal is knowledge reliability—grading whether a model knows specific facts and declines when it doesn’t—rather than being an intelligence test; “hallucination” is defined as answering incorrectly when it should abstain Author reply. The note also stresses domain‑level decisions (e.g., Kotlin knowledge for coding) versus picking a single overall “best” model.

Ethan Mollick argues the benchmark leans on refusal thresholds over true hallucination rates and uses extremely narrow facts, suggesting we need richer error taxonomies and analysis beyond a single score Critique thread. He cites examples of obscure finance and literature queries and asks whether “wrong” answers that express uncertainty should be treated differently.

Lmgame Bench ran a ~60‑hand Texas Hold’em tourney where Gemini‑2.5‑Pro topped the table, DeepSeek‑V3.1 placed second, and Grok‑4‑0709 third; analysis tagged play styles from loose‑passive to loose‑aggressive, showing strategy variance under the same neutral rules Tournament recap. The team notes more rounds will improve the TrueSkill signal; replays and boards are linked in the post.