GPT‑5.1 Codex hits 70.4% on SWE‑Bench – ~26× cheaper

Multiple screenshots show a “Gemini 3.0 Pro” label appearing in the Gemini Enterprise agent model picker, though access remains blocked for general users sighting summary. Devtools strings align with a production‑bound model option, strengthening the case that final wiring is underway devtools strings, with write‑ups documenting recurring sightings across builds TestingCatalog post.

For AI platform owners, this is the clearest enterprise‑grade breadcrumb yet: start drafting routing and fallbacks so you can A/B 3.0 Pro versus your current defaults on day one.

Google’s CEO amplified a Polymarket contract showing a 69% chance Gemini 3.0 ships by Nov 22, which the community reads as a deliberate signal to expect a near‑term launch CEO hint. A separate roundup repeats the same read, framing Sundar’s post as soft confirmation of timing market odds.

So what? Leaders and PMs can prep eval sandboxes and rollout comms now, especially if you plan announcements at or right after AIE Code week.

A “Riftrunner” model id keeps appearing in design arenas and developer consoles, with testers describing it as a larger, more capable variant that matches expected Gemini 3.0 Pro behavior devtools console and outperforming peers on an SVG rendering comparison in creator tests svg comparison, following up on Riftrunner early strings and arena probes.

If you run eval harnesses, add a placeholder lane for Riftrunner so you can drop in the model id the moment it’s routable.

Several well‑followed accounts say Gemini 3 is landing next week, with one tying the reveal to the AI Engineer Code event where Google has launched onstage before timing claim. Posts narrow it further to early week, even “likely on Tuesday,” reinforcing scheduling urgency next week call tuesday hint, while broader sightings threads keep stoking the countdown speculation post.

Practical move: line up side‑by‑side prompts and traffic shaping such that switching a portion of user flows to 3.0 takes minutes, not days.

Creators report strong results from “Nano‑Banana 2,” noting more realistic images, better text rendering, and accurate reflections—pointing to a revamped Google image stack that could ship alongside Gemini 3 creator take. Others explicitly pair Nano‑Banana 2 mentions with Gemini 3 timing chatter paired mention, with more output dumps circulating outputs thread and third‑party workflows already wiring “nano banana” as a selectable node workflow example.

If your product leans on generative visuals, budget time to re‑shoot style guides and update safety filters—the output distribution may shift.

OpenAI’s GPT‑5.1 Codex leads SWE‑Bench with 70.4% accuracy versus Claude Sonnet 4.5 (Thinking) at 69.8%, while costing ~$0.31 per test vs ~$8.26 (~26× cheaper) benchmarks table. Following up on launch-top5 where new code leaderboards surfaced, this run confirms 5.1 Codex as the top value pick for repo‑level bug fixing, with latencies shown alongside the cost deltas SWE‑Bench note, and the public board now reflectable in Vals AI’s pages benchmarks page.

Vals AI reports GPT‑5.1 sets a new state of the art on its Finance Agent Benchmark, edging Claude Sonnet 4.5 (Thinking) by 0.6% on goal completion, with additional gains on LiveCodeBench (jumping from 12th to 2nd) and minor improvements on MMMU/GPQA/IOI finance benchmark post, follow‑up details. For teams prototyping agentic fintech workflows, this narrows the top tier to 5.1 vs Sonnet 4.5, and suggests routing by tool‑use behavior and cost may matter more than small headline margins.

Artificial Analysis’ latest run gives GPT‑5.1 a score of 70, +2 over GPT‑5 at similar reasoning effort, driven largely by TerminalBench improvements; it also used 81M output tokens vs 85M for GPT‑5, cutting run cost to ~$859 from ~$913 index recap. The live dashboard breaks down per‑eval deltas and cost/latency tradeoffs useful for routing and budgeting analysis site.

BEAM introduces ultra‑long conversation evals up to 10M tokens and shows LIGHT—a hybrid of episodic retrieval, working memory, and scratchpad—consistently outperforms relying on huge context windows alone, with average gains reported across models and a clear fade in long‑context models as length grows paper abstract. For agents that must persist across days, this favors explicit memory stacks over bigger windows.

Bridgewater’s AIA Forecaster combines agentic search over high‑quality news, a supervisor that reconciles disparate forecasts, and calibration (e.g., Platt scaling) to match superforecaster accuracy on ForecastBench, beating prior LLM baselines; on a liquid markets set, markets still lead but ensembles with the model improve accuracy paper abstract. For ops, this argues for supervised multi‑agent pipelines over single‑shot judgments.

A conciseness reward model that grants brevity bonuses only when final answers are correct prevents length/training collapse, delivering +8.1% accuracy with ~19.9% fewer tokens on a 7B backbone across math tasks; the bonus fades over training and scales by difficulty paper abstract. This is a practical recipe to cut inference cost in reasoning agents without sacrificing quality.

DeReC (Dense Retrieval Classification) replaces rationale‑generating LLM pipelines with dense evidence retrieval and a classifier, improving RAWFC F1 to 65.58% (from 61.20%) while cutting runtime ~95% (454m→23m). Similar speedups are shown on LIAR‑RAW paper abstract. If you need scalable veracity checks, retrieval+classifier is a strong baseline before spinning up expensive generation.

A new SWE/ML optimization leaderboard with a human baseline shows expert humans at 1.00× speedup, while top LLM‑driven systems achieve ≤0.15× on ML/HPC tasks, implying current agents slow practitioners down for performance tuning despite strong coding scores elsewhere leaderboard post. Use this as a routing signal: keep human‑in‑the‑loop for low‑level optimization and reserve agents for scaffolding, search, and glue code.

A new rubric‑based benchmark and reinforcement learning approach for instruction following is out, providing a repeatable way to grade agent outputs and train toward rubric compliance—useful when subjective spec adherence matters (e.g., tone, structure) paper thread. Expect more agent evals to standardize on rubric scoring with verifiable checks.