HDATF
日本語
相談する
メニュー

HDATF RESEARCH / ベンチマークレポート

実務を完了するAIを、 どう検証するか

公開成績から読み取れることと、実行して確かめること。 モデル選択から調査、文書作成、成果の検証まで。

HDATFリサーチレポート作成 成績資料の基準日

最初に読む結論

公開ベンチマークはモデル選択と試験設計の根拠になります。実務に使えるかは、実際の資料やツールをつなぎ、依頼された成果物を最後まで作れるかで確認します。

使える成果物、確認できる根拠、最後まで続く実行を軸に公開成績を読みます。HDATF自身の実測成績はまだ公開していません。最終章では、その差を確かめる評価計画を説明します。

73 評価と指標684 保存済み成績

01 / 成績の読み方

成績を読む前に、任せる仕事を決める。

ベンチマークは、決められた課題と採点基準で能力を比較する試験です。推論問題、文書比較、ツール操作では測る能力が異なります。まず何を試験した成績なのかを確認します。

報告書なら、ファイルが開き、根拠をたどれ、依頼した内容が含まれる必要があります。データ分析なら、提供資料から結果を再現できることも必要です。一般的なモデル順位だけでは確認できません。

公開成績で候補を絞り、試験すべき失敗を決めます。LabChinに採用する方法は、自分たちの資料、権限、ツール、成果の確認基準を組み合わせた課題で判断します。本稿は成果物、根拠、実行の順に検討します。[1]

02 / 成果物の品質

成果物は、依頼した仕事を基準に読む。

GDPvalは専門業務の成果物を比較します。最初のGold研究では44職種の220課題を扱い、専門家が作成者を知らずに比較しました。下図はその研究から保存した3件の結果です。

勝利と同点の割合は、その成果物比較の結果です。企業の仕事をAIに任せられる割合には換算できません。課題範囲、専門家の判断、モデル構成が解釈の範囲を定めます。[2]

図 01

GDPval Gold

専門家の成果物に対する勝利・同点率 (%)

%高いほど良い
  1. Claude Opus 4.147.6%
  2. GPT-5 (high)38.8%
  3. o3 (high)34.1%
GDPval Gold / %
モデルと実行構成成績出典
Claude Opus 4.1Anthropic47.6%原記録
GPT-5 (high)OpenAI38.8%原記録
o3 (high)OpenAI34.1%原記録

44職種、Gold 220課題。専門家によるブラインド比較の初回研究結果です。現在のモデル順位ではありません。

出典: OpenAI2025-09原記録

HDATFが確認するのは、作成後に人がどれだけ追加作業をするかです。不足した要件、根拠のない数値、修正内容を残します。読みやすい配置も品質の一部ですが、誤った根拠や使えないファイルを補うことはできません。

03 / 調査と根拠

答えを探す力と、根拠で説明する力を確認する。

BrowseCompは見つけにくい事実を探す能力を試験します。Parallelの公開比較は1,266問から無作為に選んだ100問を使用し、正答率と依頼費用を報告しました。下の結果は提供元による一部の問題での実験です。[3][4]

図 02

BrowseComp / Parallel study

正答率 (%)

%高いほど良い
  1. Parallel Ultra8x58%
  2. Parallel Ultra45%
  3. GPT-5 (high)38%
BrowseComp / Parallel study / %
モデルと実行構成成績出典
Parallel Ultra8xParallel58%原記録
Parallel UltraParallel45%原記録
GPT-5 (high)OpenAI38%原記録

1,266問から無作為抽出した100問。Parallelが2025年8月11〜29日に測定。1件の費用：Ultra8x $2.40、Ultra $0.30、GPT-5 $0.488。設定と予算が異なります。

出典: Parallel2025-09-09原記録

Ultra8xは1件2.40ドルで58%、Ultraは0.30ドルで45%と報告しました。予算が異なります。正確さと費用を一緒に読めますが、同一費用での優劣や基盤モデル単独の効果は示しません。

調査報告には追加の確認が必要です。DeepResearch Bench IIとResearchRubricsは明示した基準で内容を評価します。LabChinでは主張を支える原文、矛盾する資料、未回答の問いを残して検証する計画です。

04 / 実行と記憶

同じモデルでも、実行方法で結果は変わる。

ハーネスは、モデルにツールや参考資料を与え、作業を制御する実行ソフトウェアです。LangChainはGPT-5.2-Codexを固定し、この仕組みを変更しました。89課題のTerminal-Bench 2.0成績は52.8%から66.5%になりました。[5][6]

図 03

Terminal-Bench 2.0

課題成功率 (%)

%高いほど良い
  1. Deep Agents / improved66.5%
  2. Deep Agents / baseline52.8%
Terminal-Bench 2.0 / %
モデルと実行構成成績出典
Deep Agents / improvedLangChain66.5%原記録
Deep Agents / baselineLangChain52.8%原記録

89課題。GPT-5.2-Codexを固定し、指示、道具、実行制御を変更したLangChainの実験。Harbor / Daytonaで実行。差は13.7ポイントです。

出典: LangChain2026-02-17原記録

指示、ツール、実行制御をまとめて変更し、終了前の検査や環境の案内も加えています。13.7ポイントの差は変更全体の結果です。個々の寄与やHDATFでの改善幅は、この数値だけでは決められません。

自分たちの業務では課題とモデルを固定し、成果検査の有無を比較できます。ファイル、検査結果、再試行、所要時間を残し、成功だけでなく人の介入が必要だった失敗も説明します。

記憶は別途試験します。古い情報を想起するだけでなく、訂正とアクセス範囲を正しく扱う必要があります。保存したLoCoMo成績は単一事実の想起に限られます。更新の衝突や利用者間の分離を検証した結果とは扱いません。

05 / HDATFの評価計画

次に確かめるのは、HDATFが実行した実務。

公開研究から必要な問いは整理できます。製品の成績は自ら測定します。代表課題で条件を記録した比較を行い、根拠探索、分析、成果物の完成を別々に確認する計画です。一つの高得点で別の失敗を隠さないためです。

調査やデータ作業では問い、資料の版、最終成果の検査記録を残します。報告書の裏付けのない主張を見つけ、分析の入力データ、実行記録、再現可能な成果を保存します。以下は今後確認する基準です。[1][7][8][9]

評価計画です。HDATFの実測成績は未公開です。

必要な根拠を探せるか

BrowseCompはウェブ上で見つけにくい事実を探す能力を評価します。LabChinでは引用元がその主張を裏づけるかも別に確認する計画です。

BrowseComp

根拠をもとに問いに答えられるか

DeepResearch Bench IIとResearchRubricsを調査報告の評価基準にします。文章の流暢さだけでなく、必要な内容、事実の根拠、結論に至る分析を確認する計画です。

DeepResearch Bench IIResearchRubrics

使える成果物を作れるか

GDPvalとAgents’ Last Examは専門業務を扱います。依頼が完了したか、ファイルを実際に使えるか、人の修正がどの程度必要かを確認する計画です。

GDPvalAgents’ Last Exam

評価対象残す根拠確認する判断
課題依頼内容、入力ファイル、完了基準依頼した仕事が完了したか
実行モデル、ツール、権限、実行構成の版同じ条件で再実行できるか
結果成果ファイル、出典検査、人の修正内容成果物を実際に使えるか
効率全体時間、ツール利用、再試行、総費用業務全体の時間と費用を負担できるか
APPENDIX

付録。評価資料一覧

評価ごとの公開成績を比較できます。モデル名から原文を開けます。表示日の記録であり、モデル系列の比較には各系列で最も高い成績の構成を載せています。

73 / 73 評価と指標

業務と文書

AA-Briefcase15 件の成績

基盤モデルの成績Artificial Analysis

長期の知識労働

Elo高いほど良い
  1. Claude Fable 5.11,661.91
  2. Claude Opus 51,647.02
  3. GPT-6 Astra1,561.95
  4. Muse Spark 1.31,558.85
  5. Grok 4.61,545.82
  6. Claude Fable 51,533.52
  7. GLM-5.31,515.45
  8. Kimi K31,496.83
  9. GPT-5.6 Sol1,474.49
  10. GLM 5.3 Flash1,459.13
  11. Qwen3.8 2.4T A95B1,442.81
  12. DeepSeek V4 Flash 07311,433.23
  13. Qwen3.8 27B1,401.81
  14. Qwen3.8 Max1,392.33
  15. Claude Sonnet 51,358.11
AA-Briefcase / Elo
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic1,661.91原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic1,647.02原記録
GPT-6 Astra (max)OpenAI1,561.95原記録
Muse Spark 1.3 (max)Meta1,558.85原記録
Grok 4.6 (xhigh)SpaceXAI1,545.82原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic1,533.52原記録
GLM-5.3 (max)Z AI1,515.45原記録
Kimi K3 (max)Kimi1,496.83原記録
GPT-5.6 Sol (max)OpenAI1,474.49原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI1,459.13原記録
Qwen3.8 2.4T A95BAlibaba1,442.81原記録
DeepSeek V4 Flash 0731 (DeepSeek V4 Flash Vision (Reasoning, Max Effort))DeepSeek1,433.23原記録
Qwen3.8 27B (xhigh)Alibaba1,401.81原記録
Qwen3.8 MaxAlibaba1,392.33原記録
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)Anthropic1,358.11原記録
AA-Briefcase / Rubric15 件の成績

基盤モデルの成績Artificial Analysis

要件の充足

%高いほど良い
  1. Claude Fable 5.161.52%
  2. Claude Opus 557.98%
  3. Claude Fable 555.96%
  4. Muse Spark 1.354.87%
  5. Grok 4.653.84%
  6. GPT-6 Astra52.32%
  7. GLM-5.351.35%
  8. Kimi K350.97%
  9. Qwen3.8 2.4T A95B50%
  10. Qwen3.8 Max49.6%
  11. Qwen3.8 27B47.8%
  12. GLM 5.3 Flash46.36%
  13. DeepSeek V4 Flash 073146.15%
  14. Muse Spark 1.245.15%
  15. DeepSeek V4 Pro 081343.52%
AA-Briefcase / Rubric / %
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic61.52%原記録
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort)Anthropic57.98%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic55.96%原記録
Muse Spark 1.3 (max)Meta54.87%原記録
Grok 4.6 (xhigh)SpaceXAI53.84%原記録
GPT-6 Astra (xhigh)OpenAI52.32%原記録
GLM-5.3 (max)Z AI51.35%原記録
Kimi K3 (max)Kimi50.97%原記録
Qwen3.8 2.4T A95BAlibaba50%原記録
Qwen3.8 MaxAlibaba49.6%原記録
Qwen3.8 27B (xhigh)Alibaba47.8%原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI46.36%原記録
DeepSeek V4 Flash 0731 (DeepSeek V4 Flash Vision (Reasoning, Max Effort))DeepSeek46.15%原記録
Muse Spark 1.2 (xhigh)Meta45.15%原記録
DeepSeek V4 Pro 0813 (Reasoning, Max Effort)DeepSeek43.52%原記録
AA-Briefcase / Analytical quality15 件の成績

基盤モデルの成績Artificial Analysis

分析の品質

Elo高いほど良い
  1. Claude Fable 5.11,959.49
  2. Claude Opus 51,909.74
  3. GLM-5.31,788.62
  4. GPT-6 Astra1,753.64
  5. Muse Spark 1.31,711.52
  6. Grok 4.61,682.58
  7. Claude Fable 51,676.55
  8. Kimi K31,668.89
  9. GLM 5.3 Flash1,650.95
  10. Qwen3.8 2.4T A95B1,617.45
  11. DeepSeek V4 Flash 07311,617.37
  12. Qwen3.8 27B1,590.24
  13. GPT-5.6 Sol1,537.92
  14. Qwen3.8 Max1,533.14
  15. Claude Sonnet 51,418.22
AA-Briefcase / Analytical quality / Elo
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic1,959.49原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic1,909.74原記録
GLM-5.3 (max)Z AI1,788.62原記録
GPT-6 Astra (max)OpenAI1,753.64原記録
Muse Spark 1.3 (max)Meta1,711.52原記録
Grok 4.6 (xhigh)SpaceXAI1,682.58原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic1,676.55原記録
Kimi K3 (max)Kimi1,668.89原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI1,650.95原記録
Qwen3.8 2.4T A95BAlibaba1,617.45原記録
DeepSeek V4 Flash 0731 (DeepSeek V4 Flash Vision (Reasoning, Max Effort))DeepSeek1,617.37原記録
Qwen3.8 27B (xhigh)Alibaba1,590.24原記録
GPT-5.6 Sol (max)OpenAI1,537.92原記録
Qwen3.8 MaxAlibaba1,533.14原記録
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)Anthropic1,418.22原記録
AA-Briefcase / Presentation15 件の成績

基盤モデルの成績Artificial Analysis

成果物の表現品質

Elo高いほど良い
  1. GPT-5.6 Sol1,633.93
  2. Claude Opus 51,544.01
  3. GPT-6 Astra1,535.29
  4. GPT-5.6 Terra1,519.13
  5. Grok 4.61,514.68
  6. Muse Spark 1.31,506.83
  7. Claude Fable 5.11,472.7
  8. GPT-5.6 Luna1,471.03
  9. Claude Fable 51,456.83
  10. Kimi K31,435.29
  11. Claude Sonnet 51,409.5
  12. GLM 5.3 Flash1,409.12
  13. DeepSeek V4 Flash 07311,355.99
  14. GLM-5.31,354.5
  15. Muse Spark 1.21,334.35
AA-Briefcase / Presentation / Elo
モデルと実行構成成績出典
GPT-5.6 Sol (max)OpenAI1,633.93原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic1,544.01原記録
GPT-6 Astra (max)OpenAI1,535.29原記録
GPT-5.6 Terra (max)OpenAI1,519.13原記録
Grok 4.6 (xhigh)SpaceXAI1,514.68原記録
Muse Spark 1.3 (max)Meta1,506.83原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic1,472.7原記録
GPT-5.6 Luna (max)OpenAI1,471.03原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic1,456.83原記録
Kimi K3 (max)Kimi1,435.29原記録
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)Anthropic1,409.5原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI1,409.12原記録
DeepSeek V4 Flash 0731 (DeepSeek V4 Flash Vision (Reasoning, Max Effort))DeepSeek1,355.99原記録
GLM-5.3 (max)Z AI1,354.5原記録
Muse Spark 1.2 (xhigh)Meta1,334.35原記録
GDPval-AA v215 件の成績

基盤モデルの成績Artificial Analysis

実際の職務課題

Elo高いほど良い
  1. Claude Fable 5.11,765.9
  2. Claude Opus 51,738.08
  3. Muse Spark 1.31,719.65
  4. GLM-5.31,678.47
  5. GLM 5.3 Flash1,672.94
  6. Grok 4.61,664.82
  7. Qwen3.8-Flash-Next1,649.93
  8. Claude Fable 51,635.39
  9. Qwen3.8 Max1,633.53
  10. Qwen3.8 2.4T A95B1,630.39
  11. GPT-5.6 Sol1,625.68
  12. Kimi K31,585.72
  13. GPT-6 Astra1,582.24
  14. DeepSeek V4 Flash 07311,579.3
  15. Muse Spark 1.21,525.03
GDPval-AA v2 / Elo
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic1,765.9原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic1,738.08原記録
Muse Spark 1.3 (max)Meta1,719.65原記録
GLM-5.3 (max)Z AI1,678.47原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI1,672.94原記録
Grok 4.6 (xhigh)SpaceXAI1,664.82原記録
Qwen3.8-Flash-NextAlibaba1,649.93原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic1,635.39原記録
Qwen3.8 MaxAlibaba1,633.53原記録
Qwen3.8 2.4T A95BAlibaba1,630.39原記録
GPT-5.6 Sol (max)OpenAI1,625.68原記録
Kimi K3 (max)Kimi1,585.72原記録
GPT-6 Astra (max)OpenAI1,582.24原記録
DeepSeek V4 Flash 0731 (DeepSeek V4 Flash Vision (Reasoning, Max Effort))DeepSeek1,579.3原記録
Muse Spark 1.2 (xhigh)Meta1,525.03原記録
AA-AnalystAgent / pass^515 件の成績

基盤モデルの成績Artificial Analysis

表計算と文書分析の反復成功

%高いほど良い
  1. Gemini 3.7 Flash60%
  2. Claude Fable 5.157.5%
  3. Claude Opus 553.75%
  4. GPT-6 Astra51.25%
  5. Claude Fable 548.75%
  6. GPT-5.6 Sol47.5%
  7. Claude Sonnet 546.25%
  8. Gemini 3.1 Pro Preview41.25%
  9. Grok 4.641.25%
  10. Kimi K338.75%
  11. Grok 4.535%
  12. Inkling Small27.5%
  13. Inkling23.75%
  14. MiMo-V2.5-Pro20%
  15. DeepSeek V4 Pro 042418.75%
AA-AnalystAgent / pass^5 / %
モデルと実行構成成績出典
Gemini 3.7 Flash (high)Google60%原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic57.5%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic53.75%原記録
GPT-6 Astra (max)OpenAI51.25%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic48.75%原記録
GPT-5.6 Sol (max)OpenAI47.5%原記録
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)Anthropic46.25%原記録
Gemini 3.1 Pro PreviewGoogle41.25%原記録
Grok 4.6 (high)SpaceXAI41.25%原記録
Kimi K3 (max)Kimi38.75%原記録
Grok 4.5 (high)SpaceXAI35%原記録
Inkling SmallThinking Machines27.5%原記録
Inkling (xhigh)Thinking Machines23.75%原記録
MiMo-V2.5-ProXiaomi20%原記録
DeepSeek V4 Pro 0424 (DeepSeek V4 Pro (Reasoning, Max Effort))DeepSeek18.75%原記録
GDP.pdf / All-pass15 件の成績

基盤モデルの成績Artificial Analysis

業務文書の推論

%高いほど良い
  1. GPT-6 Astra33.2%
  2. GPT-5.6 Sol28.2%
  3. Claude Fable 5.128%
  4. Muse Spark 1.325.6%
  5. GPT-5.6 Terra25.6%
  6. Claude Fable 524%
  7. GPT-5.6 Luna23.8%
  8. Gemini 3.7 Flash23.6%
  9. Gemini 3.8 Flash22.8%
  10. Qwen3.8 Max21.8%
  11. Claude Opus 521.6%
  12. GPT-5.5 Instant (June 2026)20.2%
  13. Kimi K319.6%
  14. Grok 4.518.8%
  15. Grok 4.618.8%
GDP.pdf / All-pass / %
モデルと実行構成成績出典
GPT-6 Astra (max)OpenAI33.2%原記録
GPT-5.6 Sol (max)OpenAI28.2%原記録
Claude Fable 5.1 (Adaptive Reasoning, Low Effort, Default Fallback)Anthropic28%原記録
Muse Spark 1.3 (max)Meta25.6%原記録
GPT-5.6 Terra (max)OpenAI25.6%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic24%原記録
GPT-5.6 Luna (xhigh)OpenAI23.8%原記録
Gemini 3.7 Flash (high)Google23.6%原記録
Gemini 3.8 Flash (medium)Google22.8%原記録
Qwen3.8 MaxAlibaba21.8%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic21.6%原記録
GPT-5.5 Instant (June 2026)OpenAI20.2%原記録
Kimi K3 (max)Kimi19.6%原記録
Grok 4.5 (high)SpaceXAI18.8%原記録
Grok 4.6 (high)SpaceXAI18.8%原記録
AutomationBench-AA15 件の成績

基盤モデルの成績Artificial Analysis

業務アプリでの目標達成

%高いほど良い
  1. Gemini 3.7 Flash62.75%
  2. Kimi K352.71%
  3. Grok 4.551.44%
  4. GPT-5.6 Sol51.19%
  5. Gemini 3.6 Flash51.08%
  6. Gemini 3.8 Flash50.72%
  7. Claude Fable 548.58%
  8. GPT-5.6 Terra45.61%
  9. GPT-5.6 Luna42.24%
  10. Claude Sonnet 539.19%
  11. Gemini 3.1 Pro Preview37.54%
  12. Gemini 3.5 Flash-Lite32.66%
  13. GLM-5.227.84%
  14. Kimi K2.7 Code22.53%
  15. Qwen3.7 Plus20.43%
AutomationBench-AA / %
モデルと実行構成成績出典
Gemini 3.7 Flash (high)Google62.75%原記録
Kimi K3 (max)Kimi52.71%原記録
Grok 4.5 (high)SpaceXAI51.44%原記録
GPT-5.6 Sol (max)OpenAI51.19%原記録
Gemini 3.6 Flash (high)Google51.08%原記録
Gemini 3.8 Flash (high)Google50.72%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic48.58%原記録
GPT-5.6 Terra (max)OpenAI45.61%原記録
GPT-5.6 Luna (max)OpenAI42.24%原記録
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)Anthropic39.19%原記録
Gemini 3.1 Pro PreviewGoogle37.54%原記録
Gemini 3.5 Flash-LiteGoogle32.66%原記録
GLM-5.2 (max)Z AI27.84%原記録
Kimi K2.7 CodeKimi22.53%原記録
Qwen3.7 PlusAlibaba20.43%原記録
EnterpriseOps-Gym-AA15 件の成績

基盤モデルの成績Artificial Analysis

企業業務の実行

%高いほど良い
  1. Claude Fable 551.12%
  2. Gemini 3.7 Flash50.4%
  3. DeepSeek V4 Pro 081349.6%
  4. Grok 4.648.34%
  5. Claude Opus 547.48%
  6. Qwen3.8 2.4T A95B47.36%
  7. Muse Spark 1.247.27%
  8. DeepSeek V4 Flash 073147.09%
  9. Kimi K345.33%
  10. Claude Sonnet 544.67%
  11. Qwen3.8 27B44.23%
  12. GPT-5.6 Sol42.91%
  13. GLM-5.242.73%
  14. Inkling Small42.7%
  15. Gemini 3.5 Flash-Lite42.35%
EnterpriseOps-Gym-AA / %
モデルと実行構成成績出典
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic51.12%原記録
Gemini 3.7 Flash (medium)Google50.4%原記録
DeepSeek V4 Pro 0813 (Reasoning, Max Effort)DeepSeek49.6%原記録
Grok 4.6 (high)SpaceXAI48.34%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic47.48%原記録
Qwen3.8 2.4T A95BAlibaba47.36%原記録
Muse Spark 1.2 (xhigh)Meta47.27%原記録
DeepSeek V4 Flash 0731 (DeepSeek V4 Flash Vision (Reasoning, Max Effort))DeepSeek47.09%原記録
Kimi K3 (max)Kimi45.33%原記録
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)Anthropic44.67%原記録
Qwen3.8 27B (xhigh)Alibaba44.23%原記録
GPT-5.6 Sol (max)OpenAI42.91%原記録
GLM-5.2 (max)Z AI42.73%原記録
Inkling SmallThinking Machines42.7%原記録
Gemini 3.5 Flash-LiteGoogle42.35%原記録
APEX-Agents-AA14 件の成績

基盤モデルの成績Artificial Analysis

複数アプリを使う長期業務

%高いほど良い
  1. Kimi K341.3%
  2. GPT-5.6 Terra38.94%
  3. GPT-5.6 Luna35.84%
  4. GLM-5.233.7%
  5. Gemini 3.1 Pro Preview32.01%
  6. Apodex 1.131.19%
  7. DeepSeek V4 Pro 042424.26%
  8. Qwen3.7 Plus22.42%
  9. Qwen3.5 397B A17B15.34%
  10. Step 3.7 Flash14.82%
  11. gpt-oss-120b3.1%
  12. MiMo-V2.5-Pro2.43%
  13. Nemotron 3 Super 120B A12B1.84%
  14. gpt-oss-20b0.74%
APEX-Agents-AA / %
モデルと実行構成成績出典
Kimi K3 (max)Kimi41.3%原記録
GPT-5.6 Terra (max)OpenAI38.94%原記録
GPT-5.6 Luna (max)OpenAI35.84%原記録
GLM-5.2 (max)Z AI33.7%原記録
Gemini 3.1 Pro PreviewGoogle32.01%原記録
Apodex 1.1Apodex31.19%原記録
DeepSeek V4 Pro 0424 (DeepSeek V4 Pro (Reasoning, Max Effort))DeepSeek24.26%原記録
Qwen3.7 PlusAlibaba22.42%原記録
Qwen3.5 397B A17B (Reasoning)Alibaba15.34%原記録
Step 3.7 FlashStepFun14.82%原記録
gpt-oss-120b (high)OpenAI3.1%原記録
MiMo-V2.5-ProXiaomi2.43%原記録
Nemotron 3 Super 120B A12B (Reasoning)NVIDIA1.84%原記録
gpt-oss-20b (high)OpenAI0.74%原記録

システム研究

GDPval Gold3 件の成績

専門家比較の研究OpenAI

専門家が受け入れられる成果物を作れるか。 専門家の成果物に対する勝利・同点率 (%)

%高いほど良い
  1. Claude Opus 4.147.6%
  2. GPT-5 (high)38.8%
  3. o3 (high)34.1%
GDPval Gold / %
モデルと実行構成成績出典
Claude Opus 4.1Anthropic47.6%原記録
GPT-5 (high)OpenAI38.8%原記録
o3 (high)OpenAI34.1%原記録
BrowseComp / Parallel study3 件の成績

調査エージェントの研究Parallel

見つけにくい情報を探し、根拠をつなげられるか。 正答率 (%)

%高いほど良い
  1. Parallel Ultra8x58%
  2. Parallel Ultra45%
  3. GPT-5 (high)38%
BrowseComp / Parallel study / %
モデルと実行構成成績出典
Parallel Ultra8xParallel58%原記録
Parallel UltraParallel45%原記録
GPT-5 (high)OpenAI38%原記録
Terminal-Bench 2.02 件の成績

実行体系の研究LangChain

実行の仕組みを変えると結果はどう変わるか。 課題成功率 (%)

%高いほど良い
  1. Deep Agents / improved66.5%
  2. Deep Agents / baseline52.8%
Terminal-Bench 2.0 / %
モデルと実行構成成績出典
Deep Agents / improvedLangChain66.5%原記録
Deep Agents / baselineLangChain52.8%原記録
LoCoMo / single-hop3 件の成績

記憶システムの研究Mem0 research team

会話をまたいで必要な情報を保持できるか。 単一事実の想起、AI採点 (0〜100)

score高いほど良い
  1. Mem067.13
  2. LangMem62.23
  3. Zep61.7
LoCoMo / single-hop / score
モデルと実行構成成績出典
Mem0Mem067.13原記録
LangMemLangChain62.23原記録
ZepZep61.7原記録

エージェントとツール

τ³-Banking15 件の成績

基盤モデルの成績Artificial Analysis

銀行規程の検索と道具実行

%高いほど良い
  1. Muse Spark 1.352.37%
  2. Qwen3.8 Max51.34%
  3. Grok 4.650.72%
  4. GLM-5.350.31%
  5. Qwen3.8 2.4T A95B49.07%
  6. Qwen3.8 27B48.04%
  7. GLM 5.3 Flash47.22%
  8. Claude Fable 5.147.22%
  9. Kimi K345.98%
  10. Gemini 3.8 Flash45.77%
  11. Qwen3.8-Flash-Next45.36%
  12. Claude Opus 544.74%
  13. GPT-5.6 Sol44.33%
  14. GPT-6 Astra43.09%
  15. Grok 4.542.06%
τ³-Banking / %
モデルと実行構成成績出典
Muse Spark 1.3 (max)Meta52.37%原記録
Qwen3.8 MaxAlibaba51.34%原記録
Grok 4.6 (high)SpaceXAI50.72%原記録
GLM-5.3 (max)Z AI50.31%原記録
Qwen3.8 2.4T A95BAlibaba49.07%原記録
Qwen3.8 27B (xhigh)Alibaba48.04%原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI47.22%原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic47.22%原記録
Kimi K3 (max)Kimi45.98%原記録
Gemini 3.8 Flash (medium)Google45.77%原記録
Qwen3.8-Flash-NextAlibaba45.36%原記録
Claude Opus 5 (Adaptive Reasoning, High Effort)Anthropic44.74%原記録
GPT-5.6 Sol (max)OpenAI44.33%原記録
GPT-6 Astra (xhigh)OpenAI43.09%原記録
Grok 4.5 (high)SpaceXAI42.06%原記録
τ²-Bench15 件の成績

基盤モデルの成績Artificial Analysis

対話中の道具使用

%高いほど良い
  1. JT-35B-Flash99.12%
  2. GLM-5.299.12%
  3. Step 3.7 Flash98.54%
  4. Claude Fable 598.54%
  5. DeepSeek V4 Pro 042496.2%
  6. Qwen3.5 397B A17B95.61%
  7. Gemini 3.5 Flash95.61%
  8. Gemini 3.1 Pro Preview95.61%
  9. Qwen3.6 35B A3B95.32%
  10. DeepSeek V4 Flash 042094.44%
  11. MiMo-V2.5-Pro94.15%
  12. Qwen3.6 27B94.15%
  13. Mistral Medium 3.594.15%
  14. Qwen3.5 122B A10B93.57%
  15. MiMo-V2-Flash (Feb 2026)93.27%
τ²-Bench / %
モデルと実行構成成績出典
JT-35B-FlashChina Mobile99.12%原記録
GLM-5.2 (max)Z AI99.12%原記録
Step 3.7 FlashStepFun98.54%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic98.54%原記録
DeepSeek V4 Pro 0424 (DeepSeek V4 Pro (Reasoning, Max Effort))DeepSeek96.2%原記録
Qwen3.5 397B A17B (Reasoning)Alibaba95.61%原記録
Gemini 3.5 Flash (medium)Google95.61%原記録
Gemini 3.1 Pro PreviewGoogle95.61%原記録
Qwen3.6 35B A3B (Reasoning)Alibaba95.32%原記録
DeepSeek V4 Flash 0420 (DeepSeek V4 Flash (Non-reasoning))DeepSeek94.44%原記録
MiMo-V2.5-ProXiaomi94.15%原記録
Qwen3.6 27B (Reasoning)Alibaba94.15%原記録
Mistral Medium 3.5Mistral94.15%原記録
Qwen3.5 122B A10B (Reasoning)Alibaba93.57%原記録
MiMo-V2-Flash (Feb 2026)Xiaomi93.27%原記録
ITBench-AA15 件の成績

基盤モデルの成績Artificial Analysis

運用障害の原因分析

%高いほど良い
  1. GPT-5.6 Sol56.21%
  2. GPT-5.6 Terra51.04%
  3. Kimi K347.69%
  4. GLM-5.242.66%
  5. GPT-5.6 Luna40.32%
  6. DeepSeek V4 Pro 042438.32%
  7. MiMo-V2.5-Pro38.23%
  8. Gemma 4 31B37.29%
  9. Qwen3.5 397B A17B34.09%
  10. Gemini 3.1 Pro Preview30.33%
  11. Step 3.7 Flash30.27%
  12. Claude 4.5 Haiku27.31%
  13. Gemma 4 26B A4B23.63%
  14. gpt-oss-120b5.65%
  15. Nemotron 3 Super 120B A12B1.13%
ITBench-AA / %
モデルと実行構成成績出典
GPT-5.6 Sol (max)OpenAI56.21%原記録
GPT-5.6 Terra (max)OpenAI51.04%原記録
Kimi K3 (max)Kimi47.69%原記録
GLM-5.2 (max)Z AI42.66%原記録
GPT-5.6 Luna (max)OpenAI40.32%原記録
DeepSeek V4 Pro 0424 (DeepSeek V4 Pro (Reasoning, Max Effort))DeepSeek38.32%原記録
MiMo-V2.5-ProXiaomi38.23%原記録
Gemma 4 31B (Reasoning)Google37.29%原記録
Qwen3.5 397B A17B (Reasoning)Alibaba34.09%原記録
Gemini 3.1 Pro PreviewGoogle30.33%原記録
Step 3.7 FlashStepFun30.27%原記録
Claude 4.5 Haiku (Reasoning)Anthropic27.31%原記録
Gemma 4 26B A4B (Reasoning)Google23.63%原記録
gpt-oss-120b (high)OpenAI5.65%原記録
Nemotron 3 Super 120B A12B (Reasoning)NVIDIA1.13%原記録
MCP-Atlas public set / Z.ai report7 件の成績

提供元発表のモデル成績Z.ai (model-card report)

接続したツールでの課題完了。

%高いほど良い
  1. GPT-5.268%
  2. GLM-567.8%
  3. Gemini 3 Pro66.6%
  4. Claude Opus 4.565.2%
  5. Kimi K2.563.8%
  6. DeepSeek V3.262.2%
  7. GLM-4.752%
MCP-Atlas public set / Z.ai report / %
モデルと実行構成成績出典
GPT-5.2 (xhigh)OpenAI68%原記録
GLM-5 release reportZ.ai67.8%原記録
Gemini 3 Pro (GLM-5 release comparison)Google66.6%原記録
Claude Opus 4.5 (GLM-5 release comparison)Anthropic65.2%原記録
Kimi K2.5 (GLM-5 release comparison)Moonshot AI63.8%原記録
DeepSeek V3.2 (GLM-5 release comparison)DeepSeek62.2%原記録
GLM-4.7 (GLM-5 release comparison)Z.ai52%原記録

コードと実行

Terminal-Bench v2.115 件の成績

基盤モデルの成績Artificial Analysis

ターミナル課題の完了

%高いほど良い
  1. Claude Fable 5.191.39%
  2. GPT-6 Astra89.89%
  3. GPT-5.6 Sol89.51%
  4. Claude Opus 589.14%
  5. Grok 4.688.39%
  6. GPT-5.6 Terra88.01%
  7. Gemini 3.8 Flash87.64%
  8. Qwen3.8-Flash-Next86.14%
  9. Muse Spark 1.385.77%
  10. Gemini 3.7 Flash85.77%
  11. Kimi K385.02%
  12. Claude Fable 584.64%
  13. GLM 5.3 Flash84.27%
  14. GLM-5.383.9%
  15. Qwen3.8 2.4T A95B82.02%
Terminal-Bench v2.1 / %
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic91.39%原記録
GPT-6 Astra (high)OpenAI89.89%原記録
GPT-5.6 Sol (xhigh)OpenAI89.51%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic89.14%原記録
Grok 4.6 (high)SpaceXAI88.39%原記録
GPT-5.6 Terra (max)OpenAI88.01%原記録
Gemini 3.8 Flash (high)Google87.64%原記録
Qwen3.8-Flash-NextAlibaba86.14%原記録
Muse Spark 1.3 (max)Meta85.77%原記録
Gemini 3.7 Flash (high)Google85.77%原記録
Kimi K3 (max)Kimi85.02%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic84.64%原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI84.27%原記録
GLM-5.3 (max)Z AI83.9%原記録
Qwen3.8 2.4T A95BAlibaba82.02%原記録
SciCode15 件の成績

基盤モデルの成績Artificial Analysis

科学課題のプログラミング

%高いほど良い
  1. Claude Fable 5.163.08%
  2. Claude Fable 561%
  3. Gemini 3.7 Flash59.84%
  4. Muse Spark 1.359.72%
  5. Kimi K359.49%
  6. GLM-5.359.03%
  7. Gemini 3.1 Pro Preview58.68%
  8. GPT-5.6 Sol57.75%
  9. Muse Spark 1.257.41%
  10. Gemini 3.8 Flash56.6%
  11. GPT-6 Astra56.48%
  12. Grok 4.656.48%
  13. Claude Opus 556.37%
  14. Grok 4.554.98%
  15. GPT-5.6 Terra54.98%
SciCode / %
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic63.08%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic61%原記録
Gemini 3.7 Flash (medium)Google59.84%原記録
Muse Spark 1.3 (xhigh)Meta59.72%原記録
Kimi K3 (max)Kimi59.49%原記録
GLM-5.3 (max)Z AI59.03%原記録
Gemini 3.1 Pro PreviewGoogle58.68%原記録
GPT-5.6 Sol (high)OpenAI57.75%原記録
Muse Spark 1.2 (xhigh)Meta57.41%原記録
Gemini 3.8 Flash (high)Google56.6%原記録
GPT-6 Astra (max)OpenAI56.48%原記録
Grok 4.6 (high)SpaceXAI56.48%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic56.37%原記録
Grok 4.5 (high)SpaceXAI54.98%原記録
GPT-5.6 Terra (max)OpenAI54.98%原記録
LiveCodeBench15 件の成績

基盤モデルの成績Artificial Analysis

プログラミング問題

%高いほど良い
  1. gpt-oss-120b87.83%
  2. ERNIE 5.0 Thinking Preview81.16%
  3. o380.85%
  4. Apriel-v1.6-15B-Thinker80.74%
  5. Qwen3 Next 80B A3B (Reasoning)78.41%
  6. INTELLECT-377.67%
  7. gpt-oss-20b77.67%
  8. K-EXAONE76.83%
  9. Doubao Seed Code76.61%
  10. Magistral Medium 1.275.03%
  11. EXAONE 4.0 32B74.71%
  12. NVIDIA Nemotron 3 Nano 30B A3B74.07%
  13. Llama Nemotron Super 49B v1.573.65%
  14. Nova 2.0 Pro Preview73.02%
  15. Falcon-H1R-7B72.38%
LiveCodeBench / %
モデルと実行構成成績出典
gpt-oss-120b (high)OpenAI87.83%原記録
ERNIE 5.0 Thinking PreviewBaidu81.16%原記録
o3OpenAI80.85%原記録
Apriel-v1.6-15B-ThinkerServiceNow80.74%原記録
Qwen3 Next 80B A3B (Reasoning)Alibaba78.41%原記録
INTELLECT-3Prime Intellect77.67%原記録
gpt-oss-20b (high)OpenAI77.67%原記録
K-EXAONE (Reasoning)LG AI Research76.83%原記録
Doubao Seed CodeByteDance Seed76.61%原記録
Magistral Medium 1.2Mistral75.03%原記録
EXAONE 4.0 32B (Reasoning)LG AI Research74.71%原記録
NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)NVIDIA74.07%原記録
Llama Nemotron Super 49B v1.5 (Reasoning)NVIDIA73.65%原記録
Nova 2.0 Pro Preview (medium)Amazon73.02%原記録
Falcon-H1R-7BTII UAE72.38%原記録
SciCode / MiniMax report7 件の成績

提供元発表のモデル成績MiniMax (internal evaluation)

科学分野のプログラミング。

score高いほど良い
  1. Gemini 3 Pro56
  2. Claude Opus 4.652
  3. GPT-5.252
  4. Claude Opus 4.550
  5. Claude Sonnet 4.545
  6. MiniMax-M2.544.4
  7. MiniMax-M2.141
SciCode / MiniMax report / score
モデルと実行構成成績出典
Gemini 3 Pro (MiniMax internal evaluation)Google56原記録
Claude Opus 4.6 (MiniMax internal evaluation)Anthropic52原記録
GPT-5.2 (Thinking; MiniMax internal evaluation)OpenAI52原記録
Claude Opus 4.5 (MiniMax internal evaluation)Anthropic50原記録
Claude Sonnet 4.5 (MiniMax internal evaluation)Anthropic45原記録
MiniMax-M2.5 (MiniMax internal evaluation)MiniMax44.4原記録
MiniMax-M2.1 (MiniMax internal evaluation)MiniMax41原記録
Terminal Bench 2.1 / DeepSeek GA report8 件の成績

提供元発表のモデル成績DeepSeek (GA model-card report)

端末上で行うコーディングとツール利用の課題。

score高いほど良い
  1. Kimi K388.3
  2. Fable-5 (w/ fallback)88
  3. DeepSeek-V4-Pro-081387.9
  4. Opus-4.885
  5. DeepSeek-V4-Flash-073182.7
  6. GLM-5.281
  7. DeepSeek-V4-Pro (Preview)72.1
  8. DeepSeek-V4-Flash (Preview)61.8
Terminal Bench 2.1 / DeepSeek GA report / score
モデルと実行構成成績出典
Kimi K3 (Code-agent evaluation; harness and reasoning effort not disclosed for this model)Moonshot AI88.3原記録
Fable-5 (w/ fallback) (Code-agent evaluation; harness and reasoning effort not disclosed for this model; with fallback, details unspecified)Anthropic88原記録
DeepSeek-V4-Pro-0813 (DeepSeek Harness minimal; max reasoning effort; temperature=1.0; top_p=0.95)DeepSeek87.9原記録
Opus-4.8 (Code-agent evaluation; harness and reasoning effort not disclosed for this model)Anthropic85原記録
DeepSeek-V4-Flash-0731 (Code-agent evaluation; harness and reasoning effort not disclosed for this model)DeepSeek82.7原記録
GLM-5.2 (Code-agent evaluation; harness and reasoning effort not disclosed for this model)Z.ai81原記録
DeepSeek-V4-Pro (Preview) (Code-agent evaluation; harness and reasoning effort not disclosed for this model)DeepSeek72.1原記録
DeepSeek-V4-Flash (Preview) (Code-agent evaluation; harness and reasoning effort not disclosed for this model)DeepSeek61.8原記録
SWE-bench Pro refined set / Qwen3.8-Max report5 件の成績

提供元発表のモデル成績Qwen (model-card report)

Qwenが修正したSWE-bench Pro評価セットでのコード修正。

score高いほど良い
  1. Fable 580
  2. Opus 4.869.2
  3. Qwen3.8-Max67.7
  4. GPT 5.6 Sol (max)64.6
  5. Qwen3.7-Max60.6
SWE-bench Pro refined set / Qwen3.8-Max report / score
モデルと実行構成成績出典
Fable 5 (Qwen-refined set; Claude Code; temperature=1.0; top_p=0.95; 256K context; may involve fallback; reasoning effort not disclosed)Anthropic80原記録
Opus 4.8 (Qwen-refined set; Claude Code; temperature=1.0; top_p=0.95; 256K context; reasoning effort not disclosed)Anthropic69.2原記録
Qwen3.8-Max service model; Qwen-refined set; Claude Code; temperature=1.0; top_p=0.95; 256K context; reasoning effort not disclosedQwen67.7原記録
GPT 5.6 Sol (max) (Qwen-refined set; Claude Code; max (table header); temperature=1.0; top_p=0.95; 256K context)OpenAI64.6原記録
Qwen3.7-Max (Qwen-refined set; Claude Code; temperature=1.0; top_p=0.95; 256K context; reasoning effort not disclosed)Qwen60.6原記録
SWE-bench Verified / Google G31 (2026-02-19)5 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

モデルとコーディング実行環境によるリポジトリ課題の解決率です。

%高いほど良い
  1. Claude Opus 4.680.8%
  2. Gemini 3.1 Pro80.6%
  3. GPT-5.280%
  4. Claude Sonnet 4.679.6%
  5. Gemini 3 Pro76.2%
SWE-bench Verified / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
Claude Opus 4.6 (Thinking (Max); Single attempt; provider-specific scaffolding)Anthropic80.8%原記録
Gemini 3.1 Pro (Thinking (High); Single attempt; provider-specific scaffolding)Google80.6%原記録
GPT-5.2 (Thinking (xhigh); Single attempt; provider-specific scaffolding)OpenAI80%原記録
Claude Sonnet 4.6 (Thinking (Max); Single attempt; provider-specific scaffolding)Anthropic79.6%原記録
Gemini 3 Pro (Thinking (High); Single attempt; provider-specific scaffolding)Google76.2%原記録
SWE-bench Pro (Public) / Google G31 (2026-02-19)4 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

SWE-bench Pro公開版のソフトウェア開発課題です。

%高いほど良い
  1. GPT-5.3-Codex56.8%
  2. GPT-5.255.6%
  3. Gemini 3.1 Pro54.2%
  4. Gemini 3 Pro43.3%
SWE-bench Pro (Public) / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
GPT-5.3-Codex (Thinking (xhigh); Public split; single attempt; provider-specific scaffolding)OpenAI56.8%原記録
GPT-5.2 (Thinking (xhigh); Public split; single attempt; provider-specific scaffolding)OpenAI55.6%原記録
Gemini 3.1 Pro (Thinking (High); Public split; single attempt; provider-specific scaffolding)Google54.2%原記録
Gemini 3 Pro (Thinking (High); Public split; single attempt; provider-specific scaffolding)Google43.3%原記録

モデル推論

Intelligence Index v4.215 件の成績

基盤モデルの成績Artificial Analysis

10評価の総合指標

score高いほど良い
  1. Claude Fable 5.156.76
  2. GPT-6 Astra54.66
  3. Claude Opus 554.05
  4. Claude Fable 553.19
  5. Muse Spark 1.352.95
  6. GPT-5.6 Sol51.26
  7. Grok 4.650.58
  8. Kimi K350.23
  9. GLM-5.348.58
  10. Gemini 3.8 Flash47.07
  11. Qwen3.8 Max46.91
  12. Muse Spark 1.246.84
  13. GPT-5.6 Terra46.77
  14. Qwen3.8 2.4T A95B46.74
  15. GLM 5.3 Flash46.22
Intelligence Index v4.2 / score
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic56.76原記録
GPT-6 Astra (max)OpenAI54.66原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic54.05原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic53.19原記録
Muse Spark 1.3 (max)Meta52.95原記録
GPT-5.6 Sol (max)OpenAI51.26原記録
Grok 4.6 (high)SpaceXAI50.58原記録
Kimi K3 (max)Kimi50.23原記録
GLM-5.3 (max)Z AI48.58原記録
Gemini 3.8 Flash (high)Google47.07原記録
Qwen3.8 MaxAlibaba46.91原記録
Muse Spark 1.2 (xhigh)Meta46.84原記録
GPT-5.6 Terra (max)OpenAI46.77原記録
Qwen3.8 2.4T A95BAlibaba46.74原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI46.22原記録
Humanity's Last Exam15 件の成績

基盤モデルの成績Artificial Analysis

専門知識と難しい推論

%高いほど良い
  1. Claude Fable 5.159.13%
  2. Claude Fable 555.47%
  3. Claude Opus 554.87%
  4. GPT-6 Astra54.68%
  5. GPT-5.6 Sol49.49%
  6. Muse Spark 1.349.07%
  7. Gemini 3.7 Flash47.87%
  8. Gemini 3.8 Flash47.82%
  9. Gemini 3.1 Pro Preview47.03%
  10. Kimi K346.9%
  11. Muse Spark 1.245.46%
  12. Grok 4.644.07%
  13. Qwen3.8 Max43.05%
  14. GPT-5.6 Terra42.91%
  15. Grok 4.542.68%
Humanity's Last Exam / %
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic59.13%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic55.47%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic54.87%原記録
GPT-6 Astra (max)OpenAI54.68%原記録
GPT-5.6 Sol (max)OpenAI49.49%原記録
Muse Spark 1.3 (max)Meta49.07%原記録
Gemini 3.7 Flash (high)Google47.87%原記録
Gemini 3.8 Flash (high)Google47.82%原記録
Gemini 3.1 Pro PreviewGoogle47.03%原記録
Kimi K3 (max)Kimi46.9%原記録
Muse Spark 1.2 (xhigh)Meta45.46%原記録
Grok 4.6 (xhigh)SpaceXAI44.07%原記録
Qwen3.8 MaxAlibaba43.05%原記録
GPT-5.6 Terra (max)OpenAI42.91%原記録
Grok 4.5 (high)SpaceXAI42.68%原記録
GPQA Diamond15 件の成績

基盤モデルの成績Artificial Analysis

大学院水準の科学推論

%高いほど良い
  1. GPT-6 Astra96.26%
  2. Gemini 3.8 Flash95.25%
  3. Grok 4.694.95%
  4. Gemini 3.7 Flash94.55%
  5. Gemini 3.1 Pro Preview94.14%
  6. Muse Spark 1.394.14%
  7. GPT-5.6 Sol94.14%
  8. Claude Opus 593.74%
  9. Claude Fable 5.193.74%
  10. Qwen3.8 2.4T A95B93.54%
  11. Kimi K393.54%
  12. Grok 4.593.13%
  13. MiniMax-M392.93%
  14. Gemini 3.6 Flash92.83%
  15. DeepSeek V4 Pro 081392.83%
GPQA Diamond / %
モデルと実行構成成績出典
GPT-6 Astra (xhigh)OpenAI96.26%原記録
Gemini 3.8 Flash (high)Google95.25%原記録
Grok 4.6 (high)SpaceXAI94.95%原記録
Gemini 3.7 Flash (high)Google94.55%原記録
Gemini 3.1 Pro PreviewGoogle94.14%原記録
Muse Spark 1.3 (xhigh)Meta94.14%原記録
GPT-5.6 Sol (max)OpenAI94.14%原記録
Claude Opus 5 (Adaptive Reasoning, Xhigh Effort)Anthropic93.74%原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic93.74%原記録
Qwen3.8 2.4T A95BAlibaba93.54%原記録
Kimi K3 (max)Kimi93.54%原記録
Grok 4.5 (high)SpaceXAI93.13%原記録
MiniMax-M3MiniMax92.93%原記録
Gemini 3.6 Flash (high)Google92.83%原記録
DeepSeek V4 Pro 0813 (Reasoning, Max Effort)DeepSeek92.83%原記録
CritPt15 件の成績

基盤モデルの成績Artificial Analysis

研究水準の物理問題

%高いほど良い
  1. GPT-5.6 Sol32.29%
  2. GPT-6 Astra31.71%
  3. Claude Fable 5.131.14%
  4. GPT-5.5 Pro30.57%
  5. GPT-5.6 Terra30%
  6. Claude Opus 529.14%
  7. Claude Fable 528.57%
  8. Muse Spark 1.326%
  9. Gemini 3 Deep Think25.71%
  10. Kimi K323.43%
  11. GLM-5.220.86%
  12. GPT-5.6 Luna20.57%
  13. Qwen3.8 Max20%
  14. Qwen3.8 2.4T A95B20%
  15. Grok 4.619.71%
CritPt / %
モデルと実行構成成績出典
GPT-5.6 Sol (max)OpenAI32.29%原記録
GPT-6 Astra (max)OpenAI31.71%原記録
Claude Fable 5.1 (Adaptive Reasoning, Xhigh Effort, Default Fallback)Anthropic31.14%原記録
GPT-5.5 Pro (xhigh)OpenAI30.57%原記録
GPT-5.6 Terra (max)OpenAI30%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic29.14%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic28.57%原記録
Muse Spark 1.3 (xhigh)Meta26%原記録
Gemini 3 Deep ThinkGoogle25.71%原記録
Kimi K3 (max)Kimi23.43%原記録
GLM-5.2 (max)Z AI20.86%原記録
GPT-5.6 Luna (max)OpenAI20.57%原記録
Qwen3.8 MaxAlibaba20%原記録
Qwen3.8 2.4T A95BAlibaba20%原記録
Grok 4.6 (xhigh)SpaceXAI19.71%原記録
AIME 202515 件の成績

基盤モデルの成績Artificial Analysis

競技数学

%高いほど良い
  1. Nova 2.0 Lite94.33%
  2. gpt-oss-120b93.44%
  3. NVIDIA Nemotron 3 Nano 30B A3B91%
  4. K-EXAONE90.33%
  5. Nova 2.0 Omni89.67%
  6. gpt-oss-20b89.33%
  7. Nova 2.0 Pro Preview89%
  8. o388.33%
  9. INTELLECT-388%
  10. Apriel-v1.6-15B-Thinker88%
  11. ERNIE 5.0 Thinking Preview85%
  12. Qwen3 Next 80B A3B (Reasoning)84.33%
  13. Ring-flash-2.083.67%
  14. Claude 4.5 Haiku83.67%
  15. Magistral Medium 1.282%
AIME 2025 / %
モデルと実行構成成績出典
Nova 2.0 Lite (high)Amazon94.33%原記録
gpt-oss-120b (high)OpenAI93.44%原記録
NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)NVIDIA91%原記録
K-EXAONE (Reasoning)LG AI Research90.33%原記録
Nova 2.0 Omni (medium)Amazon89.67%原記録
gpt-oss-20b (high)OpenAI89.33%原記録
Nova 2.0 Pro Preview (medium)Amazon89%原記録
o3OpenAI88.33%原記録
INTELLECT-3Prime Intellect88%原記録
Apriel-v1.6-15B-ThinkerServiceNow88%原記録
ERNIE 5.0 Thinking PreviewBaidu85%原記録
Qwen3 Next 80B A3B (Reasoning)Alibaba84.33%原記録
Ring-flash-2.0InclusionAI83.67%原記録
Claude 4.5 Haiku (Reasoning)Anthropic83.67%原記録
Magistral Medium 1.2Mistral82%原記録
IFBench15 件の成績

基盤モデルの成績Artificial Analysis

複雑な指示への対応

%高いほど良い
  1. Grok 4.383.33%
  2. MiniMax-M382.86%
  3. Nemotron 3 Ultra 550B A55B81.36%
  4. Nemotron Cascade 2 30B A3B80.41%
  5. MiMo-V2.5-Pro79.86%
  6. Nova 2.0 Pro Preview79.59%
  7. Qwen3.5 397B A17B78.78%
  8. Qwen3.7 Plus77.96%
  9. Gemini 3.1 Pro Preview77.14%
  10. DeepSeek V4 Pro 042476.46%
  11. Qwen3.5 122B A10B75.71%
  12. Gemma 4 31B75.58%
  13. GPT-5.3 Codex75.37%
  14. Gemini 3.5 Flash74.56%
  15. Command A+73.95%
IFBench / %
モデルと実行構成成績出典
Grok 4.3 (medium)SpaceXAI83.33%原記録
MiniMax-M3MiniMax82.86%原記録
Nemotron 3 Ultra 550B A55B (Reasoning)NVIDIA81.36%原記録
Nemotron Cascade 2 30B A3BNVIDIA80.41%原記録
MiMo-V2.5-ProXiaomi79.86%原記録
Nova 2.0 Pro Preview (low)Amazon79.59%原記録
Qwen3.5 397B A17B (Reasoning)Alibaba78.78%原記録
Qwen3.7 PlusAlibaba77.96%原記録
Gemini 3.1 Pro PreviewGoogle77.14%原記録
DeepSeek V4 Pro 0424 (DeepSeek V4 Pro (Reasoning, Max Effort))DeepSeek76.46%原記録
Qwen3.5 122B A10B (Reasoning)Alibaba75.71%原記録
Gemma 4 31B (Reasoning)Google75.58%原記録
GPT-5.3 Codex (xhigh)OpenAI75.37%原記録
Gemini 3.5 Flash (medium)Google74.56%原記録
Command A+Cohere73.95%原記録
LongBench v2 / Moonshot report5 件の成績

提供元発表のモデル成績Moonshot AI (model-card report)

長文入力の読解と推論。

score高いほど良い
  1. Gemini 3 Pro68.2
  2. Claude Opus 4.564.4
  3. Kimi K2.561
  4. DeepSeek V3.259.8
  5. GPT-5.254.5
LongBench v2 / Moonshot report / score
モデルと実行構成成績出典
Gemini 3 Pro (High Thinking Level; publisher rerun (*))Google68.2原記録
Claude Opus 4.5 (Extended Thinking; publisher rerun (*))Anthropic64.4原記録
Kimi K2.5 (Thinking)Moonshot AI61原記録
DeepSeek V3.2 (Thinking; publisher rerun (*))DeepSeek59.8原記録
GPT-5.2 (xhigh; publisher rerun (*))OpenAI54.5原記録
AIME 2026 I / Z.ai report6 件の成績

提供元発表のモデル成績Z.ai (model-card report)

2026年AIMEの第1試験。年間全体とは別です。

score高いほど良い
  1. Claude Opus 4.593.3
  2. GLM-4.792.9
  3. GLM-592.7
  4. DeepSeek V3.292.7
  5. Kimi K2.592.5
  6. Gemini 3 Pro90.6
AIME 2026 I / Z.ai report / score
モデルと実行構成成績出典
Claude Opus 4.5 (GLM-5 release comparison)Anthropic93.3原記録
GLM-4.7 (GLM-5 release comparison)Z.ai92.9原記録
GLM-5 release reportZ.ai92.7原記録
DeepSeek V3.2 (GLM-5 release comparison)DeepSeek92.7原記録
Kimi K2.5 (GLM-5 release comparison)Moonshot AI92.5原記録
Gemini 3 Pro (GLM-5 release comparison)Google90.6原記録
HMMT November 2025 / Z.ai report7 件の成績

提供元発表のモデル成績Z.ai (model-card report)

2025年11月の数学競技問題。

score高いほど良い
  1. GPT-5.297.1
  2. GLM-596.9
  3. GLM-4.793.5
  4. Gemini 3 Pro93
  5. Claude Opus 4.591.7
  6. Kimi K2.591.1
  7. DeepSeek V3.290.2
HMMT November 2025 / Z.ai report / score
モデルと実行構成成績出典
GPT-5.2 (xhigh)OpenAI97.1原記録
GLM-5 release reportZ.ai96.9原記録
GLM-4.7 (GLM-5 release comparison)Z.ai93.5原記録
Gemini 3 Pro (GLM-5 release comparison)Google93原記録
Claude Opus 4.5 (GLM-5 release comparison)Anthropic91.7原記録
Kimi K2.5 (GLM-5 release comparison)Moonshot AI91.1原記録
DeepSeek V3.2 (GLM-5 release comparison)DeepSeek90.2原記録
AA-LCR / MiniMax report7 件の成績

提供元発表のモデル成績MiniMax (internal evaluation)

長い入力に基づく推論。

score高いほど良い
  1. Claude Opus 4.574
  2. GPT-5.273
  3. Claude Opus 4.671
  4. Gemini 3 Pro71
  5. MiniMax-M2.569.5
  6. Claude Sonnet 4.566
  7. MiniMax-M2.162
AA-LCR / MiniMax report / score
モデルと実行構成成績出典
Claude Opus 4.5 (MiniMax internal evaluation)Anthropic74原記録
GPT-5.2 (Thinking; MiniMax internal evaluation)OpenAI73原記録
Claude Opus 4.6 (MiniMax internal evaluation)Anthropic71原記録
Gemini 3 Pro (MiniMax internal evaluation)Google71原記録
MiniMax-M2.5 (MiniMax internal evaluation)MiniMax69.5原記録
Claude Sonnet 4.5 (MiniMax internal evaluation)Anthropic66原記録
MiniMax-M2.1 (MiniMax internal evaluation)MiniMax62原記録
HLE no tools / DeepSeek GA report8 件の成績

提供元発表のモデル成績DeepSeek (GA model-card report)

ツールを使わない分野横断の推論評価。

score高いほど良い
  1. Fable-5 (w/ fallback)53.3
  2. Opus-4.849.8
  3. Kimi K343.5
  4. DeepSeek-V4-Pro-081342.7
  5. GLM-5.240.5
  6. DeepSeek-V4-Flash-073137.8
  7. DeepSeek-V4-Pro (Preview)37.7
  8. DeepSeek-V4-Flash (Preview)34.8
HLE no tools / DeepSeek GA report / score
モデルと実行構成成績出典
Fable-5 (w/ fallback) (No tools; HLE reasoning effort and token budget not disclosed; with fallback, details unspecified)Anthropic53.3原記録
Opus-4.8 (No tools; HLE reasoning effort and token budget not disclosed)Anthropic49.8原記録
Kimi K3 (No tools; HLE reasoning effort and token budget not disclosed)Moonshot AI43.5原記録
DeepSeek-V4-Pro-0813 (No tools; HLE reasoning effort and token budget not disclosed)DeepSeek42.7原記録
GLM-5.2 (No tools; HLE reasoning effort and token budget not disclosed)Z.ai40.5原記録
DeepSeek-V4-Flash-0731 (No tools; HLE reasoning effort and token budget not disclosed)DeepSeek37.8原記録
DeepSeek-V4-Pro (Preview) (No tools; HLE reasoning effort and token budget not disclosed)DeepSeek37.7原記録
DeepSeek-V4-Flash (Preview) (No tools; HLE reasoning effort and token budget not disclosed)DeepSeek34.8原記録
HLE with tools / DeepSeek GA report8 件の成績

提供元発表のモデル成績DeepSeek (GA model-card report)

ツールを使う分野横断の推論評価。

score高いほど良い
  1. Fable-5 (w/ fallback)63
  2. DeepSeek-V4-Pro-081360
  3. Opus-4.857.9
  4. Kimi K356
  5. GLM-5.254.7
  6. DeepSeek-V4-Flash-073151.5
  7. DeepSeek-V4-Pro (Preview)48.2
  8. DeepSeek-V4-Flash (Preview)45.1
HLE with tools / DeepSeek GA report / score
モデルと実行構成成績出典
Fable-5 (w/ fallback) (With tools; specific tools, HLE reasoning effort and token budget not disclosed; with fallback, details unspecified)Anthropic63原記録
DeepSeek-V4-Pro-0813 (With tools; specific tools, HLE reasoning effort and token budget not disclosed)DeepSeek60原記録
Opus-4.8 (With tools; specific tools, HLE reasoning effort and token budget not disclosed)Anthropic57.9原記録
Kimi K3 (With tools; specific tools, HLE reasoning effort and token budget not disclosed)Moonshot AI56原記録
GLM-5.2 (With tools; specific tools, HLE reasoning effort and token budget not disclosed)Z.ai54.7原記録
DeepSeek-V4-Flash-0731 (With tools; specific tools, HLE reasoning effort and token budget not disclosed)DeepSeek51.5原記録
DeepSeek-V4-Pro (Preview) (With tools; specific tools, HLE reasoning effort and token budget not disclosed)DeepSeek48.2原記録
DeepSeek-V4-Flash (Preview) (With tools; specific tools, HLE reasoning effort and token budget not disclosed)DeepSeek45.1原記録
HMMT February 2026 / DeepSeek Preview report6 件の成績

提供元発表のモデル成績DeepSeek (Preview technical report)

1つの回答の成功率であるPass@1で測る数学競技問題。

%高いほど良い
  1. GPT-5.4 xHigh97.7%
  2. Opus-4.6 Max96.2%
  3. DS-V4-Pro Max95.2%
  4. Gemini-3.1-Pro High94.7%
  5. K2.6 Thinking92.7%
  6. GLM-5.1 Thinking89.4%
HMMT February 2026 / DeepSeek Preview report / %
モデルと実行構成成績出典
GPT-5.4 xHigh (xHigh; DeepSeek Preview report comparison)OpenAI97.7%原記録
Opus-4.6 Max (Max; DeepSeek Preview report comparison)Anthropic96.2%原記録
DS-V4-Pro Max (Preview; Max; temperature=1.0; 384K-token context; distinct rigorous-proof math prompt)DeepSeek95.2%原記録
Gemini-3.1-Pro High (High; DeepSeek Preview report comparison)Google94.7%原記録
K2.6 Thinking (Thinking; DeepSeek Preview report comparison)Moonshot AI92.7%原記録
GLM-5.1 Thinking (Thinking; DeepSeek Preview report comparison)Z.ai89.4%原記録
LongBench v2 / Qwen3.8-Max report4 件の成績

提供元発表のモデル成績Qwen (model-card report)

長い文書や入力の読解と推論。

score高いほど良い
  1. Opus 4.869.1
  2. GPT 5.6 Sol (max)67.1
  3. Qwen3.8-Max66.3
  4. Qwen3.7-Max65.3
LongBench v2 / Qwen3.8-Max report / score
モデルと実行構成成績出典
Opus 4.8 (LongBench v2; tools, reasoning effort and token budget not disclosed)Anthropic69.1原記録
GPT 5.6 Sol (max) (max (table header); LongBench v2 tools and token budget not disclosed)OpenAI67.1原記録
Qwen3.8-Max service model; LongBench v2 tools, reasoning effort and token budget not disclosedQwen66.3原記録
Qwen3.7-Max (LongBench v2; tools, reasoning effort and token budget not disclosed)Qwen65.3原記録
GPQA Diamond / Google G31 (2026-02-19)5 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

ツールなしで科学知識と推論を評価します。

%高いほど良い
  1. Gemini 3.1 Pro94.3%
  2. GPT-5.292.4%
  3. Gemini 3 Pro91.9%
  4. Claude Opus 4.691.3%
  5. Claude Sonnet 4.689.9%
GPQA Diamond / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
Gemini 3.1 Pro (Thinking (High); No tools)Google94.3%原記録
GPT-5.2 (Thinking (xhigh); No tools)OpenAI92.4%原記録
Gemini 3 Pro (Thinking (High); No tools)Google91.9%原記録
Claude Opus 4.6 (Thinking (Max); No tools)Anthropic91.3%原記録
Claude Sonnet 4.6 (Thinking (Max); No tools)Anthropic89.9%原記録
HLE, no tools / Google G31 (2026-02-19)5 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

HLEのテキストとマルチモーダル全問題をツールなしで解きます。

%高いほど良い
  1. Gemini 3.1 Pro44.4%
  2. Claude Opus 4.640%
  3. Gemini 3 Pro37.5%
  4. GPT-5.234.5%
  5. Claude Sonnet 4.633.2%
HLE, no tools / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
Gemini 3.1 Pro (Thinking (High); No tools)Google44.4%原記録
Claude Opus 4.6 (Thinking (Max); No tools)Anthropic40%原記録
Gemini 3 Pro (Thinking (High); No tools)Google37.5%原記録
GPT-5.2 (Thinking (xhigh); No tools)OpenAI34.5%原記録
Claude Sonnet 4.6 (Thinking (Max); No tools)Anthropic33.2%原記録
HLE, search and code / Google G31 (2026-02-19)5 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

検索とコード実行を使うHLE推論評価です。

%高いほど良い
  1. Claude Opus 4.653.1%
  2. Gemini 3.1 Pro51.4%
  3. Claude Sonnet 4.649%
  4. Gemini 3 Pro45.8%
  5. GPT-5.245.5%
HLE, search and code / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
Claude Opus 4.6 (Thinking (Max); Search and code; provider-specific tools)Anthropic53.1%原記録
Gemini 3.1 Pro (Thinking (High); Search and code; provider-specific tools)Google51.4%原記録
Claude Sonnet 4.6 (Thinking (Max); Search and code; provider-specific tools)Anthropic49%原記録
Gemini 3 Pro (Thinking (High); Search and code; provider-specific tools)Google45.8%原記録
GPT-5.2 (Thinking (xhigh); Search and code; provider-specific tools)OpenAI45.5%原記録
ARC-AGI-2 / Google G31 (2026-02-19)5 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

格子の例から未知の規則を推論します。

%高いほど良い
  1. Gemini 3.1 Pro77.1%
  2. Claude Opus 4.668.8%
  3. Claude Sonnet 4.658.3%
  4. GPT-5.252.9%
  5. Gemini 3 Pro31.1%
ARC-AGI-2 / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
Gemini 3.1 Pro (Thinking (High); ARC Prize Verified; semi-private set)Google77.1%原記録
Claude Opus 4.6 (Thinking (Max); ARC Prize Verified; semi-private set)Anthropic68.8%原記録
Claude Sonnet 4.6 (Thinking (Max); ARC Prize Verified; semi-private set)Anthropic58.3%原記録
GPT-5.2 (Thinking (xhigh); ARC Prize Verified; semi-private set)OpenAI52.9%原記録
Gemini 3 Pro (Thinking (High); ARC Prize Verified; semi-private set)Google31.1%原記録
AIME 2025, no tools / Google G3F (2025-12-17)7 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3 Flash release comparison)

2025年AIME数学競技の問題を解きます。

%高いほど良い
  1. GPT-5.2100%
  2. Gemini 3 Flash95.2%
  3. Gemini 3 Pro95%
  4. Grok 4.1 Fast91.9%
  5. Gemini 2.5 Pro88%
  6. Claude Sonnet 4.587%
  7. Gemini 2.5 Flash72%
AIME 2025, no tools / Google G3F (2025-12-17) / %
モデルと実行構成成績出典
GPT-5.2 (Extra high; No tools)OpenAI100%原記録
Gemini 3 Flash (Thinking; exact level unspecified; No tools)Google95.2%原記録
Gemini 3 Pro (Thinking; exact level unspecified; No tools)Google95%原記録
Grok 4.1 Fast (Reasoning; No tools)xAI91.9%原記録
Gemini 2.5 Pro (Thinking; exact level unspecified; No tools)Google88%原記録
Claude Sonnet 4.5 (Thinking; high preferred, otherwise best available reported setting; No tools)Anthropic87%原記録
Gemini 2.5 Flash (Thinking; exact level unspecified; No tools)Google72%原記録
AIME 2025, code execution / Google G3F (2025-12-17)4 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3 Flash release comparison)

2025年AIME数学競技の問題を解きます。

%高いほど良い
  1. Gemini 3 Pro100%
  2. Claude Sonnet 4.5100%
  3. Gemini 3 Flash99.7%
  4. Gemini 2.5 Flash75.7%
AIME 2025, code execution / Google G3F (2025-12-17) / %
モデルと実行構成成績出典
Gemini 3 Pro (Thinking; exact level unspecified; Code execution)Google100%原記録
Claude Sonnet 4.5 (Thinking; high preferred, otherwise best available reported setting; Code execution)Anthropic100%原記録
Gemini 3 Flash (Thinking; exact level unspecified; Code execution)Google99.7%原記録
Gemini 2.5 Flash (Thinking; exact level unspecified; Code execution)Google75.7%原記録
HLE, no tools / Anthropic A51 (2026-09-01)3 件の成績

提供元発表のモデル成績Anthropic (Fable 5.1 release comparison)

元のHLEマルチモーダル2,500問をツールなしで解きます。

%高いほど良い
  1. Claude Fable 5.160.9%
  2. Claude Fable 557.8%
  3. Claude Opus 556.6%
HLE, no tools / Anthropic A51 (2026-09-01) / %
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive thinking (auto); max effort; default sampling; five-trial mean; total token cap 1M; no compaction; production safeguards enabled; No tools)Anthropic60.9%原記録
Claude Fable 5 (Detailed reasoning setting and budget unverified; production safeguards enabled; No tools)Anthropic57.8%原記録
Claude Opus 5 (Detailed reasoning setting and budget unverified; No tools)Anthropic56.6%原記録
HLE, tools / Anthropic A51 (2026-09-01)3 件の成績

提供元発表のモデル成績Anthropic (Fable 5.1 release comparison)

検索、ウェブ取得、コードツールを使う元のHLE推論評価です。

%高いほど良い
  1. Claude Fable 5.165%
  2. Claude Fable 563.8%
  3. Claude Opus 563.6%
HLE, tools / Anthropic A51 (2026-09-01) / %
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive thinking (auto); max effort; default sampling; five-trial mean; total token cap 1M; no compaction; production safeguards enabled; Search; restricted fetch; programmatic tool calling; code execution)Anthropic65%原記録
Claude Fable 5 (Detailed reasoning setting and budget unverified; production safeguards enabled; Search; restricted fetch; programmatic tool calling; code execution)Anthropic63.8%原記録
Claude Opus 5 (Detailed reasoning setting and budget unverified; Search; restricted fetch; programmatic tool calling; code execution)Anthropic63.6%原記録
ARC-AGI-2 / Anthropic A51 (2026-09-01)4 件の成績

提供元発表のモデル成績Anthropic (Fable 5.1 release comparison)

9月発表比較表にある未知の格子規則の推論評価です。

%高いほど良い
  1. GPT-5.6 Sol92.5%
  2. Claude Opus 590.42%
  3. Claude Fable 5.1 / Claude Mythos 5.190%
  4. Claude Fable 5 / Claude Mythos 589.2%
ARC-AGI-2 / Anthropic A51 (2026-09-01) / %
モデルと実行構成成績出典
GPT-5.6 Sol (Exact reasoning setting and token budget unverified)OpenAI92.5%原記録
Claude Opus 5 (Exact reasoning setting and token budget unverified)Anthropic90.42%原記録
Claude Fable 5.1 / Claude Mythos 5.1 (Fable 5.1: max effort; semi-private set; ARC Prize Verified)Anthropic90%原記録
Claude Fable 5 / Claude Mythos 5 (Exact reasoning setting and token budget unverified)Anthropic89.2%原記録
HLE-Verified / Google G38 (2026-09-02)6 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.8 Flash release comparison)

検証と修正を行った1,811問の専門推論評価です。

%高いほど良い
  1. Gemini 3.8 Flash54.9%
  2. GPT-5.6 Sol54.5%
  3. Claude Opus 554.4%
  4. Gemini 3.7 Flash53.6%
  5. GPT-5.6 Terra51.1%
  6. Claude Sonnet 531%
HLE-Verified / Google G38 (2026-09-02) / %
モデルと実行構成成績出典
Gemini 3.8 Flash (Default sampling; exact thinking level unspecified; HLE-Verified; 1811 items; tool use unspecified)Google54.9%原記録
GPT-5.6 Sol (Exact reasoning level unspecified; HLE-Verified; 1811 items; tool use unspecified)OpenAI54.5%原記録
Claude Opus 5 (Exact thinking level unspecified; HLE-Verified; 1811 items; tool use unspecified)Anthropic54.4%原記録
Gemini 3.7 Flash (Exact thinking level unspecified; HLE-Verified; 1811 items; tool use unspecified)Google53.6%原記録
GPT-5.6 Terra (Maximum reasoning preferred; otherwise best available reported setting; HLE-Verified; 1811 items; tool use unspecified)OpenAI51.1%原記録
Claude Sonnet 5 (Maximum thinking preferred; otherwise best available reported setting; HLE-Verified; 1811 items; tool use unspecified)Anthropic31%原記録

文書と文脈理解

AA-LCR v1.115 件の成績

基盤モデルの成績Artificial Analysis

長い文脈の推論

%高いほど良い
  1. Kimi K388.67%
  2. Claude Fable 5.185.33%
  3. Muse Spark 1.384.33%
  4. Gemini 3.8 Flash84%
  5. GPT-5.6 Sol84%
  6. GPT-5.6 Luna83.67%
  7. Muse Glimmer83.33%
  8. GPT-5.3 Codex83.33%
  9. MiniMax-M383%
  10. GPT-5.6 Terra83%
  11. Gemini 3.7 Flash83%
  12. Agnes 2.5 Pro Beta83%
  13. Claude Fable 582.33%
  14. Claude Sonnet 582%
  15. Qwen3.8 27B82%
AA-LCR v1.1 / %
モデルと実行構成成績出典
Kimi K3 (max)Kimi88.67%原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic85.33%原記録
Muse Spark 1.3 (max)Meta84.33%原記録
Gemini 3.8 Flash (medium)Google84%原記録
GPT-5.6 Sol (max)OpenAI84%原記録
GPT-5.6 Luna (max)OpenAI83.67%原記録
Muse Glimmer (high)Meta83.33%原記録
GPT-5.3 Codex (xhigh)OpenAI83.33%原記録
MiniMax-M3MiniMax83%原記録
GPT-5.6 Terra (max)OpenAI83%原記録
Gemini 3.7 Flash (medium)Google83%原記録
Agnes 2.5 Pro BetaSapiens AI83%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic82.33%原記録
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)Anthropic82%原記録
Qwen3.8 27B (xhigh)Alibaba82%原記録
MMMU-Pro15 件の成績

基盤モデルの成績Artificial Analysis

視覚推論

%高いほど良い
  1. GPT-6 Astra86.88%
  2. Gemini 3.8 Flash85.61%
  3. Gemini 3.7 Flash85.49%
  4. Claude Opus 584.74%
  5. Gemini 3.5 Flash83.87%
  6. GPT-5.6 Sol83.41%
  7. Gemini 3.6 Flash83.24%
  8. Gemini 3.1 Pro Preview82.43%
  9. Qwen3.8 Max82.31%
  10. Muse Spark 1.382.02%
  11. GPT-5.6 Terra80.69%
  12. Kimi K380.52%
  13. Qwen3.7 Plus80.46%
  14. Grok 4.580.4%
  15. Qwen3.8-Flash-Next79.77%
MMMU-Pro / %
モデルと実行構成成績出典
GPT-6 Astra (max)OpenAI86.88%原記録
Gemini 3.8 Flash (high)Google85.61%原記録
Gemini 3.7 Flash (high)Google85.49%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic84.74%原記録
Gemini 3.5 Flash (medium)Google83.87%原記録
GPT-5.6 Sol (max)OpenAI83.41%原記録
Gemini 3.6 Flash (high)Google83.24%原記録
Gemini 3.1 Pro PreviewGoogle82.43%原記録
Qwen3.8 MaxAlibaba82.31%原記録
Muse Spark 1.3 (xhigh)Meta82.02%原記録
GPT-5.6 Terra (max)OpenAI80.69%原記録
Kimi K3 (max)Kimi80.52%原記録
Qwen3.7 PlusAlibaba80.46%原記録
Grok 4.5 (high)SpaceXAI80.4%原記録
Qwen3.8-Flash-NextAlibaba79.77%原記録
AA-Omniscience / Index15 件の成績

基盤モデルの成績Artificial Analysis

知識の信頼性

score高いほど良い
  1. GPT-6 Astra43.73
  2. Claude Fable 5.143.45
  3. Claude Fable 543.3
  4. Claude Opus 537.07
  5. Gemini 3.1 Pro Preview31.88
  6. Grok 4.630.48
  7. Gemini 3.8 Flash29.55
  8. Muse Spark 1.227.2
  9. Gemini 3.7 Flash26.48
  10. Grok 4.525.32
  11. Muse Spark 1.324.93
  12. Gemini 3.6 Flash22.13
  13. GPT-5.6 Sol21.97
  14. Gemini 3.5 Flash20.82
  15. Kimi K319.7
AA-Omniscience / Index / score
モデルと実行構成成績出典
GPT-6 Astra (high)OpenAI43.73原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic43.45原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic43.3原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic37.07原記録
Gemini 3.1 Pro PreviewGoogle31.88原記録
Grok 4.6 (high)SpaceXAI30.48原記録
Gemini 3.8 Flash (high)Google29.55原記録
Muse Spark 1.2 (xhigh)Meta27.2原記録
Gemini 3.7 Flash (high)Google26.48原記録
Grok 4.5 (high)SpaceXAI25.32原記録
Muse Spark 1.3 (max)Meta24.93原記録
Gemini 3.6 Flash (high)Google22.13原記録
GPT-5.6 Sol (max)OpenAI21.97原記録
Gemini 3.5 Flash (medium)Google20.82原記録
Kimi K3 (max)Kimi19.7原記録
AA-Omniscience / Accuracy15 件の成績

基盤モデルの成績Artificial Analysis

知識問題の正答率

%高いほど良い
  1. Claude Fable 5.167.23%
  2. Claude Fable 565.35%
  3. GPT-6 Astra62.6%
  4. Claude Opus 560.87%
  5. GPT-5.6 Sol59.4%
  6. Gemini 3.7 Flash55.32%
  7. Gemini 3.1 Pro Preview54.85%
  8. Gemini 3.8 Flash54.6%
  9. GPT-5.3 Codex52.88%
  10. Grok 4.551.55%
  11. Gemini 3.5 Flash51.05%
  12. Gemini 3.6 Flash49.97%
  13. DeepSeek V4 Pro 081349.1%
  14. Grok 4.648.23%
  15. Kimi K347.58%
AA-Omniscience / Accuracy / %
モデルと実行構成成績出典
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic67.23%原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic65.35%原記録
GPT-6 Astra (max)OpenAI62.6%原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic60.87%原記録
GPT-5.6 Sol (max)OpenAI59.4%原記録
Gemini 3.7 Flash (high)Google55.32%原記録
Gemini 3.1 Pro PreviewGoogle54.85%原記録
Gemini 3.8 Flash (high)Google54.6%原記録
GPT-5.3 Codex (xhigh)OpenAI52.88%原記録
Grok 4.5 (high)SpaceXAI51.55%原記録
Gemini 3.5 Flash (medium)Google51.05%原記録
Gemini 3.6 Flash (high)Google49.97%原記録
DeepSeek V4 Pro 0813 (Reasoning, Max Effort)DeepSeek49.1%原記録
Grok 4.6 (high)SpaceXAI48.23%原記録
Kimi K3 (max)Kimi47.58%原記録
AA-Omniscience / Hallucination15 件の成績

基盤モデルの成績Artificial Analysis

幻覚率、低いほど良い

%低いほど良い
  1. MiniCPM5-1B0.9%
  2. G9v3-3B11.66%
  3. G9v3-39A5B13.04%
  4. Command A+14.16%
  5. LFM2.5-2.6B16%
  6. Grok 4.316.94%
  7. Qwen3.8 27B18.09%
  8. MiniMax-M318.43%
  9. Quasar 438B21.44%
  10. K-EXAONE 2.0 080322.6%
  11. Grok 4.624%
  12. Solar Pro 424.4%
  13. MiMo-V2.5-Pro24.7%
  14. Solar Open2 250B25.38%
  15. Granite 4.2 30B25.58%
AA-Omniscience / Hallucination / %
モデルと実行構成成績出典
MiniCPM5-1B (Non-reasoning)OpenBMB0.9%原記録
G9v3-3BAI9Stars11.66%原記録
G9v3-39A5BAI9Stars13.04%原記録
Command A+Cohere14.16%原記録
LFM2.5-2.6BLiquid AI16%原記録
Grok 4.3 (medium)SpaceXAI16.94%原記録
Qwen3.8 27B (Non-reasoning)Alibaba18.09%原記録
MiniMax-M3MiniMax18.43%原記録
Quasar 438B (max, based on GLM-5.2)Multiverse Computing21.44%原記録
K-EXAONE 2.0 0803 (K-EXAONE 2.0)LG AI Research22.6%原記録
Grok 4.6 (medium)SpaceXAI24%原記録
Solar Pro 4Upstage24.4%原記録
MiMo-V2.5-ProXiaomi24.7%原記録
Solar Open2 250BUpstage25.38%原記録
Granite 4.2 30BIBM25.58%原記録
MathVision / Moonshot report5 件の成績

提供元発表のモデル成績Moonshot AI (model-card report)

図を含む数学問題。

score高いほど良い
  1. Gemini 3 Pro86.1
  2. Kimi K2.584.2
  3. GPT-5.283
  4. Claude Opus 4.577.1
  5. Qwen3-VL-235B-A22B74.6
MathVision / Moonshot report / score
モデルと実行構成成績出典
Gemini 3 Pro (High Thinking Level; publisher rerun (*))Google86.1原記録
Kimi K2.5 (Thinking)Moonshot AI84.2原記録
GPT-5.2 (xhigh)OpenAI83原記録
Claude Opus 4.5 (Extended Thinking; publisher rerun (*))Anthropic77.1原記録
Qwen3-VL-235B-A22B (Thinking)Qwen74.6原記録
OCRBench / Moonshot report5 件の成績

提供元発表のモデル成績Moonshot AI (model-card report)

画像内の文字認識。

score高いほど良い
  1. Kimi K2.592.3
  2. Gemini 3 Pro90.3
  3. Qwen3-VL-235B-A22B87.5
  4. Claude Opus 4.586.5
  5. GPT-5.280.7
OCRBench / Moonshot report / score
モデルと実行構成成績出典
Kimi K2.5 (Thinking)Moonshot AI92.3原記録
Gemini 3 Pro (High Thinking Level; publisher rerun (*))Google90.3原記録
Qwen3-VL-235B-A22B (Thinking)Qwen87.5原記録
Claude Opus 4.5 (Extended Thinking; publisher rerun (*))Anthropic86.5原記録
GPT-5.2 (xhigh; publisher rerun (*))OpenAI80.7原記録
OmniDocBench 1.5 / Moonshot report5 件の成績

提供元発表のモデル成績Moonshot AI (model-card report)

文書読解。発表値は正規化編集距離の補数を100点換算。

score高いほど良い
  1. Kimi K2.588.8
  2. Gemini 3 Pro88.5
  3. Claude Opus 4.587.7
  4. GPT-5.285.7
  5. Qwen3-VL-235B-A22B82
OmniDocBench 1.5 / Moonshot report / score
モデルと実行構成成績出典
Kimi K2.5 (Thinking)Moonshot AI88.8原記録
Gemini 3 Pro (High Thinking Level)Google88.5原記録
Claude Opus 4.5 (Extended Thinking; publisher rerun (*))Anthropic87.7原記録
GPT-5.2 (xhigh)OpenAI85.7原記録
Qwen3-VL-235B-A22B (Thinking; publisher rerun (*))Qwen82原記録
VideoMMMU / Moonshot report5 件の成績

提供元発表のモデル成績Moonshot AI (model-card report)

複数分野の動画理解。

score高いほど良い
  1. Gemini 3 Pro87.6
  2. Kimi K2.586.6
  3. GPT-5.285.9
  4. Claude Opus 4.584.4
  5. Qwen3-VL-235B-A22B80
VideoMMMU / Moonshot report / score
モデルと実行構成成績出典
Gemini 3 Pro (High Thinking Level)Google87.6原記録
Kimi K2.5 (Thinking)Moonshot AI86.6原記録
GPT-5.2 (xhigh)OpenAI85.9原記録
Claude Opus 4.5 (Extended Thinking; publisher rerun (*))Anthropic84.4原記録
Qwen3-VL-235B-A22B (Thinking)Qwen80原記録
MotionBench / Moonshot report4 件の成績

提供元発表のモデル成績Moonshot AI (model-card report)

動画内の動きの理解。

score高いほど良い
  1. Kimi K2.570.4
  2. Gemini 3 Pro70.3
  3. GPT-5.264.8
  4. Claude Opus 4.560.3
MotionBench / Moonshot report / score
モデルと実行構成成績出典
Kimi K2.5 (Thinking)Moonshot AI70.4原記録
Gemini 3 Pro (High Thinking Level)Google70.3原記録
GPT-5.2 (xhigh)OpenAI64.8原記録
Claude Opus 4.5 (Extended Thinking)Anthropic60.3原記録
IFBench / MiniMax report7 件の成績

提供元発表のモデル成績MiniMax (internal evaluation)

詳細な指示への準拠。

score高いほど良い
  1. GPT-5.275
  2. MiniMax-M2.570
  3. MiniMax-M2.170
  4. Gemini 3 Pro70
  5. Claude Opus 4.558
  6. Claude Sonnet 4.557
  7. Claude Opus 4.653
IFBench / MiniMax report / score
モデルと実行構成成績出典
GPT-5.2 (Thinking; MiniMax internal evaluation)OpenAI75原記録
MiniMax-M2.5 (MiniMax internal evaluation)MiniMax70原記録
MiniMax-M2.1 (MiniMax internal evaluation)MiniMax70原記録
Gemini 3 Pro (MiniMax internal evaluation)Google70原記録
Claude Opus 4.5 (MiniMax internal evaluation)Anthropic58原記録
Claude Sonnet 4.5 (MiniMax internal evaluation)Anthropic57原記録
Claude Opus 4.6 (MiniMax internal evaluation)Anthropic53原記録
MMMLU / DeepSeek Base report3 件の成績

提供元発表のモデル成績DeepSeek (Base technical report)

5つの例を与え、正解との完全一致率で測る多言語知識。

%高いほど良い
  1. DeepSeek-V4-Pro-Base90.3%
  2. DeepSeek-V4-Flash-Base88.8%
  3. DeepSeek-V3.2-Base87.9%
MMMLU / DeepSeek Base report / %
モデルと実行構成成績出典
DeepSeek-V4-Pro-Base (Base; MMMLU EM; 5-shot; shared internal evaluation setup; sampling and tools not disclosed)DeepSeek90.3%原記録
DeepSeek-V4-Flash-Base (Base; MMMLU EM; 5-shot; shared internal evaluation setup; sampling and tools not disclosed)DeepSeek88.8%原記録
DeepSeek-V3.2-Base (Base; MMMLU EM; 5-shot; shared internal evaluation setup; sampling and tools not disclosed)DeepSeek87.9%原記録
OmniDocBench 1.5 / Qwen3.8-27B report5 件の成績

提供元発表のモデル成績Qwen (model-card report)

提供元の発表値で比較する文書画像の読み取り。

score高いほど良い
  1. Qwen3.7-Plus91.4
  2. Qwen3.8-27B91.1
  3. Qwen3.6-27B89.4
  4. Opus4.6 Max86.6
  5. Muse Glimmer-30B75.8
OmniDocBench 1.5 / Qwen3.8-27B report / score
モデルと実行構成成績出典
Qwen3.7-Plus (OmniDocBench 1.5; tools, reasoning effort and image settings not disclosed)Qwen91.4原記録
Qwen3.8-27B (OmniDocBench 1.5; tools, reasoning effort and image settings not disclosed)Qwen91.1原記録
Qwen3.6-27B (OmniDocBench 1.5; tools, reasoning effort and image settings not disclosed)Qwen89.4原記録
Opus4.6 Max (Max (table header); OmniDocBench 1.5 tools and image settings not disclosed)Anthropic86.6原記録
Muse Glimmer-30B (OmniDocBench 1.5; tools, reasoning effort and image settings not disclosed)Meta75.8原記録
MMMU-Pro / Google G31 (2026-02-19)5 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

ツールなしで画像など複数形式の情報を理解し推論します。

%高いほど良い
  1. Gemini 3 Pro81%
  2. Gemini 3.1 Pro80.5%
  3. GPT-5.279.5%
  4. Claude Sonnet 4.674.5%
  5. Claude Opus 4.673.9%
MMMU-Pro / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
Gemini 3 Pro (Thinking (High); No tools; Standard (10 options) and Vision average)Google81%原記録
Gemini 3.1 Pro (Thinking (High); No tools; Standard (10 options) and Vision average)Google80.5%原記録
GPT-5.2 (Thinking (xhigh); No tools; Standard (10 options) and Vision average)OpenAI79.5%原記録
Claude Sonnet 4.6 (Thinking (Max); No tools; Standard (10 options) and Vision average)Anthropic74.5%原記録
Claude Opus 4.6 (Thinking (Max); No tools; Standard (10 options) and Vision average)Anthropic73.9%原記録
MRCR v2, 128k / Google G31 (2026-02-19)5 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

長い文脈から指定情報8件を取り出す評価の128kまでの累積平均です。

%高いほど良い
  1. Gemini 3.1 Pro84.9%
  2. Claude Sonnet 4.684.9%
  3. Claude Opus 4.684%
  4. GPT-5.283.8%
  5. Gemini 3 Pro77%
MRCR v2, 128k / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
Gemini 3.1 Pro (Thinking (High); MRCR v2; 8 needles; 128k cumulative average)Google84.9%原記録
Claude Sonnet 4.6 (Thinking (Max); MRCR v2; 8 needles; 128k cumulative average)Anthropic84.9%原記録
Claude Opus 4.6 (Thinking (Max); MRCR v2; 8 needles; 128k cumulative average)Anthropic84%原記録
GPT-5.2 (Thinking (xhigh); MRCR v2; 8 needles; 128k cumulative average)OpenAI83.8%原記録
Gemini 3 Pro (Thinking (High); MRCR v2; 8 needles; 128k cumulative average)Google77%原記録
MRCR v2, 1M / Google G31 (2026-02-19)2 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.1 Pro release comparison)

1Mトークン地点で指定情報8件を取り出す評価です。

%高いほど良い
  1. Gemini 3.1 Pro26.3%
  2. Gemini 3 Pro26.3%
MRCR v2, 1M / Google G31 (2026-02-19) / %
モデルと実行構成成績出典
Gemini 3.1 Pro (Thinking (High); MRCR v2; 8 needles; 1M pointwise)Google26.3%原記録
Gemini 3 Pro (Thinking (High); MRCR v2; 8 needles; 1M pointwise)Google26.3%原記録
CharXiv Reasoning / Google G38 (2026-09-02)6 件の成績

提供元発表のモデル成績Google DeepMind (Gemini 3.8 Flash release comparison)

ツールなしで複雑な図表の情報をまとめます。

%高いほど良い
  1. Gemini 3.8 Flash86.2%
  2. GPT-5.6 Terra85.9%
  3. GPT-5.6 Sol85.8%
  4. Gemini 3.7 Flash84.5%
  5. Claude Opus 583.7%
  6. Claude Sonnet 570.1%
CharXiv Reasoning / Google G38 (2026-09-02) / %
モデルと実行構成成績出典
Gemini 3.8 Flash (Default sampling; exact thinking level unspecified; No tools)Google86.2%原記録
GPT-5.6 Terra (Maximum reasoning preferred; otherwise best available reported setting; No tools)OpenAI85.9%原記録
GPT-5.6 Sol (Exact reasoning level unspecified; No tools)OpenAI85.8%原記録
Gemini 3.7 Flash (Exact thinking level unspecified; No tools)Google84.5%原記録
Claude Opus 5 (Exact thinking level unspecified; No tools)Anthropic83.7%原記録
Claude Sonnet 5 (Maximum thinking preferred; otherwise best available reported setting; No tools)Anthropic70.1%原記録

検索モデル

MTEB / OpenAI embeddings report3 件の成績

検索モデルの成績OpenAI (developer documentation)

テキスト埋め込みモデルの評価。生成モデルの推論とは別です。

%高いほど良い
  1. text-embedding-3-large64.6%
  2. text-embedding-3-small62.3%
  3. text-embedding-ada-00261%
MTEB / OpenAI embeddings report / %
モデルと実行構成成績出典
text-embedding-3-large (Published MTEB result; evaluation dimensions unspecified)OpenAI64.6%原記録
text-embedding-3-small (Published MTEB result; evaluation dimensions unspecified)OpenAI62.3%原記録
text-embedding-ada-002 (Published MTEB result; evaluation dimensions unspecified)OpenAI61%原記録

時間と費用

Intelligence Index / Cost per task15 件の成績

基盤モデルの成績Artificial Analysis

課題当たりの加重平均費用

USD低いほど良い
  1. GLM 5.3 Flash0.18
  2. Muse Spark 1.20.55
  3. Gemini 3.8 Flash0.74
  4. GPT-5.6 Terra0.81
  5. Muse Spark 1.30.96
  6. Qwen3.8 2.4T A95B1.1
  7. Qwen3.8 Max1.19
  8. GPT-5.6 Sol1.25
  9. Grok 4.61.25
  10. GLM-5.31.26
  11. Kimi K31.58
  12. GPT-6 Astra2.57
  13. Claude Opus 54.21
  14. Claude Fable 55.62
  15. Claude Fable 5.16.12
Intelligence Index / Cost per task / USD
モデルと実行構成成績出典
GLM 5.3 Flash (GLM-5.3-Flash)Z AI0.18原記録
Muse Spark 1.2 (xhigh)Meta0.55原記録
Gemini 3.8 Flash (high)Google0.74原記録
GPT-5.6 Terra (max)OpenAI0.81原記録
Muse Spark 1.3 (max)Meta0.96原記録
Qwen3.8 2.4T A95BAlibaba1.1原記録
Qwen3.8 MaxAlibaba1.19原記録
GPT-5.6 Sol (max)OpenAI1.25原記録
Grok 4.6 (high)SpaceXAI1.25原記録
GLM-5.3 (max)Z AI1.26原記録
Kimi K3 (max)Kimi1.58原記録
GPT-6 Astra (max)OpenAI2.57原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic4.21原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic5.62原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic6.12原記録
Output speed15 件の成績

基盤モデルの成績Artificial Analysis

出力生成速度

tokens/s高いほど良い
  1. Gemini 3.8 Flash280.76
  2. Muse Spark 1.2269.24
  3. Muse Spark 1.3233.15
  4. GPT-5.6 Terra113.93
  5. GLM-5.383.31
  6. GPT-5.6 Sol75.78
  7. Claude Fable 5.168.22
  8. GPT-6 Astra64.26
  9. Claude Fable 562.97
  10. Grok 4.659.89
  11. Claude Opus 555.95
  12. GLM 5.3 Flash50.63
  13. Kimi K341.61
  14. Qwen3.8 2.4T A95B40.23
  15. Qwen3.8 Max40.15
Output speed / tokens/s
モデルと実行構成成績出典
Gemini 3.8 Flash (high)Google280.76原記録
Muse Spark 1.2 (xhigh)Meta269.24原記録
Muse Spark 1.3 (max)Meta233.15原記録
GPT-5.6 Terra (max)OpenAI113.93原記録
GLM-5.3 (max)Z AI83.31原記録
GPT-5.6 Sol (max)OpenAI75.78原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic68.22原記録
GPT-6 Astra (max)OpenAI64.26原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic62.97原記録
Grok 4.6 (high)SpaceXAI59.89原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic55.95原記録
GLM 5.3 Flash (GLM-5.3-Flash)Z AI50.63原記録
Kimi K3 (max)Kimi41.61原記録
Qwen3.8 2.4T A95BAlibaba40.23原記録
Qwen3.8 MaxAlibaba40.15原記録
API input price15 件の成績

基盤モデルの成績Artificial Analysis

入力100万トークンの価格

USD / 1M低いほど良い
  1. GLM 5.3 Flash0.15
  2. Gemini 3.8 Flash0.75
  3. Muse Spark 1.31.25
  4. Muse Spark 1.21.25
  5. GLM-5.31.4
  6. Grok 4.62
  7. Qwen3.8 Max2
  8. GPT-5.6 Terra2
  9. Qwen3.8 2.4T A95B2
  10. Kimi K33
  11. GPT-5.6 Sol4
  12. Claude Opus 55
  13. Claude Fable 5.110
  14. GPT-6 Astra10
  15. Claude Fable 510
API input price / USD / 1M
モデルと実行構成成績出典
GLM 5.3 Flash (GLM-5.3-Flash)Z AI0.15原記録
Gemini 3.8 Flash (high)Google0.75原記録
Muse Spark 1.3 (max)Meta1.25原記録
Muse Spark 1.2 (xhigh)Meta1.25原記録
GLM-5.3 (max)Z AI1.4原記録
Grok 4.6 (high)SpaceXAI2原記録
Qwen3.8 MaxAlibaba2原記録
GPT-5.6 Terra (max)OpenAI2原記録
Qwen3.8 2.4T A95BAlibaba2原記録
Kimi K3 (max)Kimi3原記録
GPT-5.6 Sol (max)OpenAI4原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic5原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic10原記録
GPT-6 Astra (max)OpenAI10原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic10原記録
API output price15 件の成績

基盤モデルの成績Artificial Analysis

出力100万トークンの価格

USD / 1M低いほど良い
  1. GLM 5.3 Flash0.5
  2. Gemini 3.8 Flash3.75
  3. Muse Spark 1.34.25
  4. Muse Spark 1.24.25
  5. GLM-5.34.4
  6. Grok 4.66
  7. Qwen3.8 Max6
  8. Qwen3.8 2.4T A95B6
  9. GPT-5.6 Terra12
  10. Kimi K315
  11. GPT-5.6 Sol20
  12. Claude Opus 525
  13. Claude Fable 5.150
  14. GPT-6 Astra50
  15. Claude Fable 550
API output price / USD / 1M
モデルと実行構成成績出典
GLM 5.3 Flash (GLM-5.3-Flash)Z AI0.5原記録
Gemini 3.8 Flash (high)Google3.75原記録
Muse Spark 1.3 (max)Meta4.25原記録
Muse Spark 1.2 (xhigh)Meta4.25原記録
GLM-5.3 (max)Z AI4.4原記録
Grok 4.6 (high)SpaceXAI6原記録
Qwen3.8 MaxAlibaba6原記録
Qwen3.8 2.4T A95BAlibaba6原記録
GPT-5.6 Terra (max)OpenAI12原記録
Kimi K3 (max)Kimi15原記録
GPT-5.6 Sol (max)OpenAI20原記録
Claude Opus 5 (Adaptive Reasoning, Max Effort)Anthropic25原記録
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)Anthropic50原記録
GPT-6 Astra (max)OpenAI50原記録
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)Anthropic50原記録

本文の出典と関連資料

  1. Anthropic. Demystifying evals for AI agents
  2. OpenAI. GDPval
  3. OpenAI. BrowseComp
  4. Parallel. Deep Research price-performance
  5. LangChain. Improving Deep Agents with harness engineering
  6. LangChain. Deep Agents source code
  7. DeepResearch Bench II. Evaluation code
  8. ResearchRubrics. Evaluation code
  9. Agents’ Last Exam. Tasks and evaluation