The ORCFLO Indexis an independent benchmark that evaluates large language models the way business professionals actually use them — across real-world tasks spanning analysis, writing, extraction, summarization, and behavioral reliability. Each model is scored on three dimensions (quality, cost, and speed) by a panel of four independent judges. This report evaluates Claude Sonnet 4.6 in the context of 32 models from ORCFLO Index — May 10, 2026 cohort · 32 total models tested · Anthropic, Google, OpenAI, Mistral.
Claude Sonnet 4.6
Claude Sonnet 4.6 lands at #14 of 32 with a quality score of 88.17, placing it in the upper-middle of the field but well behind frontier leaders like GPT 5 (95.04), Gemini 3 Pro (93.17), and stablemate Claude Opus 4.6 (92.78). The model shows a pronounced split personality: elite on Writing, Analysis, Hallucination, and Output Consistency, but trailing badly on Instruction Following and Summarization. A high-craft generalist with sharp blind spots that buyers must scope around.
Key Findings
- Top-tier writing and analysis — Writing ranks #4 of 32 (93.0) and Analysis ranks #8 (92.1), placing Sonnet 4.6 in the leader tier for long-form reasoning and prose tasks despite its #14 overall position.
- Exceptional reliability signals — Hallucination scores 95.5 (rank #3) and Output Consistency hits 96.6 (rank #4), among the most trustworthy in the cohort for factual integrity and repeatable outputs.
- Instruction Following is a critical weakness — at 80.5 the model ranks #31 of 32, second-to-last in the entire field, a striking gap relative to its other behavioral scores.
- Cost and speed are uncompetitive — at $0.0118 per case (#24) and 13.7s response time (#24), Sonnet 4.6 is more expensive and slower than several higher-ranked peers including Gemini 3 Flash and Gemini 2.5 Flash.
Model Performance: Quality & Cost
The chart below plots quality against cost for all 32 models in the ORCFLO Index. Each dot represents the average quality score a model achieved across the full basket of real-world business tasks, alongside the cost in credits to complete the entire test suite. Models in the upper-left quadrant deliver the highest quality at the lowest cost.
Model Performance: Quality & Time Elapsed
Quality alone doesn’t tell the full story — response time determines whether a model is viable for time-sensitive workflows. The chart below plots each model’s quality score against the total time required to complete the test suite. Models in the upper-left deliver the best quality with the least delay.
Category Scorecard
The ORCFLO Indexevaluates models using real-world business tasks — not academic puzzles or synthetic benchmarks. Each test case is designed to expose specific differences in how models handle the work professionals actually do. Scores are averaged across each category and ranked independently across all 32 models.
| Category | Score | Rank | Tier |
|---|---|---|---|
| Abilities — Core language tasks: what the model can produce when given a well-formed prompt. | |||
| AnalysisReasoning, strategic judgment, disqualifying-factor detection | #8 | Leader | |
| ExtractionField accuracy, null handling, format compliance, zero fabrication | #24 | Contender | |
| SummarizationCompression quality, key-point retention, length compliance | #26 | Trailing | |
| WritingTone, structure, persuasion, audience adaptation | #4 | Leader | |
| Behaviors — How the model acts under pressure: reliability, compliance, and restraint. | |||
| HallucinationFabrication detection, factual grounding, source fidelity | #3 | Leader | |
| Instruction FollowingConstraint adherence, format compliance, multi-part directives | #31 | Trailing | |
| Refusal CalibrationAppropriate refusal vs. over-refusal on legitimate requests | #10 | Strong | |
| Stability — Repeatability and predictability across identical inputs. | |||
| Output ConsistencyRun-to-run reproducibility, format stability, score variance | #4 | Leader | |
Strengths and Cautions
Strengths
- Writing quality (rank #4, 93.0) — among the top five models in the cohort for prose tasks, outperforming most frontier models including GPT 5.4 and Claude Opus 4.5 in this category.
- Hallucination control (rank #3, 95.5) — one of the three most factually grounded models tested, making it suitable for use cases where fabrication risk is unacceptable.
- Output Consistency (rank #4, 96.6) — leader-tier reproducibility, valuable for production workflows where variance across runs creates downstream cost.
Cautions
- Instruction Following at rank #31 of 32 (80.5) is a serious concern for templated workflows, structured outputs, or any task with strict formatting requirements.
- Summarization trails at rank #26 of 32 (75.8), well into the bottom quartile — buyers needing high-volume document condensation should look elsewhere.
- Cost-to-quality ratio is unfavorable — Gemini 3 Flash (#11, $0.0020) and Gemini 2.5 Flash (#13, $0.0004) deliver comparable or better quality at a fraction of Sonnet 4.6's $0.0118 per case.
Head-to-Head: Frontier Models
Claude Sonnet 4.6 is Anthropic’s mid-tier specialist in the ORCFLO Index. The table below compares it against the top-performing models from each major provider. Tier assignments use 25% quartiles across the full 32-model field.
| Model | Quality Avg | Quality Rank | Cost Rank | Speed Rank |
|---|---|---|---|---|
| Gemini 3 Flash (Preview) | 88.7 | #11 | #12 | #16 |
| Claude Opus 4.7 | 88.6 | #12 | #30 | #25 |
| Gemini 2.5 Flash | 88.4 | #13 | #7 | #18 |
| Claude Sonnet 4.6 | 88.2 | #14 | #24 | #24 |
| o3 | 86.2 | #15 | #26 | #22 |
| GPT 5 Nano | 85.9 | #16 | #10 | #30 |
| o4-mini | 85.9 | #17 | #21 | #12 |
When to Use Claude Sonnet 4.6
The ORCFLO Index
This evaluation covers 40 cases across 8 categories. All tasks are text-only and English-only. Code generation, multi-turn conversation, multimodal tasks, and agentic workflows are not tested. Each contestant is scored by a panel of four independent judges — Gemini 2.5 Pro, Claude Opus 4.7, GPT 5.5, and Mistral Large — with final scores averaged across all four. Cost and speed measurements reflect API pricing and latency as of the test date (May 10, 2026) and will change as providers update their offerings.
How We Test
The ORCFLO Indexevaluates large language models across three independent dimensions — quality, cost, and speed — using real-world business tasks designed to expose the differences that matter for model selection. Each model is scored by a panel of four independent judges to reduce single-model bias.
- Test Cases
- 40 cases across 8 categories spanning Abilities (Analysis, Extraction, Summarization, Writing), Behaviors (Hallucination, Instruction Following, Refusal Calibration), and Stability (Output Consistency).
- Judge Panel
- Gemini 2.5 Pro, Claude Opus 4.7, GPT 5.5, and Mistral Large. Each judge scores independently. Final score is the average across all four.
- Scoring
- Three independent ranks: quality (higher is better), cost (lower is better), speed (faster is better). No composite score — composites hide the tradeoffs that drive model-selection decisions.
- Tier Definitions
- LeaderQuality ≥ 90.8Ranks 1–8Strong≥ 85.9Ranks 9–16Contender≥ 80.9Ranks 17–24Trailing< 80.9Ranks 25–32