The ORCFLO Indexis an independent benchmark that evaluates large language models the way business professionals actually use them — across 40 real-world tasks spanning analysis, writing, extraction, summarization, and behavioral reliability. Each model is scored on three dimensions (quality, cost, and speed) by a panel of four independent judges. This report evaluates Gemini 3 Pro (Preview) in the context of 39 models from Anthropic, OpenAI, Google, Mistral, and others.
Gemini 3 Pro (Preview)
Gemini 3 Pro (Preview) ranks #5 overall in quality across 39 models. It's a top-tier performer, placing in the Leader quartile with particular strength in Output Consistency (95.0) and Extraction (92.5). It's moderately priced (#20 in cost) and slower (#30 in speed).
Key Findings
- #12 in Output Consistency. Gemini 3 Pro (Preview) scores 95.0 in this category, placing it in the Strong tier across all 39 models.
- #1 in Extraction. Scores 92.5, reinforcing strength across multiple Abilities categories.
- Cost: 0.0080 credits for all 40 tests. Ranks #20 of 39 in cost efficiency. Average response time: 26.7s (rank #30).
- vs. Gemini 2.5 Pro: 3.1 points ahead of in overall quality. Slower by 3.9s on average.
Model Performance: Quality & Cost
The chart below plots quality against cost for all 39 models in the ORCFLO Index. Each dot represents the average quality score a model achieved across our full basket of 40 real-world business tasks, alongside the total cost in credits to complete the entire test suite. Models in the upper-left quadrant deliver the highest quality at the lowest cost.
Model Performance: Quality & Time Elapsed
Quality alone doesn’t tell the full story — response time determines whether a model is viable for time-sensitive workflows. The chart below plots each model’s quality score against the total time required to complete all 40 tests. Models in the upper-left deliver the best quality with the least delay.
Category Scorecard
The ORCFLO Indexevaluates models using real-world business tasks — not academic puzzles or synthetic benchmarks. Each test case is designed to expose specific differences in how models handle tasks that professionals actually encounter: strategic analysis, document extraction, business writing, summarization, and behavioral reliability. Scores are averaged across five cases per category, and each category is ranked independently across all 39 models.
| Category | Score | Rank | Tier | vs. Gemini 2.5 Pro |
|---|---|---|---|---|
| Abilities — Core language tasks: what the model can produce when given a well-formed prompt. | ||||
| Analysis Reasoning, strategic judgment, disqualifying-factor detection | #10 | Leader | +1.9 | |
| Extraction Field accuracy, null handling, format compliance, zero fabrication | #1 | Leader | +13.8 | |
| Summarization Compression quality, key-point retention, length compliance | #5 | Leader | −2.4 | |
| Writing Tone, structure, persuasion, audience adaptation | #11 | Strong | +1.4 | |
| Behaviors — How the model acts under pressure: reliability, compliance, and restraint. | ||||
| Hallucination Fabrication detection, factual grounding, source fidelity | #15 | Strong | −1.0 | |
| Instruction Following Constraint adherence, format compliance, multi-part directives | #12 | Strong | +1.5 | |
| Refusal Calibration Appropriate refusal vs. over-refusal on legitimate requests | #13 | Strong | +4.8 | |
| Stability — Repeatability and predictability across identical inputs. | ||||
| Output Consistency Run-to-run reproducibility, format stability, score variance | #12 | Strong | +4.8 | |
Strengths and Weaknesses
Strengths
- #12 in Output Consistency — scores 95.0, placing in the Strong tier. Run-to-run reproducibility, format stability, score variance
- #1 in Extraction — scores 92.5, placing in the Leader tier. Field accuracy, null handling, format compliance, zero fabrication
- 26.7s average response — rank #30 in speed, suitable for most non-real-time workflows.
Cautions
- Not the cheapest option — while cost-competitive, models like Gemini 2.5 Flash deliver acceptable quality at a fraction of the cost.
Head-to-Head: Frontier Models
Gemini 3 Pro (Preview) is Google’s Google's top-performing frontier model in the ORCFLO Index. The table below compares it against the top-performing models from each major provider. Tier assignments use 25% quartiles across the full 39-model field. in the ORCFLO Index. The table below compares it against the top-performing models from each major provider. Tier assignments use 25% quartiles across the full 39-model field.
| Model | Quality Avg | Quality Rank | Cost Rank | Speed Rank | Tier |
|---|---|---|---|---|---|
| Claude Opus 4.6 | 90.6 | #2 | #31 | #28 | Leader |
| GPT 5.2 Pro | 90.3 | #3 | #35 | #35 | Leader |
| Gemini 3 Pro (Preview) | 89.9 | #5 | #20 | #30 | Leader |
| GPT 5 | 89.7 | #6 | #32 | #34 | Leader |
| GPT 5.1 | 89.5 | #7 | #25 | #25 | Leader |
| GPT 5.4 | 88.7 | #8 | #28 | #24 | Leader |
| Claude Opus 4.5 | 88.3 | #9 | #29 | #18 | Leader |
| Claude Opus 4.7 | 87 | #11 | #33 | #22 | Strong |
| Gemini 2.5 Pro | 86.8 | #12 | #21 | #29 | Strong |
| Gemini 3 Flash (Preview) | 85.2 | #15 | #13 | #17 | Strong |
| Claude Sonnet 4.6 | 84.7 | #16 | #26 | #23 | Strong |
When to Use Gemini 3 Pro (Preview)
The ORCFLO Index
This evaluation covers 40 cases across 8 categories. All tasks are text-only and English-only. Code generation, multi-turn conversation, multimodal tasks, and agentic workflows are not tested. All four judge models (Gemini 2.5 Pro, Claude Opus 4.6, GPT 5.2, and Mistral Large) are also contestants in the benchmark, which introduces potential scoring bias. We actively monitor for judge bias to ensure that no judge can systematically advantage its own provider's models. Cost and speed measurements reflect API pricing and latency as of the test date and will change as providers update their offerings.
How We Test
The ORCFLO Indexevaluates large language models across three independent dimensions — quality, cost, and speed — using real-world business tasks designed to expose the differences that matter for model selection. Each model is scored by a panel of four independent judges to reduce single-model bias.
- Test Cases
- 40 cases across 8 categories spanning Abilities (Analysis, Extraction, Summarization, Writing), Behaviors (Hallucination, Instruction Following, Refusal Calibration), and Stability (Output Consistency).
- Judge Panel
- Gemini 2.5 Pro, Claude Opus 4.6, GPT-5.2, Mistral Large. Each judge scores independently. Final score is the average across all four judges.
- Scoring
- Three independent ranks: quality (higher is better), cost (lower is better), speed (faster is better). No composite score — because composite scores hide the tradeoffs that drive model selection decisions.
- Tier Definitions
- LeaderTop 25%Ranks 1–10Strong25–50%Ranks 11–20Contender50–75%Ranks 21–30TrailingBottom 25%Ranks 31–39
Full methodology, case descriptions, and historical results are available at orcflo.com/explore/models.