ORCFLO Index

Model Evaluation: GPT 5.4 — May 10, 2026

The ORCFLO Indexis an independent benchmark that evaluates large language models the way business professionals actually use them — across real-world tasks spanning analysis, writing, extraction, summarization, and behavioral reliability. Each model is scored on three dimensions (quality, cost, and speed) by a panel of four independent judges. This report evaluates GPT 5.4 in the context of 32 models from ORCFLO Index — May 10, 2026 cohort · 32 total models tested · Anthropic, Google, OpenAI, Mistral.

OpenAIOn ORCFLO

GPT 5.4

The Bottom Line

GPT 5.4 ranks #8 of 32 overall with a quality score of 90.76, placing it behind six other frontier entries including GPT 5, GPT 5.5, GPT 5.1, GPT 5.2, Gemini 3 Pro, and Claude Opus 4.6. Its behavioral discipline is the headline: top-ten finishes in Instruction Following (#3), Hallucination (#7), and Refusal Calibration (#8). Cost (#25) and speed (#23) are weak. A reliable, well-behaved generalist that you pay a premium to operate.

Quality

90.8

#8 of 321-Leader

+4.8 vs median · -4.3 from #1

Cost

2.2×median

$0.0146 per case

#25 of 324-Highest

170× cheapest in field

Speed

1.3×median

13.7s per case

#23 of 323-Moderate

5.7× fastest in field

Key Findings

Behavioral leader, ability mid-pack: The model places in the leader tier across all three behavior categories but only reaches #9 in Analysis and #13 in both Writing and Extraction.
Instruction Following ranks #3 of 32 at 94.3, making it one of the most directive-compliant models in the cohort — a meaningful edge for structured workflows.
Summarization is the soft spot: at 86.6 (#23), GPT 5.4 trails most of its quality peers and drops out of the strong tier in this category.
Operating costs are uncompetitive: at $0.0146 per case (#25) and 13.7s latency (#23), GPT 5.4 is more expensive and slower than higher-ranked siblings GPT 5.1 and GPT 5.2.

Model Performance: Quality & Cost

The chart below plots quality against cost for all 32 models in the ORCFLO Index. Each dot represents the average quality score a model achieved across the full basket of real-world business tasks, alongside the cost in credits to complete the entire test suite. Models in the upper-left quadrant deliver the highest quality at the lowest cost.

Figure 1. Quality vs. cost across all 32 models. Upper-left quadrant = highest value. GPT 5.4 highlighted. P50 median lines shown on both axes.

Model Performance: Quality & Time Elapsed

Quality alone doesn’t tell the full story — response time determines whether a model is viable for time-sensitive workflows. The chart below plots each model’s quality score against the total time required to complete the test suite. Models in the upper-left deliver the best quality with the least delay.

Figure 2. Quality vs. response time across all 32 models. Upper-left quadrant = best performance. GPT 5.4 highlighted.

Category Scorecard

The ORCFLO Indexevaluates models using real-world business tasks — not academic puzzles or synthetic benchmarks. Each test case is designed to expose specific differences in how models handle the work professionals actually do. Scores are averaged across each category and ranked independently across all 32 models.

GPT 5.4 Performance by Category

Category	Score	Rank	Tier
Abilities — Core language tasks: what the model can produce when given a well-formed prompt.
AnalysisReasoning, strategic judgment, disqualifying-factor detection	92.1	#9	Strong
ExtractionField accuracy, null handling, format compliance, zero fabrication	85.7	#13	Strong
SummarizationCompression quality, key-point retention, length compliance	86.6	#23	Contender
WritingTone, structure, persuasion, audience adaptation	89.8	#13	Strong
Behaviors — How the model acts under pressure: reliability, compliance, and restraint.
HallucinationFabrication detection, factual grounding, source fidelity	95.2	#7	Leader
Instruction FollowingConstraint adherence, format compliance, multi-part directives	94.3	#3	Leader
Refusal CalibrationAppropriate refusal vs. over-refusal on legitimate requests	91.0	#8	Leader
Stability — Repeatability and predictability across identical inputs.
Output ConsistencyRun-to-run reproducibility, format stability, score variance	91.4	#15	Strong

Strengths and Cautions

Strengths

Instruction Following (#3 of 32, 94.3): Among the top three models in the cohort for adhering to prompts and constraints — valuable for compliance-sensitive automation.
Hallucination control (#7 of 32, 95.2): Leader-tier factual reliability, suitable for content where accuracy matters more than throughput.
Analysis (#9 of 32, 92.1): Strong analytical reasoning that holds its own against the higher-ranked Gemini and Claude peers.

Cautions

Summarization weakness (#23 of 32, 86.6): Notably lower than peers — avoid for high-volume document condensation workloads.
Cost-quality trade-off is poor: GPT 5.1 (#5) delivers higher quality at $0.0100 versus GPT 5.4's $0.0146, making the sibling a better value within the same family.
Speed rank #23 of 32 (13.7s): Slower than Claude Opus 4.5 (10.2s) and GPT 5.1 (7.6s), which constrains interactive or latency-sensitive applications.

Head-to-Head: Frontier Models

GPT 5.4 is OpenAI’s upper mid-tier contender in the ORCFLO Index. The table below compares it against the top-performing models from each major provider. Tier assignments use 25% quartiles across the full 32-model field.

Frontier Model Comparison

Model	Quality Avg	Quality Rank	Cost Rank	Speed Rank
GPT 5.1	92.7	#5	#22	#13
Gemini 2.5 Pro	92.0	#6	#19	#27
GPT 5.2	91.9	#7	#23	#19
GPT 5.4	90.8	#8	#25	#23
Claude Opus 4.5	90.7	#9	#28	#15
GPT 5 Mini	89.6	#10	#14	#31
Gemini 3 Flash (Preview)	88.7	#11	#12	#16

When to Use GPT 5.4

Best pickCompliance-driven workflows where strict instruction adherence and low hallucination rates are non-negotiable.

Best pickAnalytical tasks requiring structured reasoning — research synthesis, evaluation, or decision support.

ConsiderGeneral-purpose enterprise assistant work, but only if the buyer cannot justify GPT 5.1 (#5) at a lower cost.

AvoidHigh-volume summarization pipelines — the #23 ranking in this category undercuts the value proposition.

AvoidLatency-sensitive customer-facing applications where 13.7s response times degrade user experience.

The ORCFLO Index

This evaluation covers 40 cases across 8 categories. All tasks are text-only and English-only. Code generation, multi-turn conversation, multimodal tasks, and agentic workflows are not tested. Each contestant is scored by a panel of four independent judges — Gemini 2.5 Pro, Claude Opus 4.7, GPT 5.5, and Mistral Large — with final scores averaged across all four. Cost and speed measurements reflect API pricing and latency as of the test date (May 10, 2026) and will change as providers update their offerings.

How We Test

The ORCFLO Indexevaluates large language models across three independent dimensions — quality, cost, and speed — using real-world business tasks designed to expose the differences that matter for model selection. Each model is scored by a panel of four independent judges to reduce single-model bias.

Test Cases: 40 cases across 8 categories spanning Abilities (Analysis, Extraction, Summarization, Writing), Behaviors (Hallucination, Instruction Following, Refusal Calibration), and Stability (Output Consistency).
Judge Panel: Gemini 2.5 Pro, Claude Opus 4.7, GPT 5.5, and Mistral Large. Each judge scores independently. Final score is the average across all four.
Scoring: Three independent ranks: quality (higher is better), cost (lower is better), speed (faster is better). No composite score — composites hide the tradeoffs that drive model-selection decisions.
Tier Definitions: Leader
Quality ≥ 90.8
Ranks 1–8
Strong
≥ 85.9
Ranks 9–16
Contender
≥ 80.9
Ranks 17–24
Trailing
< 80.9
Ranks 25–32