Overview
Every AI model has a limited amount of information it can consider at once — this is called its context window. When a step in your workflow involves many tool calls, long outputs, or complex conversations, the context window can fill up. ORCFLO monitors this automatically and takes action to keep your workflow running smoothly.
What is a Context Window?
Think of a context window like a desk. The AI model can only work with whatever fits on the desk at one time. This includes the step's instructions, any data passed in from previous steps, tool call results, and the model's own responses.
Different models have different context window sizes. Larger windows can handle more information, but every model has a limit. When a step's conversation approaches that limit, ORCFLO needs to make room.
| Usage Level | What It Means | Action Needed |
|---|---|---|
| Below 75% | Normal operation. The model has plenty of room. | None |
| 75% – 89% | High usage. The step is consuming a significant amount of context. | Monitor. Consider splitting into smaller steps. |
| 90%+ | Near capacity. Compaction may be triggered soon. | Strongly consider breaking this step into smaller, focused steps. |
| Compacted | The context window filled up and older messages were trimmed. | Redesign the step to reduce scope. |
Context Usage Indicator
During and after execution, steps that use 75% or more of their context window display a small indicator badge in the step header. This badge shows a mini progress ring and the current usage percentage.
- Orange badge (75%–89%): High usage. The step is working but approaching the limit.
- Orange badge (90%+): Near capacity. Compaction may be triggered if the step continues.
- Amber badge with warning icon: Context compaction has occurred. Some earlier conversation history was trimmed to make room.
Hover for details
Hover over the indicator badge to see a tooltip with the exact usage percentage and guidance on what to do.
Context Compaction
When a step's context window fills up, ORCFLO automatically performs context compactionto keep the step running. This is a safety mechanism — without it, the step would fail with an overflow error.
Compaction is automatic
You don't need to configure or trigger compaction. ORCFLO handles it behind the scenes whenever a step approaches its context limit.
How Compaction Works
Usage threshold reached
ORCFLO detects that the step's conversation is approaching the model's context limit.
Older messages trimmed
The oldest messages in the conversation are removed while preserving the system prompt, the most recent messages, and critical context.
Step continues
The step resumes execution with a smaller context. The model may lose some awareness of earlier tool results or intermediate reasoning.
Compaction can affect quality
When context is compacted, the model loses access to older parts of the conversation. This can lead to repeated work, forgotten instructions, or lower-quality output. If a step triggers compaction, it's a strong signal to break it into smaller steps.
Best Practices
Keeping context usage low leads to better results and more reliable workflows.
- Keep steps focused:Each step should do one thing well. A step that searches the web, analyzes results, and writes a report is doing too much — split it into three steps.
- Limit tool count per step: Every tool call and its result takes up context space. If a step uses many tools, the results accumulate quickly.
- Use targeted prompts: Concise, specific instructions use less context than long, open-ended ones. Tell the model exactly what you need.
- Choose the right model: If a step consistently runs high on context, consider using a model with a larger context window. See Model Selection for details.
- Watch the indicator:If you see the context usage badge appear on a step, it's worth reviewing whether that step can be simplified.
Troubleshooting
A step shows the compaction warning
This means the step's context window filled up and older messages were trimmed. The step still completed, but quality may be reduced. To fix this:
- Break the step into two or more smaller steps
- Reduce the number of tools assigned to the step
- Simplify the prompt to be more focused
A step failed with a context overflow error
In rare cases, the context window can overflow before compaction can act (for example, if a single tool result is extremely large). If this happens:
- Check which tool returned a very large result and consider whether it's necessary
- Use a model with a larger context window
- Split the workload across multiple steps so no single step handles too much data