若干层压缩.
压缩 tool result
在 message 中, assistant message 包含模型回答以及 tool_use 等, 而 user message 包含用户输入以及 tool_result 等. 如果当前 user message group (把 assistant message 作为边界, 两个 assistant message 中间所有的 user messages 全部合并起来算) 中可压缩的 tool result 总字符数超过 MAX_TOOL_RESULTS_PER_MESSAGE_CHARS, 就迭代地把最长的 tool_result 替换成 preview (包含原始 tool result 的本地持久化路径, 已经开头若干字符的预览) 直到总长度小于上限.
对 tool use id, 内部维护变量 seenIds (set, 表示已经见过的 tool use id) 和 replacements (dict, 记录被压缩的 tool use id 到 preview 的映射). 之后发送 llm api 调用前, 对于已经处理过的 tool use id 都会按照之前同样的处理 (压缩过的就重新替换成一样的 preview, 没压缩过的依旧不压缩) 避免破坏 prompt cache.
* Enforce the per-message budget on aggregate tool result size.
*
* For each user message whose tool_result blocks together exceed the
* per-message limit (see getPerMessageBudgetLimit), the largest FRESH
* (never-before-seen) results in THAT message are persisted to disk and
* replaced with previews.
* Messages are evaluated independently — a 150K result in one message and
* a 150K result in another are both under budget and untouched.
*
* State is tracked by tool_use_id in `state`. Once a result is seen its
* fate is frozen: previously-replaced results get the same replacement
* re-applied every turn from the cached preview string (zero I/O,
* byte-identical), and previously-unreplaced results are never replaced
* later (would break prompt cache).
*
* Each turn adds at most one new user message with tool_result blocks,
* so the per-message loop typically does the budget check at most once;
* all prior messages just re-apply cached replacements.
/**
* Default maximum aggregate size in characters for tool_result blocks within
* a SINGLE user message (one turn's batch of parallel tool results). When a
* message's blocks together exceed this, the largest blocks in that message
* are persisted to disk and replaced with previews until under budget.
* Messages are evaluated independently — a 150K result in one turn and a
* 150K result in the next are both untouched.
*
* This prevents N parallel tools from each hitting the per-tool max and
* collectively producing e.g. 10 × 40K = 400K in one turn's user message.
*/
export const MAX_TOOL_RESULTS_PER_MESSAGE_CHARS = 200_000
/**
* Build a message for large tool results with preview
*/
export function buildLargeToolResultMessage(
result: PersistedToolResult,
): string {
let message = `${PERSISTED_OUTPUT_TAG}\n`
message += `Output too large (${formatFileSize(result.originalSize)}). Full output saved to: ${result.filepath}\n\n`
message += `Preview (first ${formatFileSize(PREVIEW_SIZE_BYTES)}):\n`
message += result.preview
message += result.hasMore ? '\n...\n' : '\n'
message += PERSISTED_OUTPUT_CLOSING_TAG
return message
}
/**
* Extract candidate tool_result blocks grouped by API-level user message.
*
* normalizeMessagesForAPI merges consecutive user messages into one
* (Bedrock compat; 1P does the same server-side), so parallel tool
* results that arrive as N separate user messages in our state become
* ONE user message on the wire. The budget must group the same way or
* it would see N under-budget messages instead of one over-budget
* message and fail to enforce exactly when it matters most.
*
* A "group" is a maximal run of user messages NOT separated by an
* assistant message. Only assistant messages create wire-level
* boundaries — normalizeMessagesForAPI filters out progress entirely
* and merges attachment / system(local_command) INTO adjacent user
* blocks, so those types do NOT break groups here either.
*
* This matters for abort-during-parallel-tools paths: agent_progress
* messages (non-ephemeral, persisted in REPL state) can interleave
* between fresh tool_result messages. If we flushed on progress, those
* tool_results would split into under-budget groups, slip through
* unreplaced, get frozen, then be merged by normalizeMessagesForAPI
* into one over-budget wire message — defeating the feature.
*
* Only groups with at least one eligible candidate are returned.
*/
function collectCandidatesByMessage(
messages: Message[],
): ToolResultCandidate[][]
...
/**
* Extract candidate tool_result blocks from a single user message: blocks
* that are non-empty, non-image, and not already compacted by tag (i.e. by
* the per-tool limit, or an earlier iteration of this same query call).
* Returns [] for messages with no eligible blocks.
*/
function collectCandidatesFromMessage(message: Message): ToolResultCandidate[] ...
后面有 snip compact 环节, 但因为代码缺失从略.
Micro-compact
- 如果发送请求时距离上一个请求过了太久, 服务端的 prompt cache 早就失效了, 干脆把旧的 tool results 清理掉 (替换成 TIME_BASED_MC_CLEARED_MESSAGE) 再发请求 (保留最近 keepRecent 个, 其他压缩).
- 如果 prompt cache 还在, 就用 Anthropic 特有的 context editing 接口压缩 tool results.
// Time-based trigger runs first and short-circuits. If the gap since the
// last assistant message exceeds the threshold, the server cache has expired
// and the full prefix will be rewritten regardless — so content-clear old
// tool results now, before the request, to shrink what gets rewritten.
// Cached MC (cache-editing) is skipped when this fires: editing assumes a
// warm cache, and we just established it's cold.
const compactableIds = collectCompactableToolIds(messages)
// Floor at 1: slice(-0) returns the full array (paradoxically keeps
// everything), and clearing ALL results leaves the model with zero working
// context. Neither degenerate is sensible — always keep at least the last.
const keepRecent = Math.max(1, config.keepRecent)
const keepSet = new Set(compactableIds.slice(-keepRecent))
const clearSet = new Set(compactableIds.filter(id => !keepSet.has(id)))
// Inline from utils/toolResultStorage.ts — importing that file pulls in
// sessionStorage → utils/messages → services/api/errors, completing a
// circular-deps loop back through this file via promptCacheBreakDetection.
// Drift is caught by a test asserting equality with the source-of-truth.
export const TIME_BASED_MC_CLEARED_MESSAGE = '[Old tool result content cleared]'
// Only compact these tools
const COMPACTABLE_TOOLS = new Set<string>([
FILE_READ_TOOL_NAME,
...SHELL_TOOL_NAMES,
GREP_TOOL_NAME,
GLOB_TOOL_NAME,
WEB_SEARCH_TOOL_NAME,
WEB_FETCH_TOOL_NAME,
FILE_EDIT_TOOL_NAME,
FILE_WRITE_TOOL_NAME,
])
/**
* Cached microcompact path - uses cache editing API to remove tool results
* without invalidating the cached prefix.
*
* Key differences from regular microcompact:
* - Does NOT modify local message content (cache_reference and cache_edits are added at API layer)
* - Uses count-based trigger/keep thresholds from GrowthBook config
* - Takes precedence over regular microcompact (no disk persistence)
* - Tracks tool results and queues cache edits for the API layer
*/
之后有 context collapse 环节, 因为代码缺失从略.
Auto-compact
如果上下文大于阈值, 会先 trySessionMemoryCompaction (TODO: 还没看), 如果失败或不适用再构造 prompt 专门调一次 llm 总结. 连续失败 3 次则不再同个 session 尝试 autocompact.
// Reserve this many tokens for output during compaction
// Based on p99.99 of compact summary output being 17,387 tokens.
const MAX_OUTPUT_TOKENS_FOR_SUMMARY = 20_000
export function getEffectiveContextWindowSize(model: string): number {
const reservedTokensForSummary = Math.min(
getMaxOutputTokensForModel(model),
MAX_OUTPUT_TOKENS_FOR_SUMMARY,
)
let contextWindow = getContextWindowForModel(model, getSdkBetas())
...
return contextWindow - reservedTokensForSummary
}
export const AUTOCOMPACT_BUFFER_TOKENS = 13_000
const autocompactThreshold =
effectiveContextWindow - AUTOCOMPACT_BUFFER_TOKENS
export function getCompactPrompt(customInstructions?: string): string {
let prompt = NO_TOOLS_PREAMBLE + BASE_COMPACT_PROMPT
if (customInstructions && customInstructions.trim() !== '') {
prompt += `\n\nAdditional Instructions:\n${customInstructions}`
}
prompt += NO_TOOLS_TRAILER
return prompt
}
// Stop trying autocompact after this many consecutive failures.
// BQ 2026-03-10: 1,279 sessions had 50+ consecutive failures (up to 3,272)
// in a single session, wasting ~250K API calls/day globally.
const MAX_CONSECUTIVE_AUTOCOMPACT_FAILURES = 3
// Increment consecutive failure count for circuit breaker.
// The caller threads this through autoCompactTracking so the
// next query loop iteration can skip futile retry attempts.
const prevFailures = tracking?.consecutiveFailures ?? 0
const nextFailures = prevFailures + 1
if (nextFailures >= MAX_CONSECUTIVE_AUTOCOMPACT_FAILURES) {
logForDebugging(
`autocompact: circuit breaker tripped after ${nextFailures} consecutive failures — skipping future attempts this session`,
{ level: 'warn' },
)
}
// Aggressive no-tools preamble. The cache-sharing fork path inherits the
// parent's full tool set (required for cache-key match), and on Sonnet 4.6+
// adaptive-thinking models the model sometimes attempts a tool call despite
// the weaker trailer instruction. With maxTurns: 1, a denied tool call means
// no text output → falls through to the streaming fallback (2.79% on 4.6 vs
// 0.01% on 4.5). Putting this FIRST and making it explicit about rejection
// consequences prevents the wasted turn.
const NO_TOOLS_PREAMBLE = `CRITICAL: Respond with TEXT ONLY. Do NOT call any tools.
- Do NOT use Read, Bash, Grep, Glob, Edit, Write, or ANY other tool.
- You already have all the context you need in the conversation above.
- Tool calls will be REJECTED and will waste your only turn — you will fail the task.
- Your entire response must be plain text: an <analysis> block followed by a <summary> block.
`
const NO_TOOLS_TRAILER =
'\n\nREMINDER: Do NOT call any tools. Respond with plain text only — ' +
'an <analysis> block followed by a <summary> block. ' +
'Tool calls will be rejected and you will fail the task.'
const BASE_COMPACT_PROMPT = `Your task is to create a detailed summary of the conversation so far, paying close attention to the user's explicit requests and your previous actions.
This summary should be thorough in capturing technical details, code patterns, and architectural decisions that would be essential for continuing development work without losing context.
${DETAILED_ANALYSIS_INSTRUCTION_BASE}
Your summary should include the following sections:
1. Primary Request and Intent: Capture all of the user's explicit requests and intents in detail
2. Key Technical Concepts: List all important technical concepts, technologies, and frameworks discussed.
3. Files and Code Sections: Enumerate specific files and code sections examined, modified, or created. Pay special attention to the most recent messages and include full code snippets where applicable and include a summary of why this file read or edit is important.
4. Errors and fixes: List all errors that you ran into, and how you fixed them. Pay special attention to specific user feedback that you received, especially if the user told you to do something differently.
5. Problem Solving: Document problems solved and any ongoing troubleshooting efforts.
6. All user messages: List ALL user messages that are not tool results. These are critical for understanding the users' feedback and changing intent.
7. Pending Tasks: Outline any pending tasks that you have explicitly been asked to work on.
8. Current Work: Describe in detail precisely what was being worked on immediately before this summary request, paying special attention to the most recent messages from both user and assistant. Include file names and code snippets where applicable.
9. Optional Next Step: List the next step that you will take that is related to the most recent work you were doing. IMPORTANT: ensure that this step is DIRECTLY in line with the user's most recent explicit requests, and the task you were working on immediately before this summary request. If your last task was concluded, then only list next steps if they are explicitly in line with the users request. Do not start on tangential requests or really old requests that were already completed without confirming with the user first.
If there is a next step, include direct quotes from the most recent conversation showing exactly what task you were working on and where you left off. This should be verbatim to ensure there's no drift in task interpretation.
Here's an example of how your output should be structured:
<example>
<analysis>
[Your thought process, ensuring all points are covered thoroughly and accurately]
</analysis>
<summary>
1. Primary Request and Intent:
[Detailed description]
2. Key Technical Concepts:
- [Concept 1]
- [Concept 2]
- [...]
3. Files and Code Sections:
- [File Name 1]
- [Summary of why this file is important]
- [Summary of the changes made to this file, if any]
- [Important Code Snippet]
- [File Name 2]
- [Important Code Snippet]
- [...]
4. Errors and fixes:
- [Detailed description of error 1]:
- [How you fixed the error]
- [User feedback on the error if any]
- [...]
5. Problem Solving:
[Description of solved problems and ongoing troubleshooting]
6. All user messages:
- [Detailed non tool use user message]
- [...]
7. Pending Tasks:
- [Task 1]
- [Task 2]
- [...]
8. Current Work:
[Precise description of current work]
9. Optional Next Step:
[Optional Next step to take]
</summary>
</example>
Please provide your summary based on the conversation so far, following this structure and ensuring precision and thoroughness in your response.
There may be additional summarization instructions provided in the included context. If so, remember to follow these instructions when creating the above summary. Examples of instructions include:
<example>
## Compact Instructions
When summarizing the conversation focus on typescript code changes and also remember the mistakes you made and how you fixed them.
</example>
<example>
# Summary instructions
When you are using compact - please focus on test output and code changes. Include file reads verbatim.
</example>
`