Deep Analysis

OpenClaw Prompt Construction & Agent Loop

How a user message flows through string assembly, the pi-agent-core harness, stream wrapping, tool execution, and compaction — with full lineage tracking.

github.com/openclaw/openclaw ~3,290 lines in run/attempt.ts 14 source files analyzed
01
System Prompt Construction
All 25+ string components that assemble into the final system prompt, with exact file lineage

System Prompt Assembly — Flow Diagram

Mermaid
Component Lineage Table — 25 String Components
Component / Function Source File Lines Description
buildAgentSystemPrompt() src/agents/system-prompt.ts 439–1048 Master builder — orchestrates all sections in order
buildFullBootstrapPromptLines() src/agents/bootstrap-prompt.ts 1–13 Full bootstrap: read BOOTSTRAP.md, complete workflow
buildLimitedBootstrapPromptLines() src/agents/bootstrap-prompt.ts 15–25 Limited: explain blocker, suggest next step
BOOTSTRAP.md content .pi/prompts/cl.md 58 lines Changelog audit skill — audit entries before release
BOOTSTRAP.md content .pi/prompts/is.md 22 lines Issue analysis skill — audit GitHub issues
buildBootstrapPromptWarning() src/agents/bootstrap-budget.ts — Prepends "⚠️ Bootstrap budget..." warning if near limit
buildSkillsSection() src/agents/system-prompt.ts 156–172 Skills instructions: "scan available_skills..."
buildMemorySection() src/agents/system-prompt.ts 174–187 Memory citations guidance via plugins/memory-state.ts
buildDocsSection() src/agents/system-prompt.ts 395–425 Docs links: docs.openclaw.ai, GitHub, Discord, ClawhHub
buildMessagingSection() src/agents/system-prompt.ts 335–382 sessions_spawn, sessions_send, subagents guidance
buildVoiceSection() src/agents/system-prompt.ts 384–393 TTS hint via buildTtsSystemPromptHint()
buildHeartbeatSection() src/agents/system-prompt.ts 127–138 "If heartbeat poll: reply HEARTBEAT_OK"
buildExecutionBiasSection() src/agents/system-prompt.ts 299–313 "Actionable request: act in this turn..."
buildToolCallStyleSection() src/agents/system-prompt.ts 747–765 Default: don't narrate routine tool calls
Safety Constitution src/agents/system-prompt.ts 672–678 No independent goals, prioritize safety, no manipulation
CLI Quick Reference src/agents/system-prompt.ts 777–788 openclaw gateway status/restart, config.schema.lookup
Self-Update Rules src/agents/system-prompt.ts 793–803 Updates only on explicit user request
Silent Reply Rules src/agents/system-prompt.ts 955–970 SILENT_REPLY_TOKEN when nothing to say
buildProjectContextSection() src/agents/system-prompt.ts 95–125 Loads workspace files: agents.md, soul.md, identity.md...
CONTEXT_FILE_ORDER map src/agents/system-prompt.ts 44–52 File priority: agents.md(10) → soul.md(20) → ... → memory.md(70)
buildWebchatCanvasSection() src/agents/system-prompt.ts 275–297 [embed...] directives for webchat only
buildOwnerIdentityLine() src/agents/system-prompt.ts 232–246 "Authorized senders: [hash or raw numbers]"
buildTimeSection() src/agents/system-prompt.ts 248–253 "## Current Date & Time — Time zone: {tz}"
SYSTEM_PROMPT_CACHE_BOUNDARY src/agents/system-prompt-cache-boundary.ts — "--- SYSTEM PROMPT CACHE BOUNDARY ---" separator
buildRuntimeLine() src/agents/system-prompt.ts 1006–1048 "Runtime: agent=X | os=Y | node=Z | model=A | channel=B"
Provider overrides src/plugins/provider-runtime.js — transformProviderSystemPrompt() wraps final output
Context File Loading Priority
Files are sorted by CONTEXT_FILE_ORDER map then alphabetically. heartbeat.md is marked dynamic (below cache boundary).
agents.md → 10 soul.md → 20 identity.md → 30 user.md → 40 tools.md → 50 bootstrap.md → 60 memory.md → 70 heartbeat.md → dynamic
Bootstrap Prompt Files
.pi/prompts/cl.md — Changelog Audit
---
description: Audit changelog entries
---
Audit changelog entries for all commits
since the last release.

## Process
1. Find last release tag
2. List commits since that tag
3. Read each package's [Unreleased] section
4. Check: skip changelog updates,
   doc-only, release housekeeping
5. Cross-package duplication rule:
   → duplicate to coding-agent
6. Add New Features section
7. Report missing entries
.pi/prompts/is.md — Issue Analysis
---
description: Analyze GitHub issues
---
Analyze GitHub issue(s): $ARGUMENTS

1. Read the issue in full
2. For bugs:
   - Ignore root cause (likely wrong)
   - Read all related code files
   - Trace code path
   - Propose a fix
3. For features:
   - Read related files
   - Propose concise implementation
   - List affected files

Do NOT implement unless asked.
02
The Agent Run Attempt
From runEmbeddedAttempt() to the turn loop — all 3,290 lines of the core agent harness

runEmbeddedAttempt() — High-Level Flow

Mermaid
runEmbeddedAttempt() — Phase-by-Phase Breakdown
PhaseLinesKey FunctionsPurpose
1. Setup 579–610 resolveUserPath(), AbortController, mkdir workspace Resolve paths, create abort controller, ensure workspace exists
2. Model Resolution — resolveModelAsync(), normalizeResolvedModel() Resolve provider, model ID, API format (openai/anthropic/etc.)
3. Sandbox — resolveSandboxContext() Determine if running in Docker sandbox, mount paths
4. Session Manager 1284–1290 SessionManager.open(), guardSessionManager() Open/update session file, install tool result guards
5. System Prompt 1200–1201 buildSystemPromptOverride() → buildEmbeddedSystemPrompt() Assemble all 25+ prompt components listed in Section 01
6. Hook Runner 2258–2267 resolvePromptBuildHookResult() Fire llm_input hooks (async, fire-and-forget)
7. Tool Setup 1330–1413 splitSdkTools(), toClientToolDefinitions(), createAgentSession() Build tool allowlist, create session with tools
8. Stream Wrapping 1556–1866 10× wrapStreamFn*() wrappers Wrap streamFn with transforms (see Section 03)
9. History Prep 1869–1967 sanitizeSessionHistory(), limitHistoryTurns(), filterHeartbeatPairs() Prune, truncate, sanitize transcript before LLM call
10. Prompt Submit 2649–2702 activeSession.prompt() — THE CORE CALL Send to LLM; pi-agent-core handles turn loop internally
11. Compaction Wait 2750–2784 waitForCompactionRetryWithAggregateTimeout() Wait for background compaction (up to 60s aggregate)
12. Result Return — EmbeddedRunAttemptResult Return transcript, usage metrics, compaction count
Key Insight: The Turn Loop is Inside pi-agent-core
activeSession.prompt() (attempt.ts:2690–2696) delegates the turn loop to @mariozechner/pi-agent-core. That library's prompt() method:
  • Sends the current transcript (messages + system prompt) to the LLM API
  • Receives a streaming response
  • When the model emits tool_use blocks → executes tools sequentially
  • Appends each tool_result back to the transcript
  • Loops back to step 1 until stop_reason ≠ "tool_use"
  • Emits streamed text blocks back to OpenClaw via subscription callbacks
03
Stream Wrapping Chain
10 sequential wrappers that transform the raw LLM stream — applied in layers

Stream Wrapper Stack — Applied Bottom-Up

Mermaid
Stream Wrapper Detail
#Wrapper FunctionSourceWhat It Does
1resolveEmbeddedAgentStreamFn()stream-resolution.ts:64Selects provider transport: OpenAI WS, Anthropic Vertex, or pi streamSimple
2Yield Abort Gateattempt.ts:1778Intercepts "sessions_yield" abort signal; returns createYieldAbortedResponse()
3wrapStreamFnSanitizeMalformedToolCalls()attempt.ts:1791Fixes malformed tool call JSON in streaming response
4wrapStreamFnTrimToolCallNames()attempt.ts:1796Trims whitespace from tool names; guards against unknown-tool loop (threshold: 3)
5wrapStreamFnRepairMalformedToolCallArguments()attempt.ts:1810Repairs malformed JSON args in tool call input blocks
6wrapStreamFnDecodeXaiToolCallArguments()attempt.ts:1816Decodes HTML entities (e.g. <) in xAI tool call arguments
7createAnthropicPayloadLogger().wrapStreamFn()anthropic-payload-log.tsLogs full request/response payloads for debugging
8wrapStreamFnHandleSensitiveStopReason()attempt.ts:1829Catches provider-specific "sensitive" stop reasons before they bubble as errors
9streamWithIdleTimeout()llm-idle-timeout.tsAborts if model is silent for > idleTimeoutMs (configurable per trigger)
10wrapStreamFnWithDiagnosticModelCallEvents()attempt.ts:1853Emits diagnostic events per model API call (runId, sessionId, provider, model, api, transport)
Conditional wrappers (applied additionally when needed):
C1wrapStreamFnTextTransforms()plugin-text-transforms.tsProvider-specific input/output text transforms (lowercasing, etc.)
C2cacheTrace.wrapStreamFn()cache-trace.tsRecords prompt cache stages for observability
C3Google Prompt Cachegoogle-prompt-cache.tsUses Google's cached prompt feature if available
04
Turn Loop & Parallelism
How the loop handles sequential tool execution, concurrent compaction, and async hooks

Turn Loop — Tool Execution & Parallelism

Mermaid
✓ Parallel Activities
  • Compaction — runs in background while LLM is thinking. Uses compactWithSafetyTimeout() with configurable grace period (compaction-timeout.js). If it overruns, the run proceeds with pre-compaction state.
  • LLM Input Hooks — llm_input hooks fire asynchronously (fire-and-forget) before each LLM call (attempt.ts:2539–2566). No blocking.
  • Payload Logging — createAnthropicPayloadLogger records payloads without blocking the stream.
  • Context Engine — assembleAttemptContextEngine() can preprocess messages in parallel (attempt.ts:1923–1958).
  • Session Compaction Checkpoint — background snapshot saved during session init (compact.ts:839).
✗ Sequential Only
  • Tool calls within a turn — always sequential, order-dependent. Model emits tool_use blocks one at a time; each result is appended before the next LLM call.
  • History truncation — synchronous: limitHistoryTurns() must complete before activeSession.prompt() is called.
  • System prompt override — synchronous: applySystemPromptOverrideToSession() must complete before prompt().
  • Abort timer scheduling — synchronous: scheduleAbortTimer() sets up timeout before prompt starts.
  • Stream wrapper application — synchronous chain applied before any LLM I/O.
Compaction Flow

Compaction — Preemptive vs Background

Mermaid
Compaction Guard
After compaction, a CACHE_TTL custom entry is appended to the transcript to prevent double-compaction on the next turn. The guard in prepareCompaction() checks the last entry type before triggering again.
05
Harness Selection
How OpenClaw picks between the built-in pi harness and plugin harnesses (Codex, Claude Code, etc.)

Agent Harness Selection — Decision Tree

Mermaid
Harness Policy Resolution
Selection ReasonConditionHarness
pinnedagentHarnessId pinned via configpi or plugin (forced)
forced_piruntime="pi" in policypiHarness (embedded)
forced_pluginruntime="plugin-id" (specific)Named plugin harness
forced_plugin_fallback_to_piplugin not registered + fallback≠nonepiHarness
auto_pluginruntime="auto" + plugin supports providerHighest-priority supporting plugin
auto_pi_fallbackruntime="auto" + no plugin supportspiHarness (built-in)
pi-agent-core Imports
The core runtime imports two packages from @mariozechner:
@mariozechner/pi-agent-core @mariozechner/pi-coding-agent @mariozechner/pi-ai
These are the core abstractions: createAgentSession(), SessionManager, estimateTokens(), DefaultResourceLoader, and the StreamFn type.
06
Core Types & Tool Payloads
EmbeddedRunAttemptParams, tool call/request/result types, and session manager interfaces
EmbeddedRunAttemptParams — Key Fields
FieldTypeDescription
sessionIdstringUnique run session ID
sessionKeystringRouting key (user/session)
sessionFilestringDisk path to session JSON file
providerstring"anthropic" | "openai" | "openai-codex"...
modelIdstring"claude-sonnet-4-7-20250514"
modelProviderRuntimeModelResolved model with api, baseUrl, contextWindow
promptstringCurrent user message
thinkLevelThinkLevel"off" | "fast" | "medium" | "high"
reasoningLevelReasoningLevel"off" | "summary" | "block"
timeoutMsnumberRun timeout (default: from config)
configOpenClawConfigFull runtime config
workspaceDirstringAgent working directory
authStorageAuthStorageAPI key storage
toolsAllowstring[]Explicit tool allowlist
abortSignalAbortSignalExternal abort from gateway
Tool Result Payload Structure
// From run/payloads.ts
ToolUse:      { id, name, input: Record<string, unknown> }
ToolResult:   { toolUseId, content: string }
StopReason:   "tool_use" | "end_turn" | "max_tokens" | "stop_sequence" | ...

// Message flow per turn:
UserMessage → activeSession.prompt() →
  LLM → tool_use{id, "read", {path:"/foo.txt"}} →
  → ToolExecutor.execute("read", {path:"/foo.txt"}) →
  → ToolResult{toolUseId: id, content: "file contents..."} →
  → transcript.push(tool_result) →
  → LLM (next iteration)
07
Complete Architecture
End-to-end flow from user message to LLM API call to response

Full System Architecture

Mermaid
Key Files Referenced
FileRoleLines
src/agents/system-prompt.tsMaster system prompt builder — all 25+ sections1,048
src/agents/bootstrap-prompt.tsBootstrap prompt lines builder (full/limited)25
src/agents/context.tsMemory section and context file handling—
src/agents/pi-embedded-runner/run/attempt.tsrunEmbeddedAttempt — THE core harness3,290
src/agents/pi-embedded-runner/compact.tsBackground compaction orchestration1,287
src/agents/pi-embedded-runner/model.tsModel resolution + context window cache1,121
src/agents/pi-embedded-runner/stream-resolution.tsStream transport selection (WS/HTTP/Vertex)132
src/agents/harness/selection.tsHarness selection decision tree399
src/agents/pi-embedded-runner/run/payloads.tsTool call/request/result type definitions—
src/plugins/provider-runtime.jsProvider plugin hooks + system prompt transforms—
src/plugins/memory-state.jsMemory citations prompt section builder—
src/agents/bootstrap-budget.tsBootstrap warning prepend logic—
src/agents/session-write-lock.jsSession write lock for concurrent safety—
src/agents/session-compaction-checkpoints.jsCompaction checkpoint snapshot system—
.pi/prompts/cl.mdBootstrap skill: Changelog Audit58
.pi/prompts/is.mdBootstrap skill: Issue Analysis22