Deep Analysis

OpenClaw Prompt Construction & Agent Loop

How a user message flows through string assembly, the pi-agent-core harness, stream wrapping, tool execution, and compaction — with full lineage tracking.

github.com/openclaw/openclaw ~3,290 lines in run/attempt.ts 14 source files analyzed
01
System Prompt Construction
All 25+ string components that assemble into the final system prompt, with exact file lineage

System Prompt Assembly — Flow Diagram

Mermaid
Component Lineage Table — 25 String Components
Component / Function Source File Lines Description
buildAgentSystemPrompt() src/agents/system-prompt.ts 439–1048 Master builder — orchestrates all sections in order
buildFullBootstrapPromptLines() src/agents/bootstrap-prompt.ts 1–13 Full bootstrap: read BOOTSTRAP.md, complete workflow
buildLimitedBootstrapPromptLines() src/agents/bootstrap-prompt.ts 15–25 Limited: explain blocker, suggest next step
BOOTSTRAP.md content .pi/prompts/cl.md 58 lines Changelog audit skill — audit entries before release
BOOTSTRAP.md content .pi/prompts/is.md 22 lines Issue analysis skill — audit GitHub issues
buildBootstrapPromptWarning() src/agents/bootstrap-budget.ts Prepends "⚠️ Bootstrap budget..." warning if near limit
buildSkillsSection() src/agents/system-prompt.ts 156–172 Skills instructions: "scan available_skills..."
buildMemorySection() src/agents/system-prompt.ts 174–187 Memory citations guidance via plugins/memory-state.ts
buildDocsSection() src/agents/system-prompt.ts 395–425 Docs links: docs.openclaw.ai, GitHub, Discord, ClawhHub
buildMessagingSection() src/agents/system-prompt.ts 335–382 sessions_spawn, sessions_send, subagents guidance
buildVoiceSection() src/agents/system-prompt.ts 384–393 TTS hint via buildTtsSystemPromptHint()
buildHeartbeatSection() src/agents/system-prompt.ts 127–138 "If heartbeat poll: reply HEARTBEAT_OK"
buildExecutionBiasSection() src/agents/system-prompt.ts 299–313 "Actionable request: act in this turn..."
buildToolCallStyleSection() src/agents/system-prompt.ts 747–765 Default: don't narrate routine tool calls
Safety Constitution src/agents/system-prompt.ts 672–678 No independent goals, prioritize safety, no manipulation
CLI Quick Reference src/agents/system-prompt.ts 777–788 openclaw gateway status/restart, config.schema.lookup
Self-Update Rules src/agents/system-prompt.ts 793–803 Updates only on explicit user request
Silent Reply Rules src/agents/system-prompt.ts 955–970 SILENT_REPLY_TOKEN when nothing to say
buildProjectContextSection() src/agents/system-prompt.ts 95–125 Loads workspace files: agents.md, soul.md, identity.md...
CONTEXT_FILE_ORDER map src/agents/system-prompt.ts 44–52 File priority: agents.md(10) → soul.md(20) → ... → memory.md(70)
buildWebchatCanvasSection() src/agents/system-prompt.ts 275–297 [embed...] directives for webchat only
buildOwnerIdentityLine() src/agents/system-prompt.ts 232–246 "Authorized senders: [hash or raw numbers]"
buildTimeSection() src/agents/system-prompt.ts 248–253 "## Current Date & Time — Time zone: {tz}"
SYSTEM_PROMPT_CACHE_BOUNDARY src/agents/system-prompt-cache-boundary.ts "--- SYSTEM PROMPT CACHE BOUNDARY ---" separator
buildRuntimeLine() src/agents/system-prompt.ts 1006–1048 "Runtime: agent=X | os=Y | node=Z | model=A | channel=B"
Provider overrides src/plugins/provider-runtime.js transformProviderSystemPrompt() wraps final output
Context File Loading Priority
Files are sorted by CONTEXT_FILE_ORDER map then alphabetically. heartbeat.md is marked dynamic (below cache boundary).
agents.md → 10 soul.md → 20 identity.md → 30 user.md → 40 tools.md → 50 bootstrap.md → 60 memory.md → 70 heartbeat.md → dynamic
Bootstrap Prompt Files
.pi/prompts/cl.md — Changelog Audit
---
description: Audit changelog entries
---
Audit changelog entries for all commits
since the last release.

## Process
1. Find last release tag
2. List commits since that tag
3. Read each package's [Unreleased] section
4. Check: skip changelog updates,
   doc-only, release housekeeping
5. Cross-package duplication rule:
   → duplicate to coding-agent
6. Add New Features section
7. Report missing entries
.pi/prompts/is.md — Issue Analysis
---
description: Analyze GitHub issues
---
Analyze GitHub issue(s): $ARGUMENTS

1. Read the issue in full
2. For bugs:
   - Ignore root cause (likely wrong)
   - Read all related code files
   - Trace code path
   - Propose a fix
3. For features:
   - Read related files
   - Propose concise implementation
   - List affected files

Do NOT implement unless asked.
02
The Agent Run Attempt
From runEmbeddedAttempt() to the turn loop — all 3,290 lines of the core agent harness

runEmbeddedAttempt() — High-Level Flow

Mermaid
runEmbeddedAttempt() — Phase-by-Phase Breakdown
PhaseLinesKey FunctionsPurpose
1. Setup 579–610 resolveUserPath(), AbortController, mkdir workspace Resolve paths, create abort controller, ensure workspace exists
2. Model Resolution resolveModelAsync(), normalizeResolvedModel() Resolve provider, model ID, API format (openai/anthropic/etc.)
3. Sandbox resolveSandboxContext() Determine if running in Docker sandbox, mount paths
4. Session Manager 1284–1290 SessionManager.open(), guardSessionManager() Open/update session file, install tool result guards
5. System Prompt 1200–1201 buildSystemPromptOverride() → buildEmbeddedSystemPrompt() Assemble all 25+ prompt components listed in Section 01
6. Hook Runner 2258–2267 resolvePromptBuildHookResult() Fire llm_input hooks (async, fire-and-forget)
7. Tool Setup 1330–1413 splitSdkTools(), toClientToolDefinitions(), createAgentSession() Build tool allowlist, create session with tools
8. Stream Wrapping 1556–1866 10× wrapStreamFn*() wrappers Wrap streamFn with transforms (see Section 03)
9. History Prep 1869–1967 sanitizeSessionHistory(), limitHistoryTurns(), filterHeartbeatPairs() Prune, truncate, sanitize transcript before LLM call
10. Prompt Submit 2649–2702 activeSession.prompt() — THE CORE CALL Send to LLM; pi-agent-core handles turn loop internally
11. Compaction Wait 2750–2784 waitForCompactionRetryWithAggregateTimeout() Wait for background compaction (up to 60s aggregate)
12. Result Return EmbeddedRunAttemptResult Return transcript, usage metrics, compaction count
Key Insight: The Turn Loop is Inside pi-agent-core
activeSession.prompt() (attempt.ts:2690–2696) delegates the turn loop to @mariozechner/pi-agent-core. That library's prompt() method:
  • Sends the current transcript (messages + system prompt) to the LLM API
  • Receives a streaming response
  • When the model emits tool_use blocks → executes tools sequentially
  • Appends each tool_result back to the transcript
  • Loops back to step 1 until stop_reason ≠ "tool_use"
  • Emits streamed text blocks back to OpenClaw via subscription callbacks
03
Stream Wrapping Chain
10 sequential wrappers that transform the raw LLM stream — applied in layers

Stream Wrapper Stack — Applied Bottom-Up

Mermaid
Stream Wrapper Detail
#Wrapper FunctionSourceWhat It Does
1resolveEmbeddedAgentStreamFn()stream-resolution.ts:64Selects provider transport: OpenAI WS, Anthropic Vertex, or pi streamSimple
2Yield Abort Gateattempt.ts:1778Intercepts "sessions_yield" abort signal; returns createYieldAbortedResponse()
3wrapStreamFnSanitizeMalformedToolCalls()attempt.ts:1791Fixes malformed tool call JSON in streaming response
4wrapStreamFnTrimToolCallNames()attempt.ts:1796Trims whitespace from tool names; guards against unknown-tool loop (threshold: 3)
5wrapStreamFnRepairMalformedToolCallArguments()attempt.ts:1810Repairs malformed JSON args in tool call input blocks
6wrapStreamFnDecodeXaiToolCallArguments()attempt.ts:1816Decodes HTML entities (e.g. <) in xAI tool call arguments
7createAnthropicPayloadLogger().wrapStreamFn()anthropic-payload-log.tsLogs full request/response payloads for debugging
8wrapStreamFnHandleSensitiveStopReason()attempt.ts:1829Catches provider-specific "sensitive" stop reasons before they bubble as errors
9streamWithIdleTimeout()llm-idle-timeout.tsAborts if model is silent for > idleTimeoutMs (configurable per trigger)
10wrapStreamFnWithDiagnosticModelCallEvents()attempt.ts:1853Emits diagnostic events per model API call (runId, sessionId, provider, model, api, transport)
Conditional wrappers (applied additionally when needed):
C1wrapStreamFnTextTransforms()plugin-text-transforms.tsProvider-specific input/output text transforms (lowercasing, etc.)
C2cacheTrace.wrapStreamFn()cache-trace.tsRecords prompt cache stages for observability
C3Google Prompt Cachegoogle-prompt-cache.tsUses Google's cached prompt feature if available
04
Turn Loop & Parallelism
How the loop handles sequential tool execution, concurrent compaction, and async hooks

Turn Loop — Tool Execution & Parallelism

Mermaid
✓ Parallel Activities
  • Compaction — runs in background while LLM is thinking. Uses compactWithSafetyTimeout() with configurable grace period (compaction-timeout.js). If it overruns, the run proceeds with pre-compaction state.
  • LLM Input Hooksllm_input hooks fire asynchronously (fire-and-forget) before each LLM call (attempt.ts:2539–2566). No blocking.
  • Payload LoggingcreateAnthropicPayloadLogger records payloads without blocking the stream.
  • Context EngineassembleAttemptContextEngine() can preprocess messages in parallel (attempt.ts:1923–1958).
  • Session Compaction Checkpoint — background snapshot saved during session init (compact.ts:839).
✗ Sequential Only
  • Tool calls within a turn — always sequential, order-dependent. Model emits tool_use blocks one at a time; each result is appended before the next LLM call.
  • History truncation — synchronous: limitHistoryTurns() must complete before activeSession.prompt() is called.
  • System prompt override — synchronous: applySystemPromptOverrideToSession() must complete before prompt().
  • Abort timer scheduling — synchronous: scheduleAbortTimer() sets up timeout before prompt starts.
  • Stream wrapper application — synchronous chain applied before any LLM I/O.
Compaction Flow

Compaction — Preemptive vs Background

Mermaid
Compaction Guard
After compaction, a CACHE_TTL custom entry is appended to the transcript to prevent double-compaction on the next turn. The guard in prepareCompaction() checks the last entry type before triggering again.
05
Harness Selection
How OpenClaw picks between the built-in pi harness and plugin harnesses (Codex, Claude Code, etc.)

Agent Harness Selection — Decision Tree

Mermaid
Harness Policy Resolution
Selection ReasonConditionHarness
pinnedagentHarnessId pinned via configpi or plugin (forced)
forced_piruntime="pi" in policypiHarness (embedded)
forced_pluginruntime="plugin-id" (specific)Named plugin harness
forced_plugin_fallback_to_piplugin not registered + fallback≠nonepiHarness
auto_pluginruntime="auto" + plugin supports providerHighest-priority supporting plugin
auto_pi_fallbackruntime="auto" + no plugin supportspiHarness (built-in)
pi-agent-core Imports
The core runtime imports two packages from @mariozechner:
@mariozechner/pi-agent-core @mariozechner/pi-coding-agent @mariozechner/pi-ai
These are the core abstractions: createAgentSession(), SessionManager, estimateTokens(), DefaultResourceLoader, and the StreamFn type.
06
Core Types & Tool Payloads
EmbeddedRunAttemptParams, tool call/request/result types, and session manager interfaces
EmbeddedRunAttemptParams — Key Fields
FieldTypeDescription
sessionIdstringUnique run session ID
sessionKeystringRouting key (user/session)
sessionFilestringDisk path to session JSON file
providerstring"anthropic" | "openai" | "openai-codex"...
modelIdstring"claude-sonnet-4-7-20250514"
modelProviderRuntimeModelResolved model with api, baseUrl, contextWindow
promptstringCurrent user message
thinkLevelThinkLevel"off" | "fast" | "medium" | "high"
reasoningLevelReasoningLevel"off" | "summary" | "block"
timeoutMsnumberRun timeout (default: from config)
configOpenClawConfigFull runtime config
workspaceDirstringAgent working directory
authStorageAuthStorageAPI key storage
toolsAllowstring[]Explicit tool allowlist
abortSignalAbortSignalExternal abort from gateway
Tool Result Payload Structure
// From run/payloads.ts
ToolUse:      { id, name, input: Record<string, unknown> }
ToolResult:   { toolUseId, content: string }
StopReason:   "tool_use" | "end_turn" | "max_tokens" | "stop_sequence" | ...

// Message flow per turn:
UserMessage → activeSession.prompt() →
  LLM → tool_use{id, "read", {path:"/foo.txt"}} →
  → ToolExecutor.execute("read", {path:"/foo.txt"}) →
  → ToolResult{toolUseId: id, content: "file contents..."} →
  → transcript.push(tool_result) →
  → LLM (next iteration)
07
Complete Architecture
End-to-end flow from user message to LLM API call to response

Full System Architecture

Mermaid
Key Files Referenced
FileRoleLines
src/agents/system-prompt.tsMaster system prompt builder — all 25+ sections1,048
src/agents/bootstrap-prompt.tsBootstrap prompt lines builder (full/limited)25
src/agents/context.tsMemory section and context file handling
src/agents/pi-embedded-runner/run/attempt.tsrunEmbeddedAttempt — THE core harness3,290
src/agents/pi-embedded-runner/compact.tsBackground compaction orchestration1,287
src/agents/pi-embedded-runner/model.tsModel resolution + context window cache1,121
src/agents/pi-embedded-runner/stream-resolution.tsStream transport selection (WS/HTTP/Vertex)132
src/agents/harness/selection.tsHarness selection decision tree399
src/agents/pi-embedded-runner/run/payloads.tsTool call/request/result type definitions
src/plugins/provider-runtime.jsProvider plugin hooks + system prompt transforms
src/plugins/memory-state.jsMemory citations prompt section builder
src/agents/bootstrap-budget.tsBootstrap warning prepend logic
src/agents/session-write-lock.jsSession write lock for concurrent safety
src/agents/session-compaction-checkpoints.jsCompaction checkpoint snapshot system
.pi/prompts/cl.mdBootstrap skill: Changelog Audit58
.pi/prompts/is.mdBootstrap skill: Issue Analysis22