Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Agent SDK

This guide is for developers who want to extend Yanshi’s agent runtime — plug in a custom planning loop, a custom reflection strategy, a custom memory-summary backend, a custom tool, or a custom memory store — without forking the workspace.

If you only want to run built-in agents (ReAct, Plan-Execute, Multi-Agent supervisors), read AGENT_RUNTIME.md and MULTI_AGENT.md first.

Goals

  • 30 minutes from “I cloned the repo” to “my custom reflection strategy runs inside ReActAgent”.
  • Every extension point is a Rust trait with a stable contract and at least one runnable example.
  • The SDK does not require external API access; every example uses the mock LLM provider or runs without a model at all.

Five-minute quickstart

The fastest possible warm-up is a custom reflection strategy. The strategy is a single trait with two methods, and the runtime accepts any Arc<dyn ReflectionStrategy>.

# 1. Build and run the canonical example.
cargo run -p yanshi-agents --example custom_reflection

Expected output (truncated):

Answer: echo: sdk
Stop reason: FinalAnswer
Triggers observed by LoggingReflection: [Final]
--- Steps ---
   0: observe: Echo the word sdk.
   1: plan: call echo
   2: tool_call: echo
   3: tool_result: echo -> echo: sdk
   4: plan: echo returned the expected text
   5: final: echo: sdk
   6: reflect: [logging] final answer at step 6: echo: sdk

Read yanshi-agents/examples/custom_reflection.rs end-to-end. The whole example is ~120 lines and exercises:

  • The ReflectionStrategy trait (name, reflect).
  • Wiring through ReActAgent::with_reflection_strategy.
  • The mock LLM provider (no API key needed).
  • Structured AgentStep output, including the inserted Reflect step.

Extension points

TraitCratePurposeExample
AgentRuntimeyanshi-agentsReplace the entire planning / execution loopcustom_runtime.rs
ReflectionStrategyyanshi-agentsInject reflection text into the step tracecustom_reflection.rs
VerificationStrategyyanshi-agentsGate a candidate final answer; reject to loop againcustom_verification.rs
MemorySummaryBackendyanshi-agents::reactCompress prompt memory when it overflows the budgetcustom_memory_summary.rs
AgentMemoryHookyanshi-agentsNon-failing observer for memory reads / writes(in tests, see react/agent.rs)
Toolyanshi-toolsAdd a new tool callable by any agentagent_native_react.rs (EchoTool)
MemoryStoreyanshi-memoryAdd a new conversation memory backendSessionMemory / SqliteMemory impls

Each trait below has the same anatomy: what it does, the contract, how to plug it in, and gotchas.

AgentRuntime

#![allow(unused)]
fn main() {
#[async_trait]
pub trait AgentRuntime: Send {
  async fn run(&mut self, context: AgentContext) -> Result<AgentRunResult, AgentRuntimeError>;
  fn runtime_name(&self) -> &'static str;
}
}

What it does. Owns one full agent invocation. The runtime consumes an AgentContext (session id, input, model, persona, limits, cancellation token) and returns a structured AgentRunResult (answer, stop reason, step trace, event stream).

Contract. Implementations MUST:

  1. Honour RuntimeLimits (max_steps, max_tool_calls, timeout_ms, token_budget); whichever is hit first stops the loop with the matching AgentStopReason. cost_limit_usd (T1.1) is a fifth, optional bound — ReActAgent/PlanExecuteAgent honour it against a configurable per-model PricingTable, but a custom runtime is not required to track cost; if it doesn’t, the field is simply inert for that runtime (same as leaving it None).
  2. Honour context.cancellation_token if present (poll AgentCancellationToken::is_cancelled between steps; stop with AgentStopReason::Cancelled).
  3. Emit AgentSteps with monotonically increasing indices, in chronological order, so trace replay and event listeners stay coherent.
  4. Use AgentRuntimeError::InvalidContext for pre-flight validation problems and AgentRuntimeError::ExecutionFailed for anything raised after the loop starts. Both error variants short-circuit the run (no AgentRunResult is produced).

Wiring. Anywhere you would call a built-in runtime, call yours:

#![allow(unused)]
fn main() {
let mut runtime = MyRuntime::new(...);
let result = AgentRuntime::run(&mut runtime, AgentContext::new(...)).await?;
}

AgentNode (DAG embedding) and the multi-agent supervisors take any AgentRuntime, so a custom runtime composes into hybrid workflows for free.

Gotchas.

  • AgentStepKind is closed — you cannot add a new variant out-of-tree. If you need a step kind that does not exist (e.g. a domain-specific Critique step), open an issue first; reusing Plan / Reflect / ToolResult is usually the right answer.
  • The runtime is responsible for writing user input and assistant responses to memory. If you skip that, multi-turn conversation breaks silently.
  • Long-running steps should still poll cancellation; otherwise users see unresponsive runs even after cancel().

See custom_runtime.rs for the smallest viable shell — no LLM, no tools, just the trait contract.

ReflectionStrategy

#![allow(unused)]
fn main() {
#[async_trait]
pub trait ReflectionStrategy: Send + Sync {
  fn name(&self) -> &'static str;
  async fn reflect(
    &self,
    context: &ReflectionContext,
  ) -> Result<Option<Reflection>, ReflectionError>;
}
}

What it does. Inserts a Reflect step into the trace at well-defined points (Step / Failure / Final) without changing the planning loop.

Contract.

  • name() is stable, lowercase, snake_case (used in the persisted step record and trace UI).
  • Filter on context.trigger first; return Ok(None) for triggers you do not handle. The runtime handles None by skipping the reflection step entirely.
  • Treat your own failures as soft: prefer Ok(None) over Err. The runtime will continue without inserting a reflection if you return Ok(None). Only return Err(ReflectionError::Failed) when the caller can act on it.
  • Reflections run inline with the loop; don’t call slow blocking APIs without an explicit timeout.

Wiring.

#![allow(unused)]
fn main() {
let agent = ReActAgent::new(config, memory, tools)
  .with_reflection_strategy(Arc::new(MyStrategy::new()));
}

Built-ins: NoOpReflection, FailureReflection, FinalReflection (in yanshi_agents::reflection).

VerificationStrategy

#![allow(unused)]
fn main() {
#[async_trait]
pub trait VerificationStrategy: Send + Sync {
  fn name(&self) -> &'static str;
  async fn verify(
    &self,
    context: &VerificationContext,
  ) -> Result<VerificationOutcome, VerificationError>;
}
}

What it does. Gates a candidate final answer before the loop stops. Unlike ReflectionStrategy, its verdict changes control flow: VerificationOutcome::Rejected { feedback } sends ReActAgent back around for another attempt instead of terminating with AgentStopReason::FinalAnswer.

Contract.

  • name() is stable, lowercase, kebab- or snake_case (used in log lines and as your strategy’s identity in the trace).
  • VerificationOutcome::Approved accepts the candidate answer; the run stops as normal.
  • VerificationOutcome::Rejected { feedback } rejects it. feedback is appended to memory as the next observation (the same mechanism tool results use), so phrase it as you would a tool result the model should react to.
  • Treat your own failures as soft: return Err(VerificationError::Failed) only when genuinely actionable. The runtime treats a strategy error as non-fatal and accepts the candidate answer rather than getting the run stuck.
  • Verification runs inline with the loop and gates every candidate answer; don’t call slow blocking APIs without an explicit timeout.
  • Rejections are bounded by ReActConfig::max_verification_attempts (default 2, set via with_max_verification_attempts(...)). Exhausting the bound force-accepts the candidate answer rather than looping forever or erroring — a strategy that never approves degrades gracefully instead of hanging the run.

Wiring.

#![allow(unused)]
fn main() {
let agent = ReActAgent::new(config, memory, tools)
  .with_verification_strategy(Arc::new(MyStrategy::new()));
}

Disable the gate at runtime with ReActConfig::with_verification_enabled(false) — no Verify step or VerificationCompleted event is recorded even with a strategy attached.

Built-in: AlwaysApprove (in yanshi_agents::verification) — useful as a base to wrap with real domain logic (an LLM-judge call, a test-runner invocation, a schema check, …).

See custom_verification.rs for a strategy that rejects a candidate answer missing a citation marker and approves the revised one.

MemorySummaryBackend

#![allow(unused)]
fn main() {
#[async_trait]
pub trait MemorySummaryBackend: Send + Sync {
  fn name(&self) -> &'static str;
  async fn summarize(&self, context: MemorySummaryContext) -> Result<Option<String>, ReActError>;
}
}

What it does. Compresses the older slice of a session’s memory into a single summary string when the prompt-memory token estimate exceeds the configured memory_prompt_token_budget.

Contract.

  • Triggered only when ReActConfig::memory_prompt_token_budget is set, memory_summary_strategy != Disabled, and the running token estimate exceeds the budget.
  • Ok(Some(summary)) injects summary as a synthetic system message ahead of the kept messages.
  • Ok(None) skips the summary (the loop silently truncates).
  • Err(ReActError::MemorySummary { .. }) aborts the run; reserve it for genuine failures.
  • Backends should stay deterministic where possible; LLM-backed backends should set a tight timeout to avoid stalling the loop.

Wiring.

#![allow(unused)]
fn main() {
let agent = ReActAgent::new(config, memory, tools)
  .with_memory_summary_backend(Arc::new(MyBackend::new()));
}

Built-ins: RecentOnlyMemorySummary (records what was dropped), CompactMemorySummary (deterministic rule-based bullet list).

See custom_memory_summary.rs for both direct invocation (drives the trait without a full run) and the production wiring.

AgentMemoryHook

#![allow(unused)]
fn main() {
pub trait AgentMemoryHook: Send + Sync {
  fn on_memory_read(&self, _context: &MemoryHookContext) {}
  fn on_memory_write(&self, _context: &MemoryHookContext) {}
}
}

What it does. Non-failing observer for memory operations. Use it to record metrics, build secondary summaries, or fan out to a search index. Hooks intentionally cannot fail; if your hook needs to surface errors, log them instead.

Wiring.

#![allow(unused)]
fn main() {
let agent = ReActAgent::new(config, memory, tools)
  .with_memory_hook(Arc::new(MyHook::new()));
}

Tool

#![allow(unused)]
fn main() {
#[async_trait]
pub trait Tool: Send + Sync {
  fn name(&self) -> &str;
  fn description(&self) -> &str;
  fn parameters_schema(&self) -> Value;
  fn metadata(&self) -> ToolMetadata { ... }
  fn idempotency(&self, params: &Value) -> ToolIdempotency { ... }
  fn requires_capabilities(&self) -> Vec<Capability> { ... }
  async fn execute(&self, params: Value) -> Result<ToolOutput, ToolError>;
}
}

What it does. Defines a callable tool surfaced to any agent through ToolRegistry. The same trait powers built-in tools (FileTool, HttpTool, ShellTool), MCP-bridged tools, and workflow-as-tool (WorkflowTool).

Contract.

  • name() must be globally unique inside one registry.
  • parameters_schema() is JSON Schema; the LLM-facing prompt serialises it directly.
  • metadata() declares ToolSource, ToolPermissionSet, and ToolIdempotency. Set permissions accurately — the policy layer and OS sandbox both consult them.
  • idempotency(params) overrides metadata when replay safety depends on parameters (e.g. http GET vs. http POST).
  • execute() should return ToolOutput::error(...) for tool-level failures (the agent loop handles it) and Err(ToolError) only for protocol-level failures (the run aborts).

See agent_native_react.rs (EchoTool) and the built-in implementations under yanshi-tools/src/builtin/.

MemoryStore

#![allow(unused)]
fn main() {
#[async_trait]
pub trait MemoryStore: Send + Sync {
  async fn add_message(&mut self, message: Message) -> Result<(), MemoryError>;
  async fn get_history(&self, session_id: &str, limit: usize) -> Result<Vec<Message>, MemoryError>;
  async fn get_all(&self, session_id: &str) -> Result<Vec<Message>, MemoryError>;
  async fn search(&self, session_id: &str, query: &str, limit: usize) -> Result<Vec<Message>, MemoryError>;
  async fn clear_session(&mut self, session_id: &str) -> Result<(), MemoryError>;
  async fn session_token_count(&self, session_id: &str) -> Result<u32, MemoryError>;
  async fn to_prompt(&self, session_id: &str) -> Result<String, MemoryError> { ... }
}
}

What it does. Pluggable conversation memory backend. Built-ins: SessionMemory (in-process, token-windowed), SqliteMemory (persistent), SemanticMemory (similarity search via yanshi-rag).

Contract.

  • get_history(session_id, limit) returns the most recent limit messages oldest-first (callers depend on chronological order).
  • add_message is the only mutator; runtimes never reach in to rewrite history. If your backend supports compaction, do it transparently.
  • session_token_count should be cheap; it is polled inside the RuntimeLimits::token_budget guard.
  • All methods are async; use whatever runtime you like internally (Tokio is canonical).

Closed enums and stability

These enums are deliberately closed; do not extend them out-of-tree:

  • AgentStepKind (runtime step variants).
  • AgentStopReason (run termination reasons).
  • ReflectionTrigger (Step / Failure / Final).
  • ToolPermission, Capability, ToolSource (security boundary).

If a closed enum is genuinely missing a variant for your use case, open an issue describing the use case before forking. Keeping these closed keeps trace replay, sandbox enforcement, and multi-agent supervisors honest.

Testing without an API key

Two patterns:

  1. No model: implement AgentRuntime directly. See custom_runtime.rs.
  2. Mock provider: register the mock provider and pre-load it via YANSHI_MOCK_RESPONSES (a JSON-encoded list of canned replies). See agent_native_react.rs, custom_reflection.rs, and custom_verification.rs for the standard wiring.

FlowValue field reference (F-DOC-2)

yanshi_core::value::FlowValue is the data wrapper passed between nodes. Custom AsyncNode implementations that produce file / URL outputs must use the exact field names below — they won’t autocomplete and media_type is a common wrong guess.

VariantFieldsExample
Json(Value)tuple-style, one serde_json::ValueFlowValue::Json(json!({"answer": 42}))
File { path, mime_type }path: PathBuf, mime_type: Option<String>FlowValue::File { path: PathBuf::from("/tmp/x.png"), mime_type: Some("image/png".into()) }
Url { url, mime_type }url: String, mime_type: Option<String>FlowValue::Url { url: "https://...".into(), mime_type: None }

Field name is mime_type not media_type (matches IANA “media type” vernacular as mime_type only because that was the original field name; renaming would be a breaking change, kept). Both File and Url carry the same optional MIME hint so downstream nodes can dispatch on type without re-sniffing.

The Serialize impl tags each variant with an explicit type field ("json" / "file" / "url") so trace JSON is self-describing.

Loading ~/.yanshi/.env from standalone binaries (F-A7-7)

The yanshi CLI auto-loads ~/.yanshi/.env on every invocation (P9.3). Standalone binaries that link yanshi-llm directly — e.g. examples/applications/blog-to-podcast/, code-reviewer-write, research-assistant — don’t go through the CLI entry point and need to load the env themselves before Yanshi::init().

Canonical snippet (used by every standalone example in this repo):

fn load_yanshi_dotenv() {
  if let Some(home) = std::env::home_dir() {
    let _ = dotenvy::from_path(home.join(".yanshi").join(".env"));
  }
}

#[tokio::main]
async fn main() -> Result<()> {
  load_yanshi_dotenv();
  Yanshi::init().await?;
  // ... rest of the binary
}

dotenvy::from_path is best-effort by design — missing file returns Err which the snippet swallows. That’s correct behaviour for an optional env file; production deployments that require specific keys should let Yanshi::init() surface its own “missing API key” error rather than gating on dotenv presence.

A future yanshi-dotenv helper crate could DRY this snippet across the example binaries; for now the duplication is acknowledged ergonomics-debt (F-A7-7) and the snippet above is the canonical form to copy.

  • AGENT_RUNTIME.md — runtime boundary, ReAct loop, hybrid composition with DAG.
  • MULTI_AGENT.md — Handoff / Blackboard / Debate supervisors and their step kinds.
  • TOOL_PERMISSIONS.md — three-way capability merge between tool, skill, and CLI flag.
  • TRACING_DESIGN.md — how AgentEvents are persisted and how OTel spans are produced.

Documentation hygiene

cargo doc -p yanshi-agents -p yanshi-tools -p yanshi-memory --no-deps must report zero warnings. The SDK extension surface is the contract this guide describes; if rustdoc breaks, this guide is broken.