Simple Agent LLM Flow Example
A comprehensive demonstration of integrating LLM API calls within Yanshi’s async workflow system, showcasing how AI responses can drive intelligent routing and decision-making in agent workflows.
Overview
This example demonstrates the integration of Large Language Model (LLM) capabilities with Yanshi’s async execution framework. It shows how to:
- Create LLM-powered agent nodes using the moonshot demo pattern
- Process and analyze AI responses programmatically
- Make intelligent routing decisions based on response characteristics
- Manage complex conversational flows with shared state
- Combine deterministic logic with AI-driven decision making
Architecture
Flow Structure
The example implements a sophisticated 5-node workflow:
[User Input] → [Initial LLM] → [Response Processor] → [Decision Node] → [Specialized Nodes]
Node Types
- LLMAgentNode: Custom AsyncNode that makes LLM API calls
- ResponseProcessorNode: Analyzes LLM responses for sentiment and complexity
- DecisionNode: Routes flow based on response analysis
- Specialized Nodes: Different LLM strategies for various scenarios
Workflow Diagram
flowchart TD
A[User Input & Context] --> B[Initial LLM Node]
B --> C[Response Processor]
C --> D[Decision Node]
D --> E[Success Node<br/>Positive Sentiment]
D --> F[Retry Node<br/>Negative Sentiment]
D --> G[Detailed Node<br/>Long Response]
D --> H[Simple Node<br/>Short Response]
E --> I[Final Response]
F --> I
G --> I
H --> I
style B fill:#e1f5fe
style C fill:#f3e5f5
style D fill:#fff3e0
style E fill:#e8f5e8
style F fill:#ffebee
style G fill:#f1f8e9
style H fill:#fce4ec
Key Components
1. LLMAgentNode
The core LLM integration component that:
- Uses dynamic prompt templates with placeholder substitution
- Follows the moonshot demo pattern:
Yanshi::model("moonshot-v1-8k").prompt().execute() - Provides comprehensive error handling and logging
- Stores responses in shared state for downstream processing
Key Features:
#![allow(unused)]
fn main() {
// Dynamic prompt building
fn build_prompt(&self, shared_state: &SharedState) -> String {
let mut prompt = self.prompt_template.clone();
if let Some(Value::String(user_input)) = shared_state.get("user_input") {
prompt = prompt.replace("{user_input}", &user_input);
}
// ... more placeholder replacements
prompt
}
}
2. ResponseProcessorNode
Analyzes LLM responses with:
- Word Count Analysis: Measures response complexity
- Sentiment Detection: Basic positive/negative/neutral classification
- Structured Output: Creates analysis metadata for decision making
Analysis Logic:
#![allow(unused)]
fn main() {
let sentiment = if llm_response.contains("good") ||
llm_response.contains("great") ||
llm_response.contains("excellent") {
"positive"
} else if llm_response.contains("bad") ||
llm_response.contains("terrible") ||
llm_response.contains("awful") {
"negative"
} else {
"neutral"
};
}
3. DecisionNode
Implements intelligent routing based on:
- Sentiment-Based Routing:
- Positive → Success Node
- Negative → Retry Node
- Neutral → Length-based routing
- Complexity-Based Routing:
- Long responses (>10 words) → Detailed Node
- Short responses (≤10 words) → Simple Node
4. Specialized Terminal Nodes
Different LLM strategies for various scenarios:
- Success Node: Celebratory follow-up for positive responses
- Retry Node: Encouraging retry for negative responses
- Detailed Node: Summarization for complex responses
- Simple Node: Expansion for brief responses
Configuration
Prerequisites
- LLM Configuration: Ensure proper
models.ymlconfiguration - API Keys: Set up moonshot API credentials in environment or
.env - Network Access: Connectivity to moonshot LLM provider
Environment Setup
# Set API key (example)
export MOONSHOT_API_KEY="your-api-key-here"
# Or create .env file
echo "MOONSHOT_API_KEY=your-api-key-here" >> .env
Models Configuration
Ensure models.yml includes moonshot model configuration:
models:
moonshot-v1-8k:
vendor: "moonshot"
model_id: "moonshot-v1-8k"
temperature: 0.7
max_tokens: 1000
Running the Example
Basic Execution
cargo run --example simple_agent_llm_flow
With Debug Logging
RUST_LOG=debug cargo run --example simple_agent_llm_flow
Expected Output
🌟 Simple Agent Flow with LLM Integration Demo
This demonstrates how LLM (moonshot demo) can be invoked within an agent flow
🚀 Starting the agent flow...
🤖 [initial_llm] Preparing LLM request with prompt: You are a helpful assistant...
✅ [initial_llm] LLM configuration loaded successfully
🚀 [initial_llm] Executing LLM request to model: moonshot-v1-8k
✅ [initial_llm] LLM Response received: Deep Learning is a subset of machine learning...
💾 [initial_llm] Stored LLM response in shared state
🔍 [processor] Processing LLM response: Deep Learning is a subset of machine learning...
📊 [processor] Analysis complete: 45 words, neutral sentiment
📋 [processor] Analysis stored in shared state
🤔 [decision] Making decision based on sentiment: neutral, words: 45
✅ [decision] Decision made: route to detailed_node
🤖 [detailed] Preparing LLM request with prompt: The previous response was detailed...
✅ [detailed] LLM Response received: Deep Learning uses neural networks to learn patterns.
💾 [detailed] Stored LLM response in shared state
🎉 Flow completed successfully!
Final result: detailed_node
📋 Flow execution summary:
✅ Initial LLM node executed
Response: Deep Learning is a subset of machine learning that uses artificial neural networks...
✅ Response processor executed
Analysis: {"word_count":45,"sentiment":"neutral","original_response":"Deep Learning is..."}
✅ Decision node executed
Decision: "detailed_node"
✅ Final node 'detailed' executed
Final response: Deep Learning uses neural networks to learn patterns.
🏁 Demo completed!
Customization
1. Modify Prompts
Edit prompt templates in the node creation:
#![allow(unused)]
fn main() {
let llm_node = LLMAgentNode::new(
"initial_llm",
"You are an expert in {topic}. Please explain: {user_input}",
"moonshot-v1-8k"
);
}
2. Add New Analysis Criteria
Extend the ResponseProcessorNode:
#![allow(unused)]
fn main() {
// Add topic classification
let topic = if llm_response.contains("technology") {
"tech"
} else if llm_response.contains("science") {
"science"
} else {
"general"
};
}
3. Implement New Routing Logic
Modify the DecisionNode routing:
#![allow(unused)]
fn main() {
let decision = match (sentiment, word_count, topic) {
("positive", _, "tech") => "tech_success_node",
("negative", _, _) => "retry_node",
(_, count, _) if count > 50 => "summary_node",
_ => "default_node"
};
}
4. Add More LLM Models
Use different models for different nodes:
#![allow(unused)]
fn main() {
let analysis_node = LLMAgentNode::new(
"analyzer",
"Analyze the sentiment of: {response}",
"gpt-4" // Different model
);
}
Advanced Usage
1. Parallel Processing
Create parallel analysis nodes:
#![allow(unused)]
fn main() {
let sentiment_node = LLMAgentNode::new("sentiment", "...", "model1");
let topic_node = LLMAgentNode::new("topic", "...", "model2");
let parallel_flow = AsyncFlow::new_parallel(vec![
Box::new(sentiment_node),
Box::new(topic_node)
]);
}
2. Error Recovery
Implement retry logic in nodes:
#![allow(unused)]
fn main() {
// In exec_async
let mut attempts = 0;
loop {
match LLMYanshi::model(model).prompt(prompt).execute().await {
Ok(response) => return Ok(response),
Err(e) if attempts < 3 => {
attempts += 1;
continue;
},
Err(e) => return Err(e)
}
}
}
3. Streaming Responses
Use streaming for real-time processing:
#![allow(unused)]
fn main() {
let stream = LLMYanshi::model(model)
.prompt(prompt)
.execute_streaming().await?;
// Process stream chunks
while let Some(chunk) = stream.next().await {
// Handle streaming response
}
}
Performance Considerations
1. Concurrent LLM Calls
- Use
AsyncFlow::new_parallel()for independent LLM operations - Implement rate limiting for API compliance
- Consider caching responses for repeated queries
2. Memory Management
- Clear large responses from shared state when no longer needed
- Use streaming for long responses
- Implement response size limits
3. Error Handling
- Implement exponential backoff for API failures
- Use circuit breaker patterns for unreliable endpoints
- Provide fallback responses for critical paths
Troubleshooting
Common Issues
-
API Key Not Found
Error: LLM initialization failed: API key not configuredSolution: Set environment variable or .env file
-
Model Not Available
Error: Model 'moonshot-v1-8k' not found in configurationSolution: Update models.yml with correct model configuration
-
Network Timeouts
Error: Request timeoutSolution: Increase timeout or check network connectivity
Debug Tips
-
Enable Detailed Logging:
RUST_LOG=yanshi_llm=debug cargo run --example simple_agent_llm_flow -
Inspect Shared State:
#![allow(unused)] fn main() { println!("Shared state: {:?}", shared.debug_dump()); } -
Add Timing Information:
#![allow(unused)] fn main() { let start = std::time::Instant::now(); // ... LLM call println!("LLM call took: {:?}", start.elapsed()); }
Related Examples
moonshot_demo.rs- Basic LLM integration patternworkflow.rs- Sequential multi-stage workflowsbatch_processing.rs- Parallel processing patternschat.rs- Interactive conversation flows
Next Steps
- Extend Analysis: Add more sophisticated NLP analysis
- Multi-Model: Use different LLMs for different tasks
- Persistence: Store conversation history and context
- Integration: Connect with external APIs and databases
- Monitoring: Add comprehensive metrics and alerting
This example serves as a foundation for building sophisticated AI-powered agent workflows that combine the reliability of structured flows with the flexibility of LLM-driven decision making.