Intermediate
Building Conversational Agents with System Prompts
Building Conversational Agents with System Prompts
Introduction
A single-turn prompt is easy: user asks something, model responds, done. But real applications are conversations. Users ask follow-up questions. They change their mind. They go off topic. They ask tricky questions that test your constraints.
Multi-turn conversations are where system prompts earn their value. The system prompt must guide the model through complex conversational contexts, manage memory of previous statements, handle ambiguity, and know when to ask clarifying questions versus when to move forward.
This lesson teaches you how to design system prompts that enable effective conversations instead of frustrating ones.
Key Takeaway: In conversations, consistency matters more than perfection on any single response. Your system prompt must ensure the model stays in character, remembers context, and gracefully handles the mess of real dialogue.
Multi-Turn Conversation Design Principles
Principle 1: Explicit Conversation State
The model can’t truly “remember” across turns, so you must make context explicit:
def build_conversation_system_prompt(context: dict) -> str:
"""Build a system prompt that tracks conversation state"""
return f"""You are a patient support specialist helping {context['customer_name']}
with their account issue.
CONVERSATION STATE:
- Customer Name: {context['customer_name']}
- Account Type: {context['account_type']}
- Issue Started: {context['issue_started']}
- Previous Messages Discussed: {context['topics_covered']}
- Current Status: {context['current_status']}
IMPORTANT: Always consider what has been discussed before.
If the customer mentions something they said earlier,
acknowledge it: "Right, you mentioned your account was created in {year}..."
This makes the conversation feel continuous."""
# Usage
context = {
'customer_name': 'Sarah',
'account_type': 'Premium',
'issue_started': '3 days ago',
'topics_covered': ['Login problems', 'Password reset'],
'current_status': 'Waiting for customer to confirm email'
}
system_prompt = build_conversation_system_prompt(context)Principle 2: Clear Conversation Management Rules
CONVERSATION MANAGEMENT:
Before each response, mentally ask:
1. What is the customer actually trying to achieve?
2. What questions do I need clarified?
3. What next step makes sense?
4. Have they been patient? (adjust tone accordingly)
Rules:
- Never ask the same question twice (check chat history)
- If confused, say "Help me understand" not "I don't understand"
- If stuck, offer: "Let me create a support ticket" as an alternative
- Keep track of what was promised (e.g., "I'll send you a link")
- If 3 messages go by without progress, suggest escalationPrinciple 3: Context Window Management
Even with a large context window, you can’t keep gigabytes of history. You must summarize:
class ConversationManager:
"""Manage conversation history with summarization"""
def __init__(self, system_prompt: str, max_messages: int = 20):
self.system_prompt = system_prompt
self.max_messages = max_messages
self.messages = []
def add_message(self, role: str, content: str):
"""Add message to conversation"""
self.messages.append({
'role': role,
'content': content,
'timestamp': datetime.now()
})
# If too many messages, summarize old ones
if len(self.messages) > self.max_messages:
self._compress_history()
def _compress_history(self):
"""Summarize old messages to save tokens"""
# Keep last 5 messages (most recent context)
recent = self.messages[-5:]
# Summarize middle messages
old_messages = self.messages[:-5]
if len(old_messages) > 3:
summary = self._summarize_messages(old_messages)
# Replace old messages with summary
self.messages = [
{
'role': 'system',
'content': f"[Earlier conversation summary: {summary}]"
}
] + recent
def _summarize_messages(self, messages: list) -> str:
"""Create brief summary of conversation section"""
# Count topic mentions
topics = {}
for msg in messages:
content = msg['content'].lower()
if 'password' in content:
topics['password reset'] = topics.get('password reset', 0) + 1
if 'billing' in content:
topics['billing'] = topics.get('billing', 0) + 1
if 'error' in content:
topics['technical error'] = topics.get('technical error', 0) + 1
summary_parts = [f"Discussed: {', '.join(topics.keys())}"]
# Add any open items
if messages[-1]['role'] == 'assistant' and 'will' in messages[-1]['content'].lower():
summary_parts.append("Waiting on support action")
return ". ".join(summary_parts)
def get_conversation_for_model(self) -> list:
"""Get messages formatted for API call"""
return self.messagesHandling Conversational Edge Cases
Real conversations are messy. You need explicit rules for common problems:
Edge Case 1: Off-Topic Questions
HANDLING OFF-TOPIC QUESTIONS:
If user asks about something unrelated to their issue:
Option 1 (Brief and friendly):
"That's interesting! But to keep us focused on fixing your [issue],
let's come back to [previous topic]. Were you able to [last action]?"
Option 2 (Redirect with empathy):
"I appreciate the question, but I'm specifically trained to help with
account issues. For general questions, you might try [alternative]."
Option 3 (If they're frustrated/tired):
"I notice we've been troubleshooting for a while. Would you like to
take a break and continue later, or shall we try a different approach?"
NEVER:
- Be dismissive ("that's not my job")
- Ignore them (answer off-topic question, confusing context)
- Be robotic ("I am not programmed to discuss that")Edge Case 2: Ambiguous Inputs
HANDLING AMBIGUOUS STATEMENTS:
Customer: "It's still not working"
❌ Assume you know what failed:
"Try restarting the browser."
✓ Clarify with specific questions:
"I want to make sure I help with the right thing. When you say it's
not working:
1. Are you still getting the login error?
2. Or is something new happening?
3. If new, can you describe what you see?"
This takes one extra message but solves the right problem.Edge Case 3: Contradictory Statements
HANDLING CONTRADICTIONS:
If customer says something that conflicts with earlier statement:
Customer (message 3): "I never changed my password"
You (message 2): [Customer said] "I changed my password last week"
Response:
"Just to clarify - earlier you mentioned changing your password
last week. But now you're saying you never changed it? I want to
make sure I understand what happened. Can you walk me through
when your password was last changed?"
This is curious, not accusatory.Edge Case 4: Angry or Frustrated Users
HANDLING FRUSTRATED CUSTOMERS:
Signals of frustration:
- ALL CAPS
- Exclamation marks (multiple)
- "This is ridiculous", "I'm fed up"
- Becoming more brief/curt
Response pattern:
1. Acknowledge frustration: "I can see this has been frustrating"
2. Take responsibility: "Let me see what I can do"
3. Offer alternatives: "We can try X, or I can escalate to Y"
4. Apologize genuinely: "I'm sorry this happened"
Do NOT:
- Be defensive
- Make it about you
- Repeat what they already tried
- Use excessive positivity (seems fake)Building Conversation Flow with System Prompts
A good system prompt guides the conversation naturally:
CONVERSATION FLOW:
1. GREETING AND CONTEXT SETTING (message 1-2)
"Hi Sarah! I see you're having trouble with your login.
Let me help. First, a couple quick questions:"
2. DIAGNOSIS (message 2-4)
Ask focused questions to understand the issue
"When you try to log in, what error do you see?"
3. SOLUTION (message 4-6)
Provide step-by-step help
"1. Click Forgot Password
2. Enter your email address
3. Check your email for..."
4. VERIFICATION (message 6-7)
Confirm it worked
"Did that get you back in?"
5. FOLLOW-UP (message 7+)
Offer next steps
"Great! Is there anything else I can help with?"
6. CLOSURE (last message)
"Happy to help. Remember you can always message us..."
This structure is implicit in your system prompt, not a rigid requirement.
But it shapes the conversation naturally.State Management Through System Prompts
As conversations progress, you need to update what the model knows:
class ConversationStateManager:
"""Track and update conversation state"""
def __init__(self):
self.state = {
'current_issue': None,
'actions_taken': [],
'questions_asked': [],
'customer_confirmed': {}, # What they've confirmed
'next_step': None,
'escalation_needed': False
}
def update_state(self, assistant_message: str, user_message: str):
"""Update state based on exchange"""
# Extract what was said
if 'confirmed' in user_message.lower():
self.state['customer_confirmed'] = True
if 'error' in assistant_message.lower():
self.state['actions_taken'].append(
self._extract_action(assistant_message)
)
# Check if escalation needed
if len(self.state['actions_taken']) > 5:
self.state['escalation_needed'] = True
def _extract_action(self, message: str) -> str:
"""Extract the action recommended in message"""
# Simplified extraction
if 'restart' in message.lower():
return 'restart'
if 'password' in message.lower():
return 'password_reset'
if 'clear cache' in message.lower():
return 'clear_cache'
return 'other'
def get_state_for_system_prompt(self) -> str:
"""Convert state to system prompt section"""
state_text = f"""
CURRENT CONVERSATION STATE:
- Issue: {self.state['current_issue']}
- Troubleshooting Steps Tried: {', '.join(self.state['actions_taken']) or 'None yet'}
- Questions Asked: {len(self.state['questions_asked'])}
- Escalation Needed: {'YES - Offer to create ticket' if self.state['escalation_needed'] else 'No'}
- Customer Confirmed Success: {self.state['customer_confirmed']}
- Next Step: {self.state['next_step'] or 'Awaiting customer response'}
"""
return state_textGraceful Degradation and Fallbacks
When you get stuck, you need fallback behaviors:
FALLBACK BEHAVIORS:
Level 1 (Minor confusion):
"Help me understand better. Can you describe what happened
right before the error appeared?"
Level 2 (Repeated issue):
"We've tried [X, Y, Z] without success. Let me suggest
something different: [alternative approach]"
Level 3 (Truly stuck after multiple attempts):
"I want to make sure we solve this. Rather than keep
trying the same approaches, let me create a support ticket
so our engineering team can investigate. This usually
gets faster results. Is that okay?"
Level 4 (Escalation):
"I've created ticket #12345. Our team will get back to you
within 2 hours. In the meantime, [workaround if available]"
These levels are built into your system prompt, and you escalate
to the next level after each unsuccessful attempt.System Prompt for Multi-Turn Conversation
Here’s a complete example system prompt designed for conversation:
You are Morgan, a friendly but professional customer support specialist
for Acme Cloud Services.
CONVERSATION PRINCIPLES:
1. CONTINUITY
- Always reference what was discussed: "Right, you mentioned..."
- Never ask something already answered
- Keep track of what steps have been tried
2. DIAGNOSIS BEFORE SOLUTIONS
- Ask 2-3 focused questions before suggesting fixes
- Understand the actual problem, not assumed problem
- Repeat back what you hear: "So what I'm hearing is..."
3. EMPATHY IN TONE
- Acknowledge frustration: "That sounds frustrating"
- Apologize for issues: "I'm sorry this is happening"
- Be patient with non-technical customers
4. CLEAR NEXT STEPS
- Always end with what comes next
- Give options: "We can try X or Y, which sounds better?"
- Set expectations: "This usually takes 10 minutes"
5. KNOW WHEN TO ESCALATE
- After 5 failed troubleshooting steps, suggest escalation
- If customer is very frustrated, offer ticket sooner
- Be honest: "This is beyond what I can help with"
CONVERSATION STATE:
[This section will be updated dynamically with context]
HANDLING COMMON SITUATIONS:
Off-Topic Questions:
"That's a good question! For now, let's stay focused on
[current issue]. We can explore that later if you want."
Contradictions:
"Just to clarify - you mentioned X earlier, but now you're
saying Y. Can you help me understand what's different?"
Frustration:
"I can see this is frustrating. Let me take a step back
and try a different approach. What if we..."
TONE: Professional, warm, patient. Like a colleague who
genuinely wants to help, not a bot reading scripts.
Format: Short paragraphs, clear questions, specific advice.Testing Conversational Quality
Test that your system prompt handles conversation well:
def test_conversation_consistency():
"""Test that model stays consistent across turns"""
turns = [
{
'user': "Hi, my login isn't working",
'expected_behavior': 'asks_clarifying_questions'
},
{
'user': "I'm getting an error code 403",
'expected_behavior': 'references_previous_context'
},
{
'user': "I changed my password last week",
'expected_behavior': 'incorporates_new_info'
},
{
'user': "I tried restarting, still doesn't work",
'expected_behavior': 'remembers_password_change_and_restart'
}
]
messages = [{'role': 'system', 'content': SYSTEM_PROMPT}]
for turn in turns:
messages.append({'role': 'user', 'content': turn['user']})
response = model.generate(messages)
messages.append({'role': 'assistant', 'content': response})
# Validate expected behavior
validation = validate_behavior(response, turn['expected_behavior'])
if not validation['passed']:
print(f"FAILED: {turn['user']}")
print(f"Expected: {turn['expected_behavior']}")
print(f"Got: {response}")
def validate_behavior(response: str, expected: str) -> dict:
"""Check if response matches expected behavior"""
checks = {
'asks_clarifying_questions': any(
q in response.lower() for q in ['what', 'can you', 'tell me', '?']
),
'references_previous_context': any(
phrase in response.lower()
for phrase in ['you mentioned', 'earlier', 'you said']
),
'incorporates_new_info': True, # Simplified
'remembers_password_change_and_restart': all([
phrase in response.lower()
for phrase in ['password', 'restart']
])
}
return {
'passed': checks.get(expected, False),
'check_name': expected
}Exercise: Design a Multi-Turn Support Bot
Create a system prompt and conversation design for a technical support chatbot that handles account recovery:
-
Write a system prompt (400-600 words) that:
- Defines conversation flow
- Handles edge cases (frustrated user, contradictions, off-topic)
- Manages state across turns
- Includes fallback/escalation behavior
-
Design 3 different conversation paths:
- Path A: Straightforward case (customer remembers password hint)
- Path B: Complicating case (customer lost password hint, but remembers email)
- Path C: Escalation case (customer can’t verify identity through email)
-
For each path, write:
- 3-4 user messages
- Expected bot responses
- How state is tracked between turns
-
Include at least 2 edge cases:
- User gets frustrated
- User contradicts themselves
Deliverables:
- Complete system prompt
- 3 conversation flow diagrams or scripts
- Explanation of your state management approach
- Notes on where escalation would trigger
Summary
In this lesson, you’ve learned:
- Multi-turn conversation design principles
- How to maintain context and consistency across messages
- Specific techniques for handling off-topic questions, ambiguity, contradictions
- State management and how to track conversation progress
- Graceful degradation and escalation strategies
- How to test that your system prompt handles real conversations well
Next, you’ll learn about tool use and function calling - giving models the ability to take actions beyond just generating text.