Intermediate
Dynamic System Prompts and Context Injection
Dynamic System Prompts and Context Injection
Introduction
So far, you’ve learned about static system prompts - the same instructions for every user. But real applications need to adapt. A customer support agent should know the customer’s account status. A sales assistant should know what products are available. A content moderator should know the current policy rules.
Dynamic system prompts inject runtime context into the base system prompt. Instead of one generic prompt, you generate a customized prompt for each user or session. This is where system prompts become truly powerful.
Key Takeaway: Static system prompts are a starting point. Production systems customize prompts with user data, session context, and business rules. The “best” system prompt changes based on who’s asking and what they’re trying to do.
The Dynamic Prompt Concept
Instead of this:
System Prompt (static, same for everyone):
"You are a helpful customer support assistant"You do this:
Base System Prompt (template):
"You are a helpful customer support assistant.
You are helping {customer_name} with their {account_type} account.
They have been a customer since {customer_since}.
..."
At runtime, inject values:
{
customer_name: "Sarah Chen",
account_type: "Enterprise",
customer_since: "2021-03-15"
}
Generated System Prompt (specific to this customer):
"You are a helpful customer support assistant.
You are helping Sarah Chen with their Enterprise account.
They have been a customer since 2021-03-15.
..."Building Dynamic System Prompt Templates
Start with a template that has placeholders:
class DynamicSystemPrompt:
"""Template-based system prompt with runtime injection"""
BASE_TEMPLATE = """You are a {role} for {company}.
CUSTOMER CONTEXT:
- Name: {customer_name}
- Account Type: {account_type}
- Status: {account_status}
- Tenure: {customer_tenure}
RELEVANT BUSINESS CONTEXT:
{business_context}
{role_specific_instructions}
BEHAVIORAL GUIDELINES:
{behavioral_guidelines}
"""
def __init__(self):
self.variables = {}
self.role_templates = {
'support': self._support_instructions,
'sales': self._sales_instructions,
'billing': self._billing_instructions
}
def generate(self, role: str, context: dict) -> str:
"""Generate a customized system prompt"""
# Merge defaults with provided context
full_context = {
'company': context.get('company', 'AcmeCorp'),
'customer_name': context.get('customer_name', 'Valued Customer'),
'account_type': context.get('account_type', 'Standard'),
'account_status': context.get('account_status', 'Active'),
'customer_tenure': self._format_tenure(context.get('customer_since')),
'role': role,
'business_context': self._get_business_context(context),
'role_specific_instructions': self.role_templates[role](context),
'behavioral_guidelines': self._get_behavioral_guidelines(context)
}
return self.BASE_TEMPLATE.format(**full_context)
def _format_tenure(self, customer_since: str) -> str:
"""Format customer tenure"""
from datetime import datetime
if not customer_since:
return "Unknown"
start_date = datetime.fromisoformat(customer_since)
today = datetime.now()
years = (today - start_date).days // 365
if years == 0:
return "Less than a year"
elif years == 1:
return "1 year"
else:
return f"{years} years"
def _get_business_context(self, context: dict) -> str:
"""Get relevant business context"""
business_data = context.get('business_data', {})
context_lines = [
f"- Account Value: ${business_data.get('annual_value', 0):,.0f}",
f"- Support Tier: {business_data.get('support_tier', 'Standard')}",
f"- Active Services: {', '.join(business_data.get('services', []))}",
]
return "\n".join(context_lines)
def _support_instructions(self, context: dict) -> str:
"""Support-specific instructions"""
tier = context.get('business_data', {}).get('support_tier', 'Standard')
instructions = """YOUR ROLE:
Help this customer solve their issues efficiently.
"""
if tier == 'Enterprise':
instructions += """ENTERPRISE CUSTOMER - Priority handling:
- Escalate urgent issues immediately
- Provide direct contact for complex issues
- Reference SLA terms if needed
"""
elif tier == 'Premium':
instructions += """PREMIUM CUSTOMER:
- Provide expedited support
- Offer pro-active suggestions
- Go extra mile when reasonable
"""
return instructions
def _sales_instructions(self, context: dict) -> str:
"""Sales-specific instructions"""
return """YOUR ROLE:
Help this customer understand how our products solve their needs.
Never pressure. Always listen first.
"""
def _billing_instructions(self, context: dict) -> str:
"""Billing-specific instructions"""
return """YOUR ROLE:
Help with billing questions and account management.
Be transparent about costs and options.
"""
def _get_behavioral_guidelines(self, context: dict) -> str:
"""Get behavioral guidelines based on customer"""
is_premium = context.get('business_data', {}).get('support_tier') in ['Enterprise', 'Premium']
if is_premium:
return """- Take ownership of issues
- Offer proactive suggestions
- Create tickets for follow-up
- Get manager approval for exceptions"""
else:
return """- Be helpful but efficient
- Follow standard procedures
- Escalate only when necessary
- Explain all costs upfront"""
# Usage
prompt_generator = DynamicSystemPrompt()
# Customer 1: Enterprise customer
enterprise_system_prompt = prompt_generator.generate(
role='support',
context={
'customer_name': 'Sarah Chen',
'account_type': 'Enterprise',
'account_status': 'Active',
'customer_since': '2019-03-15',
'business_data': {
'annual_value': 150000,
'support_tier': 'Enterprise',
'services': ['Analytics', 'Automation', 'Integrations']
}
}
)
# Customer 2: Standard customer
standard_system_prompt = prompt_generator.generate(
role='support',
context={
'customer_name': 'John Doe',
'account_type': 'Basic',
'account_status': 'Active',
'customer_since': '2024-01-01',
'business_data': {
'annual_value': 1200,
'support_tier': 'Standard',
'services': ['Analytics']
}
}
)Context Injection Patterns
Pattern 1: User Profile Injection
def inject_user_profile(base_prompt: str, user_id: str) -> str:
"""Customize prompt with user profile data"""
user_data = fetch_user_profile(user_id)
user_context = f"""
CURRENT USER:
- Name: {user_data['name']}
- Role: {user_data['role']}
- Department: {user_data['department']}
- Preferences: {user_data['preferences']}
- Recent History: {user_data['recent_activity']}
"""
return base_prompt + "\n" + user_context
def fetch_user_profile(user_id: str) -> dict:
"""Fetch user profile from database"""
# Simulated
return {
'name': 'Alice Johnson',
'role': 'Software Engineer',
'department': 'Engineering',
'preferences': {
'detail_level': 'technical',
'response_style': 'concise',
'code_language': 'Python'
},
'recent_activity': 'Asked about async patterns'
}Pattern 2: Session State Injection
class SessionContextManager:
"""Track and inject session state into prompts"""
def __init__(self, session_id: str):
self.session_id = session_id
self.context = {
'started_at': datetime.now(),
'turn_count': 0,
'topics_discussed': [],
'actions_taken': [],
'user_intent': None
}
def update_context(self, user_message: str, assistant_response: str):
"""Update session context based on conversation"""
self.context['turn_count'] += 1
# Extract topics
topics = self._extract_topics(user_message)
self.context['topics_discussed'].extend(topics)
# Track actions
if 'create ticket' in assistant_response.lower():
self.context['actions_taken'].append('ticket_created')
def get_context_string(self) -> str:
"""Get session context as string for prompt injection"""
return f"""SESSION CONTEXT:
- Turn Number: {self.context['turn_count']}
- Topics Discussed: {', '.join(set(self.context['topics_discussed']))}
- Actions Taken: {', '.join(self.context['actions_taken'])}
- Session Duration: {(datetime.now() - self.context['started_at']).seconds} seconds
"""
def _extract_topics(self, message: str) -> list:
"""Extract topics from message"""
topics = []
keywords = {
'password': ['password', 'reset', 'forgot', 'login'],
'billing': ['bill', 'charge', 'invoice', 'payment'],
'feature': ['feature', 'request', 'want', 'need'],
'error': ['error', 'broken', 'fail', 'not working']
}
for topic, keywords_list in keywords.items():
if any(k in message.lower() for k in keywords_list):
topics.append(topic)
return topics
def inject_into_prompt(self, base_prompt: str) -> str:
"""Inject session context into prompt"""
return base_prompt + "\n" + self.get_context_string()Pattern 3: Business Rules Injection
def inject_business_rules(base_prompt: str, context: dict) -> str:
"""Inject current business rules and policies"""
current_policies = get_active_policies(context.get('region'))
rules_section = f"""
ACTIVE BUSINESS RULES (Last Updated: {current_policies['updated_at']}):
Pricing:
{chr(10).join(f"- {tier}: ${price}/month" for tier, price in current_policies['pricing'].items())}
Limitations:
{chr(10).join(f"- {limit_name}: {limit_value}" for limit_name, limit_value in current_policies['limits'].items())}
Promotions:
{chr(10).join(f"- {promo['name']}: {promo['description']}" for promo in current_policies['active_promotions'])}
Do NOT mention policies or promotions not listed above.
If customer asks about unlisted policies, say 'That policy is not currently active'.
"""
return base_prompt + "\n" + rules_section
def get_active_policies(region: str = 'US') -> dict:
"""Get current policies for region"""
# Simulated - would connect to policy database
return {
'updated_at': '2024-03-19',
'pricing': {
'Basic': 9.99,
'Pro': 29.99,
'Enterprise': 'Custom'
},
'limits': {
'API calls per day': '10,000',
'Storage': '100GB',
'Concurrent connections': '100'
},
'active_promotions': [
{
'name': 'New Year Special',
'description': '30% off annual plans through March 31'
}
]
}Pattern 4: Retrieval-Augmented System Prompts
Sometimes context comes from documents or knowledge bases:
class RAGSystemPrompt:
"""System prompt augmented with retrieved context"""
def __init__(self, base_prompt: str, retriever):
self.base_prompt = base_prompt
self.retriever = retriever
def generate(self, user_query: str) -> str:
"""Generate prompt with relevant context from knowledge base"""
# Retrieve relevant documents
relevant_docs = self.retriever.search(user_query, top_k=3)
# Build context section
context_section = "RELEVANT INFORMATION:\n\n"
for doc in relevant_docs:
context_section += f"- {doc['title']}: {doc['summary']}\n"
return self.base_prompt + "\n\n" + context_section
class SimpleRetriever:
"""Simple document retriever"""
def __init__(self, documents: list):
self.documents = documents
def search(self, query: str, top_k: int = 3) -> list:
"""Find relevant documents"""
scores = []
for doc in self.documents:
# Simple keyword matching
score = sum(1 for word in query.lower().split()
if word in doc['content'].lower())
if score > 0:
scores.append((doc, score))
# Return top K by score
scores.sort(key=lambda x: x[1], reverse=True)
return [doc for doc, score in scores[:top_k]]
# Usage
documents = [
{
'title': 'API Rate Limits',
'content': 'Free tier: 100 requests/minute. Pro: 1000 requests/minute.',
'summary': 'Rate limiting policy'
},
{
'title': 'Refund Policy',
'content': 'Refunds allowed within 30 days of purchase. Must contact support.',
'summary': 'How to get a refund'
},
{
'title': 'Data Privacy',
'content': 'We encrypt all user data. GDPR compliant.',
'summary': 'Data protection practices'
}
]
rag_prompt = RAGSystemPrompt(
base_prompt="You are a helpful customer support agent.",
retriever=SimpleRetriever(documents)
)
# When answering a question:
user_query = "What are your rate limits?"
system_prompt = rag_prompt.generate(user_query)
print(system_prompt)A/B Testing Dynamic Prompts
Dynamic prompts create new testing opportunities:
def ab_test_dynamic_prompts(control_prompt_fn: callable,
variant_prompt_fn: callable,
test_contexts: list,
model_fn: callable) -> dict:
"""A/B test two different dynamic prompt generators"""
results = {
'control': [],
'variant': [],
'metrics': {}
}
for context in test_contexts:
# Generate prompts
control = control_prompt_fn(context)
variant = variant_prompt_fn(context)
# Test each
control_response = model_fn(control, context['user_query'])
variant_response = model_fn(variant, context['user_query'])
# Evaluate
control_score = evaluate_response(control_response, context)
variant_score = evaluate_response(variant_response, context)
results['control'].append(control_score)
results['variant'].append(variant_score)
# Summary
control_avg = sum(results['control']) / len(results['control'])
variant_avg = sum(results['variant']) / len(results['variant'])
results['metrics'] = {
'control_avg': control_avg,
'variant_avg': variant_avg,
'improvement': variant_avg - control_avg,
'improvement_percent': ((variant_avg - control_avg) / control_avg * 100)
}
return results
def evaluate_response(response: str, context: dict) -> float:
"""Score a response (0-1)"""
score = 0.5
# Positive signals
if context.get('account_type') == 'Enterprise':
if 'enterprise' in response.lower() or 'priority' in response.lower():
score += 0.2
# Check accuracy
if 'inaccurate' not in context.get('feedback', '').lower():
score += 0.3
return min(1.0, score)Context Window Management with Dynamic Prompts
More context uses more tokens. Be strategic:
class EfficientDynamicPrompt:
"""Dynamic prompt that optimizes token usage"""
def __init__(self, base_prompt: str):
self.base_prompt = base_prompt
def generate(self, context: dict, token_budget: int = 500) -> str:
"""Generate prompt within token budget"""
prompt = self.base_prompt
used_tokens = self._estimate_tokens(prompt)
remaining_budget = token_budget - used_tokens
# Add context in priority order
if remaining_budget > 100:
prompt += self._get_user_context(context)
remaining_budget -= 100
if remaining_budget > 50:
prompt += self._get_session_context(context)
remaining_budget -= 50
if remaining_budget > 30:
prompt += self._get_business_context(context)
return prompt
def _estimate_tokens(self, text: str) -> int:
"""Rough token estimate (1 token ~4 chars)"""
return len(text) // 4
def _get_user_context(self, context: dict) -> str:
"""User context (required)"""
return f"\nUser: {context.get('user_name', 'Unknown')}"
def _get_session_context(self, context: dict) -> str:
"""Session context (important)"""
return f"\nSession turn: {context.get('turn_count', 0)}"
def _get_business_context(self, context: dict) -> str:
"""Business context (nice to have)"""
return f"\nAccount: {context.get('account_type', 'Standard')}"Production Example: E-Commerce Support
Here’s how this all comes together in a real system:
class DynamicECommerceSupport:
"""Dynamic system prompt for e-commerce support"""
def __init__(self):
self.session_manager = SessionContextManager(session_id='new')
self.rag_retriever = SimpleRetriever(self._get_knowledge_base())
def generate_system_prompt(self, user_id: str, session_id: str) -> str:
"""Generate customized system prompt for this user"""
# Get user data
user = self._fetch_user(user_id)
# Get business context
policies = get_active_policies()
# Build prompt
base = f"""You are a helpful e-commerce support specialist.
CUSTOMER:
- Name: {user['name']}
- Account Status: {user['status']}
- Lifetime Value: ${user['lifetime_value']:,}
- VIP: {'Yes' if user['lifetime_value'] > 10000 else 'No'}
CURRENT POLICIES:
- Free shipping: {policies['free_shipping_threshold']}
- Return window: {policies['return_days']} days
- Refund method: {policies['refund_method']}
SESSION:
- Turn: {self.session_manager.context['turn_count']}
- Topics: {', '.join(self.session_manager.context['topics_discussed'])}
GUIDELINES:
- For VIP customers, offer alternatives and be flexible
- For standard customers, follow policy strictly
- Always be polite but efficient
- Escalate complex issues to manager"""
return base
def handle_message(self, user_id: str, user_message: str,
model_fn: callable) -> str:
"""Process user message with dynamic prompt"""
# Generate dynamic prompt
system_prompt = self.generate_system_prompt(user_id, 'current_session')
# Call model
response = model_fn(system_prompt, user_message)
# Update session context
self.session_manager.update_context(user_message, response)
return response
def _fetch_user(self, user_id: str) -> dict:
"""Get user from database"""
return {
'name': 'Sarah',
'status': 'Active',
'lifetime_value': 15000
}
def _get_knowledge_base(self) -> list:
"""Get support knowledge base"""
return [
{'title': 'Shipping Policy', 'content': '...', 'summary': '...'},
{'title': 'Returns', 'content': '...', 'summary': '...'},
]Exercise: Build a Dynamic Prompt System
Create a complete dynamic system prompt system for a SaaS product support chatbot:
-
Base prompt template with placeholders for:
- Customer name
- Account type (Free/Pro/Enterprise)
- Days as customer
- Recent issues
- Account status
-
Context injection functions:
- User profile injection
- Session state injection
- Business rules injection
-
Prompt generation that creates customized prompts for:
- A brand new free user
- A Pro customer with 2+ years tenure
- An Enterprise customer
- A customer with past escalations
-
A/B test comparison showing:
- Static vs dynamic prompts on 5 test cases
- Improvement metrics
- Token usage analysis
-
Production code showing how you’d integrate this into a real system
Deliverables:
- Prompt template with placeholders
- Context injection code (3+ types)
- Prompt generation function
- 3-4 example generated prompts
- A/B test results
- Integration example
Summary
In this lesson, you’ve learned:
- How to build template-based dynamic system prompts
- Context injection patterns for user data, session state, and business rules
- Retrieval-augmented system prompts that pull context from knowledge bases
- How to test and A/B test dynamic prompts
- Token budget management when adding context
- A complete production example
You’ve now completed Module 2 on System Prompts. Next, Module 3 covers multimodal prompting: working with images, audio, video, and documents.