Designing for AI: prompt patterns that ship
Deploying reliable, production-grade applications that leverage Large Language Models (LLMs) requires more than just calling an API. The quality,…
Deploying reliable, production-grade applications that leverage Large Language Models (LLMs) requires more than just calling an API. The quality, consistency, and predictability of LLM outputs are directly influenced by the prompt engineering strategies employed. This article details established prompt patterns and architectural considerations for building robust, maintainable, and verifiable LLM integrations, focusing on techniques that minimize hallucination, enforce structure, and facilitate error handling.
We'll cover patterns from initial system setup to advanced output parsing, demonstrating how to move beyond basic conversational turns to achieve structured, actionable results from models like OpenAI's GPT series (e.g., GPT-3.5-turbo, GPT-4o) and Anthropic's Claude models (e.g., Claude 3 Opus).
Establishing a Robust System Prompt
The system prompt is the foundational instruction set that defines the LLM's role, constraints, and operational context for an entire interaction session. A well-crafted system prompt sets boundaries, biases the model towards desired behaviors, and preempts common failure modes.
Defining Role and Persona
Explicitly stating the model's role helps it adopt an appropriate tone and perspective. Be precise, avoid ambiguity.
You are a highly analytical Senior Security Engineer for "SecureCorp Inc.", tasked with evaluating software vulnerabilities.
Your primary goal is to provide concise, technical assessments without speculating or offering general advice unless specifically asked.
Focus strictly on CVEs, CWEs, and potential exploitation vectors.
Setting Constraints and Guardrails
Crucially, system prompts must define what the model should not do. This includes limitations on output length, forbidden topics, or refusal conditions.
- Safety & Ethics: "Do not engage in discussions about illegal activities, hate speech, or explicit content."
- Scope Limitation: "Only respond with information directly derivable from the provided context. If information is not available, state 'Information not available in context.' Do not invent facts."
- Format Enforcement: "All outputs must be in valid JSON format, adhering to the schema provided. If you cannot produce valid JSON, return an error message in plain text."
Incorporating Schema Expectations (Implicitly)
While explicit JSON schema enforcement (covered later) is preferred, the system prompt can pre-condition the model to think in terms of structured data. For example, if the output should be a list of findings, suggest that structure early.
Your output should always be a structured list of security findings. Each finding must include a 'vulnerability_name', 'severity' (CRITICAL, HIGH, MEDIUM, LOW), and 'recommendation'.
Few-Shot Prompting for Tone and Specific Examples
Few-shot prompting involves providing one or more example input-output pairs within the prompt. This technique is invaluable for calibrating the LLM's response style, ensuring consistency, and guiding it towards complex output formats that are hard to describe purely with natural language.
Demonstrating Desired Output Format
If you need a very specific JSON structure or a nuanced writing style, examples are more effective than verbose descriptions.
System: You are a data extraction bot. Extract the customer name and order ID from the following email.
User: Email: "Hello, my name is John Doe and I'd like to inquire about my order #12345."
Assistant: {"customer_name": "John Doe", "order_id": "12345"}
User: Email: "Hi team, this is Alice Smith regarding purchase ref: PX-987. Can you help?"
Assistant: {"customer_name": "Alice Smith", "order_id": "PX-987"}
User: Email: "Order 789 for Bob Johnson has a problem."
Assistant:
This approach subtly teaches the model the exact keys and value types expected, as well as how to handle variations in input phrasing.
Refining Tone and Persona
Beyond structure, few-shot examples can establish a specific tone—e.g., highly technical, empathetic, or concise. This is particularly useful when the system prompt alone isn't enough to capture the desired nuance.
Strict Output Formatting: Force JSON Mode
For programmatic consumption, unstructured text is problematic. Modern LLMs offer mechanisms to guarantee valid JSON output, significantly reducing post-processing complexity and improving reliability. This is a critical pattern for "LLM as an API" scenarios.
Using Model-Specific JSON Modes
- OpenAI (GPT-3.5-turbo, GPT-4o+): Use the
response_format={"type": "json_object"}parameter in the API call. The prompt itself should also instruct the model to produce JSON. - Anthropic (Claude 3+): While not a dedicated API parameter (as of early 2024), strongly encouraging JSON output in the system prompt is effective. Use a structure like "ALWAYS respond with a JSON object. The JSON object should conform to the following schema:" followed by the schema. Wrapping the output with triple backticks and 'json' (
```json...```) also helps.
Example (OpenAI API call with Python)
from openai import OpenAI
client = OpenAI()
def extract_project_details(text):
response = client.chat.completions.create(
model="gpt-4o",
response_format={"type": "json_object"},
messages=[
{"role": "system", "content": "You are an assistant designed to extract project details. Always output valid JSON."},
{"role": "user", "content": f"""
Extract the project name, lead engineer, and estimated completion date from the following text.
If any detail is missing, set its value to null.
Text: "Our new initiative, Project Chimera, is led by Dr. Anya Sharma. We expect to finish by Q3 2025."
Output JSON should be of the format:
{{
"project_name": "string",
"lead_engineer": "string",
"estimated_completion_date": "string"
}}
"""}
]
)
return response.choices[0].message.content
# Example usage
json_output = extract_project_details("The Quantum Leap project, headed by Ben Carter, aims for a mid-2026 delivery.")
print(json_output)
# Expected output (may vary slightly but will be valid JSON):
# {
# "project_name": "Quantum Leap",
# "lead_engineer": "Ben Carter",
# "estimated_completion_date": "mid-2026"
# }
Even with response_format={"type": "json_object"}, it's good practice to reiterate "Always output valid JSON" in the system or user prompt for redundancy and clarity.
Leveraging Tool-Calling / Function Calling
Tool-calling (also known as function calling) is a powerful paradigm where the LLM's output is not just text, but a structured request to execute a predefined function or tool in your application. This bridges the gap between natural language understanding and programmatic action, enabling complex workflows.
How it Works
- You define a set of available functions (their names, descriptions, and parameters using JSON schema).
- You provide these definitions to the LLM API.
- The LLM decides if any user query or prompt necessitates calling one of these functions. If so, it returns a structured call to that function, including arguments extracted from the user's input.
- Your application receives this function call, executes the corresponding code, and then (optionally) feeds the function's result back to the LLM for a natural language summary or follow-up.
Example: Weather Tool (OpenAI Syntax)
import json
from openai import OpenAI
client = OpenAI()
# Define the tool (function) available to the LLM
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g., San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
},
}
]
# Simulate the actual function execution
def get_current_weather(location, unit="fahrenheit"):
# In a real app, this would call an external weather API
if "San Francisco" in location:
return f"72 degrees {unit} and sunny in San Francisco, CA."
elif "Boston" in location:
return f"50 degrees {unit} and cloudy in Boston, MA."
else:
return "Weather data not available for that location."
def run_conversation():
messages = [{"role": "user", "content": "What's the weather like in San Francisco?"}]
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
tool_choice="auto", # Let the model decide if it needs to call a tool
)
response_message = response.choices[0].message
if response_message.tool_calls:
tool_call = response_message.tool_calls[0]
function_name = tool_call.function.name
function_args = json.loads(tool_call.function.arguments)
if function_name == "get_current_weather":
function_response = get_current_weather(
location=function_args.get("location"),
unit=function_args.get("unit")
)
messages.append(response_message) # Add assistant's tool_calls to conversation
messages.append(
{
"tool_call_id": tool_call.id,
"role": "tool",
"name": function_name,
"content": function_response,
}
)
second_response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
)
print(second_response.choices[0].message.content)
else:
print(response_message.content)
run_conversation()
# Expected output: The weather in San Francisco is 72 degrees fahrenheit and sunny.
Tool-calling is excellent for:
- Database lookups or modifications.
- Sending emails/notifications.
- Integrating with external APIs (e.g., CRM, project management, financial systems).
- Executing custom business logic.
Robust Output Validation with Zod (or Pydantic)
Even with JSON mode and careful prompting, LLMs can occasionally deviate from a strict schema, especially with complex or nested structures, or under high load. Client-side validation of the LLM's output is therefore a non-negotiable step for production systems.
Why Validate?
- Schema Adherence: Ensures all required fields are present and data types are correct.
- Data Integrity: Prevents malformed data from corrupting downstream processes.
- Error Handling: Provides specific feedback when the LLM fails to meet expectations, allowing for retry logic or human intervention.
- Security: Guards against unexpected inputs if the LLM output is fed directly into other systems.
Using Zod (TypeScript/JavaScript)
Zod is a TypeScript-first schema declaration and validation library. Its concise API and strong type inference make it ideal for validating LLM outputs.
// npm install zod
import { z } from 'zod';
// Define the expected schema for a security finding
const SecurityFindingSchema = z.object({
vulnerability_name: z.string().min(5, "Vulnerability name must be at least 5 characters."),
severity: z.enum(['CRITICAL', 'HIGH', 'MEDIUM', 'LOW'], {
errorMap: () => ({ message: "Severity must be CRITICAL, HIGH, MEDIUM, or LOW." }),
}),
recommendation: z.string().min(10, "Recommendation must be at least 10 characters."),
cve_id: z.string().optional(), // Optional field
});
// Assume 'llmOutputString' is the JSON string received from the LLM
const llmOutputString = `{
"vulnerability_name": "SQL Injection",
"severity": "HIGH",
"recommendation": "Parameterize all SQL queries to prevent injection attacks.",
"cve_id": "CVE-2023-XXXX"
}`;
try {
const parsedOutput = JSON.parse(llmOutputString);
const validatedFinding = SecurityFindingSchema.parse(parsedOutput);
console.log("Validated Security Finding:", validatedFinding);
} catch (error) {
if (error instanceof z.ZodError) {
console.error("LLM Output Validation Error:", error.errors);
// Log error, trigger retry, or notify administrator
} else {
console.error("JSON Parsing Error:", error);
}
}
// Example of invalid output that would cause an error
const invalidLlmOutputString = `{
"vulnerability_name": "XSS",
"severity": "CRITICAL_SEVERITY", // Invalid enum value
"recommendation": "Fix it." // Too short
}`;
try {
const parsedOutput = JSON.parse(invalidLlmOutputString);
const validatedFinding = SecurityFindingSchema.parse(parsedOutput);
console.log("Validated Security Finding:", validatedFinding);
} catch (error) {
if (error instanceof z.ZodError) {
console.error("LLM Output Validation Error (Invalid Example):", error.errors);
/*
Output would be similar to:
[
{
"code": "invalid_enum_value",
"received": "CRITICAL_SEVERITY",
"path": ["severity"],
"message": "Severity must be CRITICAL, HIGH, MEDIUM, or LOW."
},
{
"code": "too_small",
"minimum": 10,
"type": "string",
"inclusive": true,
"exact": false,
"path": ["recommendation"],
"message": "Recommendation must be at least 10 characters."
}
]
*/
}
}
Using Pydantic (Python)
Pydantic offers similar capabilities for Python applications.
# pip install pydantic
from pydantic import BaseModel, Field, ValidationError
from typing import Optional
class SecurityFinding(BaseModel):
vulnerability_name: str = Field(min_length=5)
severity: str = Field(pattern="^(CRITICAL|HIGH|MEDIUM|LOW)$")
recommendation: str = Field(min_length=10)
cve_id: Optional[str] = None
llm_output_json_string = """
{
"vulnerability_name": "Broken Access Control",
"severity": "HIGH",
"recommendation": "Implement strict authorization checks at every layer.",
"cve_id": null
}
"""
try:
finding = SecurityFinding.model_validate_json(llm_output_json_string)
print("Validated Security Finding:", finding.model_dump_json(indent=2))
except ValidationError as e:
print("LLM Output Validation Error:", e.errors())
except Exception as e:
print("JSON Parsing or other Error:", e)
Combine these validation libraries with retry mechanisms or fallback logic for maximum resilience.
Common Pitfalls and Troubleshooting
- Over-constraining: Too many rigid constraints in the system prompt can make the model less creative or lead to outright refusals. Balance strictness with allowing the model enough flexibility.
- Hallucination within Structured Output: Even with JSON mode, the model might invent data. For critical facts, always chain the LLM with retrieval (RAG) or tool-calling to external data sources rather than relying on its internal knowledge.
- Schema Mismatches: If your LLM's output frequently fails schema validation, first review the prompt. Is the schema clearly described, perhaps with few-shot examples? Is the
response_formatparameter correctly set? Sometimes, simpler schemas are more robust. - Token Limits: Large system prompts, extensive few-shot examples, and complex tool definitions consume tokens. Monitor token usage to avoid unexpected costs or truncated responses, especially with models like GPT-3.5-turbo (which historically had smaller context windows than current larger models).
- Non-deterministic Output: Even with identical prompts, LLMs can produce slightly different outputs, especially with higher temperatures. Use a low temperature (e.g.,
temperature=0.0or0.1) for applications requiring maximum consistency. - Error Message Vagueness: If the LLM generates an unparseable response, the model's own error message might be generic. Your client-side validation (Zod/Pydantic) will provide far more specific and actionable error details.