Prompt engineering for agents is not the same as writing good chatbot prompts. When you're prompting an agent, you're not just getting a single response — you're shaping the model's reasoning process across multiple steps, tool calls, and decision points. This guide covers the patterns that consistently produce more reliable, capable agents.
The Agent System Prompt: Your Agent's Identity
Every agent starts with a system prompt that defines its role, capabilities, and constraints. This is the most important prompt you'll write. A weak system prompt leads to an agent that drifts off-task, ignores tools, or hallucinates capabilities it doesn't have.
A strong agent system prompt has four components:
- Role definition: What is this agent? What is it an expert in?
- Capability inventory: What tools does it have? When should it use them?
- Behavioral constraints: What should it never do? How should it handle uncertainty?
- Output format: What format should responses take? How verbose should reasoning be?
SYSTEM: You are a research assistant with access to web search and document retrieval tools.
Your tools:
- search_web(query): Search for current information. Use when the user asks about recent events or data.
- fetch_document(url): Retrieve and parse a webpage. Use after search to get full content.
Rules:
- Always cite your sources.
- If you cannot find reliable information, say so — do not speculate.
- Use search before answering questions about events after 2024.
- Keep responses concise unless the user asks for detail.Chain-of-Thought for Agent Reasoning
Adding "Think step by step before taking any action" to your agent prompt meaningfully improves decision quality on complex tasks, especially for tool selection. For agents using Claude (Anthropic), the extended thinking feature makes this explicit. For GPT-4o, prompt-level chain-of-thought still helps.
For critical reasoning steps, use a scratchpad pattern: ask the agent to write out its reasoning before producing a final answer. This improves accuracy and makes debugging much easier.
Before taking any action, write a brief plan:
1. What information do I need?
2. Which tool is best for getting it?
3. What should the final answer include?
Then execute your plan.Tool Description Engineering
The descriptions you write for tools are themselves prompts. The LLM reads them to decide when and how to use each tool. Good tool descriptions:
- Clearly state what the tool does and what it returns
- Specify when to use it (and equally important, when NOT to)
- Document parameters with types and examples
- Indicate known limitations or failure modes
Bad: """Search the internet."""
Good: """Search the web for current information. Use this when the user asks about recent events, prices, news, or anything that may have changed after your training cutoff. Returns a list of search results with titles and snippets. Does not return full page content — use fetch_page for that."""
Handling Multi-Step Reasoning
For complex tasks, break them into explicit subtasks rather than asking the agent to solve everything at once. The Plan-and-Execute pattern (used in LangGraph's plan-and-execute agent) is effective: first prompt the model to generate a plan, then execute each step with a separate prompt. This prevents the model from "giving up" halfway through a complex reasoning chain.
Few-Shot Examples in Agent Prompts
Few-shot examples are underused in agent engineering. Showing the model 2–3 examples of ideal tool call sequences dramatically improves reliability, especially for novel task types. Include examples of both successful calls and graceful failure handling.
Iterating on Prompts
Treat prompt iteration like software development: version your prompts, evaluate changes against a test set, and track regressions. Tools like PromptLayer, LangSmith, and Braintrust all provide prompt versioning and eval infrastructure.
Strong prompt engineering skills are one of the most transferable competencies in agentic AI right now. If you want to see what companies are paying for these skills, browse prompt engineering and agent developer roles on AgenticCareers.co.