5.3 Prompt Engineering
Key Takeaways
- Prompt engineering designs effective instructions for generative AI; the prompt is your main control because the model itself is unchanged.
- Chat completions use system (persona/behavior/constraints), user (the request), and assistant (history) roles; the system message is the strongest behavior control.
- Zero-shot gives no examples, few-shot gives example input-output pairs, chain-of-thought asks the model to reason step by step, and grounding supplies factual context.
- Grounding reduces hallucinations by constraining the model to provided data rather than its pre-trained memory.
- Temperature controls randomness: low (0-0.3) for factual/consistent answers, high (0.7-1.0) for creative/varied output.
Quick Answer: Prompt engineering is designing effective instructions for generative AI models. Key techniques include system messages (set behavior), few-shot prompting (provide examples), chain-of-thought (step-by-step reasoning), and grounding (provide factual data). A well-crafted prompt can dramatically improve the quality, accuracy, and relevance of AI responses.
What Is Prompt Engineering?
Prompt engineering is the practice of crafting effective prompts (instructions) to get the desired output from a generative AI model. Because you are not modifying the model itself, the prompt is your primary lever for controlling behavior. Think of the model as a highly capable assistant and the prompt as the instructions you give it — better instructions yield better results.
Anatomy of a Prompt
Azure OpenAI uses a chat completion API with three message roles:
| Role | Purpose | Example |
|---|---|---|
| System | Set the AI's persona, behavior, constraints, and format | "You are a helpful banking agent. Be polite and never give financial advice." |
| User | The actual question or request | "What are your hours of operation?" |
| Assistant | Previous AI responses (conversation history) | "Our branches are open Monday to Friday, 9 AM to 5 PM." |
The System Message
The system message is the most powerful tool for controlling AI behavior. It sets the persona (who the AI acts as), tone (how it communicates), format (how responses are structured), constraints (what it must NOT do), and any context it should use. A good system message might read: "You are an expert Azure certification tutor. Explain clearly, always give an example, format lists as bullet points, and if you do not know an answer, say so instead of guessing."
Key Prompt Engineering Techniques
1. Zero-Shot Prompting (No Examples)
Give the model a task with NO examples and rely on its pre-trained knowledge.
Prompt: "Classify the following text as positive, negative, or neutral: 'The movie was absolutely fantastic!'" -> Response: "Positive"
Use it for simple tasks the model already understands from the instruction alone.
2. Few-Shot Prompting (With Examples)
Provide examples of the desired input-output pattern before the actual task:
"Classify the sentiment of each review: Review: 'Great product, fast shipping!' -> Positive Review: 'Terrible quality, waste of money' -> Negative Review: 'It works fine, nothing special' -> Neutral Review: 'The support team resolved my issue in minutes' ->"
The model continues the pattern -> "Positive". Use few-shot when you want a specific format or when zero-shot results are inconsistent.
3. Chain-of-Thought Prompting
Ask the model to think step by step, showing its reasoning.
Prompt: "A store has 47 apples. They sell 23 and receive a shipment of 35. How many now? Think step by step." -> The model walks through 47 - 23 = 24, then 24 + 35 = 59. Adding "think step by step" or "explain your reasoning" significantly improves accuracy on math, logic, and multi-step problems.
4. Grounding with Context
Provide the model with specific data to answer from, rather than relying on its pre-trained memory.
Prompt: "Based on the following document, answer the question. DOCUMENT: 'The AI-900 exam has 40-60 questions, requires a score of 700/1000 to pass, and lasts about 60 minutes. It retires on June 30, 2026 and is replaced by AI-901.' QUESTION: What score is needed to pass the AI-900?" -> Response: "A score of 700 out of 1000."
Grounding constrains the model to the supplied facts, sharply reducing hallucinations. (Note: those exam details are accurate as of 2026 — the AI-900 passing score is 700/1000, the exam runs roughly 60 minutes, and Microsoft has announced it retires June 30, 2026 in favor of AI-901.)
Prompt Engineering Best Practices
| Best Practice | Description | Example |
|---|---|---|
| Be specific | State exactly what you want | "Summarize in 3 bullet points" vs. "Summarize" |
| Set the format | Specify the output format | "Respond in JSON" or "Use a markdown table" |
| Provide context | Give relevant background | Include the document, data, or situation |
| Use constraints | Say what NOT to do | "Do not include personal opinions" |
| Iterate | Refine based on results | Try different wordings if output is off |
| Break down complex tasks | Split into sequential prompts | Summarize -> extract dates -> format as timeline |
Temperature and Other Parameters
| Parameter | Range | Effect | Use Case |
|---|---|---|---|
| Temperature | 0 to 2 | Controls randomness (0 = focused, higher = creative) | 0 for factual; 0.7 for creative |
| Top P | 0 to 1 | Controls diversity of token selection | Alternative to temperature |
| Max tokens | 1 to model limit | Maximum response length | Control response size |
| Frequency penalty | -2 to 2 | Reduces repetition of tokens | Prevent repetitive output |
| Presence penalty | -2 to 2 | Encourages new topics | More diverse content |
On the Exam: Temperature is the most commonly tested parameter. Low temperature (0-0.3) = deterministic, factual, consistent responses; high temperature (0.7-1.0) = creative, varied, unpredictable responses. Also know the four techniques by their trigger words: no examples (zero-shot), examples (few-shot), "step by step" (chain-of-thought), provided documents (grounding).
What is the purpose of a system message in Azure OpenAI?
A developer provides three example input-output pairs before asking the model to classify a new text. What prompt engineering technique is this?
Which temperature setting would produce the MOST creative and varied responses?
What is the purpose of "grounding" in prompt engineering?
Adding the phrase "Think step by step" to a prompt is an example of which technique?