Microsoft Agent Framework Generative AI ( Tutorial 1 ) Created: 14 Apr 2026 Updated: 14 Apr 2026

Temperature – What Is It?

Introduction

When you send a prompt to a large language model (LLM), the model does not simply look up a fixed answer. It calculates a probability distribution over its vocabulary and samples the next token from that distribution. Temperature is the parameter that controls how that sampling is performed — and therefore how predictable or creative the model's output will be.

Understanding temperature is essential for anyone building AI-powered applications because choosing the wrong value can make an agent unreliable (too high) or dull and repetitive (too low).

What Is Temperature?

Temperature is a floating-point number — typically between 0.0 and 2.0 depending on the provider — that scales the logits (raw prediction scores) before they are converted to probabilities via the softmax function.

  1. Lower temperature (0.0 – 0.3) — The probability mass is concentrated on the top tokens. The model almost always picks the highest-probability token, producing deterministic, focused output.
  2. Medium temperature (0.4 – 0.6) — A balanced middle ground that introduces some variety without sacrificing too much coherence.
  3. Higher temperature (0.7 – 1.0) — The probability distribution flattens out, giving lower-ranked tokens a better chance of being selected. The result is more creative, varied output.
  4. Very high temperature (1.0 – 2.0) — The distribution becomes nearly uniform. Output may become incoherent or nonsensical because extremely unlikely tokens are now chosen frequently.

How Temperature Affects the Probability Distribution

Imagine the model has narrowed down the next token to three candidates with the following raw logits:

TokenLogitProbability at T=0.2Probability at T=1.0
"sun"5.0~99.7%~72.1%
"moon"2.0~0.3%~17.9%
"star"1.0~0.0%~9.9%

At low temperature the winner ("sun") takes almost all the probability. At higher temperature the distribution flattens and "moon" or "star" have a reasonable chance of being picked.

Temperature Value Guide

TemperatureBehaviorBest For
0.0Near-deterministic; almost always picks the top tokenData extraction, classification, code generation
0.1 – 0.3Focused with slight variationSummaries, Q&A, agent tool-calling
0.4 – 0.6Moderate varietyDrafting emails, general chat
0.7 – 1.0Creative and diverseCreative writing, brainstorming, storytelling
>1.0Highly random; may lose coherenceExperimental / artistic generation

Related Parameters

Temperature does not work in isolation. Two sibling parameters often interact with it:

  1. Top P (nucleus sampling) — Instead of scaling logits, Top P truncates the candidate list to the smallest set whose cumulative probability is at least P. A value of 0.9 means "consider only tokens that together account for 90% of the probability." Microsoft recommends changing either Temperature or Top P, but not both at once.
  2. Frequency / Presence Penalty — These reduce the chance of tokens that have already appeared in the output, discouraging repetition independently of temperature.

Temperature in Agent Applications

For AI agents that call tools, make decisions, and interact with external systems, low temperatures (0.0 – 0.3) are strongly recommended. Agent frameworks rely on structured, repeatable responses — a creative answer that randomly reformats JSON or invents a non-existent tool call can break the entire workflow. Even at temperature 0, LLM outputs are not fully deterministic due to floating-point arithmetic and batching differences, but they are consistent enough for reliable agent behavior.

Setting Temperature in .NET with Microsoft.Extensions.AI

The Microsoft.Extensions.AI library provides the ChatOptions class with a Temperature property. You can set it per-call so a single application can use different temperatures for different tasks — low for data extraction, high for creative content.


var options = new ChatOptions { Temperature = 0.3f };
var response = await chatClient.GetResponseAsync(prompt, options);

Full Example

using Microsoft.Extensions.AI;
using OpenAI;

namespace MicrosoftAgentFrameworkLesson.ConsoleApp;

public static class TemperatureDemo
{
// Pet adoption profiles — a fresh domain unrelated to official docs examples
private static readonly string[] PetProfiles =
[
"Luna is a 3-year-old tabby cat who loves sunny windowsills and gentle chin scratches.",
"Baxter is a 5-year-old golden retriever who knows 12 tricks and adores swimming.",
"Pepper is a 2-year-old rabbit who enjoys exploring cardboard tunnels and munching kale."
];

public static async Task RunAsync()
{
var apiKey = Environment.GetEnvironmentVariable("OPEN_AI_KEY");
if (string.IsNullOrWhiteSpace(apiKey))
{
Console.WriteLine("Please set the OPEN_AI_KEY environment variable.");
return;
}

var openAiClient = new OpenAIClient(apiKey);
IChatClient chatClient = openAiClient
.GetChatClient("gpt-4o-mini")
.AsIChatClient();

// ══════════════════════════════════════════════════════
// PART 1 — Same Prompt, Different Temperatures
// ══════════════════════════════════════════════════════
Console.WriteLine("═══ Part 1: Same Prompt, Different Temperatures ═══\n");

string adoptionPrompt =
"Write a one-sentence catchy adoption slogan for this pet:\n" +
PetProfiles[0]; // Luna the tabby cat

float[] temperatures = [0.0f, 0.3f, 0.7f, 1.0f];

foreach (float temp in temperatures)
{
Console.WriteLine($"── Temperature: {temp:F1} ──");

var options = new ChatOptions { Temperature = temp };

// Generate two responses at the same temperature to show consistency vs. variation
for (int attempt = 1; attempt <= 2; attempt++)
{
var response = await chatClient.GetResponseAsync(adoptionPrompt, options);
Console.WriteLine($" Attempt {attempt}: {response.Text}");
}

Console.WriteLine();
}

// ══════════════════════════════════════════════════════
// PART 2 — Low Temperature for Factual Tasks
// ══════════════════════════════════════════════════════
Console.WriteLine("═══ Part 2: Low Temperature for Factual Tasks ═══\n");

string factualPrompt =
"Extract the following fields from this pet profile and return them as a " +
"comma-separated list: Name, Species, Age, Favorite Activity.\n\n" +
PetProfiles[1]; // Baxter the golden retriever

Console.WriteLine($"Prompt: {factualPrompt}\n");

var lowTempOptions = new ChatOptions { Temperature = 0.0f };

for (int i = 1; i <= 3; i++)
{
var response = await chatClient.GetResponseAsync(factualPrompt, lowTempOptions);
Console.WriteLine($" Run {i}: {response.Text}");
}

Console.WriteLine("\n→ Low temperature produces consistent, predictable output for data extraction.\n");

// ══════════════════════════════════════════════════════
// PART 3 — High Temperature for Creative Tasks
// ══════════════════════════════════════════════════════
Console.WriteLine("═══ Part 3: High Temperature for Creative Tasks ═══\n");

string creativePrompt =
"Write a short, playful 2-sentence story about this pet finding a forever home:\n\n" +
PetProfiles[2]; // Pepper the rabbit

Console.WriteLine($"Prompt: {creativePrompt}\n");

var highTempOptions = new ChatOptions { Temperature = 1.0f };

for (int i = 1; i <= 3; i++)
{
var response = await chatClient.GetResponseAsync(creativePrompt, highTempOptions);
Console.WriteLine($" Story {i}: {response.Text}");
Console.WriteLine();
}

Console.WriteLine("→ High temperature produces diverse, creative variations each time.\n");

// ══════════════════════════════════════════════════════
// PART 4 — Temperature Comparison Table
// ══════════════════════════════════════════════════════
Console.WriteLine("═══ Part 4: Temperature Comparison Table ═══\n");

Console.WriteLine($"{"Temperature",-13} {"Best For",-30} {"Trait"}");
Console.WriteLine(new string('─', 70));
Console.WriteLine($"{"0.0",-13} {"Data extraction, classification",-30} {"Deterministic, consistent"}");
Console.WriteLine($"{"0.1 – 0.3",-13} {"Summaries, Q&A, code generation",-30} {"Focused, reliable"}");
Console.WriteLine($"{"0.4 – 0.6",-13} {"Balanced tasks, drafting",-30} {"Moderate variety"}");
Console.WriteLine($"{"0.7 – 1.0",-13} {"Creative writing, brainstorming",-30} {"Diverse, imaginative"}");
}
}

Reference

Microsoft Learn – LLM Fundamentals (Temperature and Determinism)


Share this lesson:

Details

Category
Microsoft Agent Framework
Sub: Generative AI ( Tutorial 1 )
View All Courses Back to Lessons

Navigation

Progress 49 / 49
Start 100% Complete
Context Window: The LLM's Working Memory
Statistics
4 Lessons in Generative AI ( Tutorial 1 )
10 SubCategories in Microsoft Agent Framework
49 Total Lessons in Microsoft Agent Framework