LLM Sampling Parameter Tuning: Best Practices for Temperature, Top‑p, Top‑k, and Min‑p

Key takeaway

Adjust temperature first, then pick either top_p (soft limit) or top_k (hard limit), and finally fine‑tune min_p only if you need a noise filter.

Why sampling matters

Even with the same model, different values for the four main sampling knobs ( temperature, top_p, top_k, min_p) can make the output feel overly deterministic or chaotic. Most failures are caused by mis‑configured sampling rather than model quality.

Parameter semantics

temperature : scales logits before softmax. T < 1 (e.g., 0.3) sharpens the distribution (more deterministic); T = 1 leaves it unchanged; T > 1 flattens it (more diverse).

top_k : keeps only the K most probable tokens – a hard cap on the candidate pool.

top_p : keeps the smallest set of tokens whose cumulative probability reaches p – a soft, probability‑aware cap.

min_p : discards any token whose probability is below min_p × max_prob. It acts as a “noise gate”.

Recommended tuning order

Decide whether the task prioritises accuracy (stable) or creativity (diverse).

Set temperature to a baseline that matches the priority.

Choose either top_p or top_k to control the candidate pool size.

Optionally adjust min_p (and frequency/presence penalties) to clean up noise or repetition.

Change only one knob at a time so the effect remains observable.

Scenario‑specific recommendations

Code generation

temperature

: 0.0 – 0.3 top_p / top_k: 0.1 – 0.5 or

K=10 – 30

min_p

: 0.01 – 0.03 frequency_penalty: 0.0 – 0.1

Python example:

code_params = {
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Write a Python function to quicksort a list"}],
    "temperature": 0.2,
    "top_p": 0.3,
    "frequency_penalty": 0.0,
}

JavaScript example (OpenAI SDK):

const { text } = await generateText({
  model: openai("gpt-4o"),
  prompt: "Write a Python function to quicksort a list",
  temperature: 0.2,
  topP: 0.3,
  frequencyPenalty: 0.0,
});

Knowledge QA / Fact lookup

temperature

: 0.0 – 0.4 top_p: 0.5 – 0.8 min_p: 0.01 – 0.05

Both frequency_penalty and presence_penalty: 0.0

Customer dialogue / Service bots

temperature

: 0.5 – 0.8 top_p: 0.85 – 0.95 min_p: 0.01 – 0.05 frequency_penalty: 0.0 – 0.3 presence_penalty: 0.0 – 0.3

Creative writing / Brainstorming

temperature

: 0.8 – 1.2 (up to 1.5 for poetry) top_p: 0.9 – 0.95 min_p: disabled (0.0) frequency_penalty: 0.3 – 0.7 presence_penalty: 0.5 – 1.0

Experimental methodology

Fix a prompt and vary only one parameter at a time.

Run each configuration 5‑10 times.

Observe stability, error rate, style consistency, and token repetition.

This turns intuition into data‑driven decisions.

Cross‑platform parameter differences

OpenAI : No top_k; temperature range 0‑2; full support for frequency_penalty and presence_penalty.

Anthropic Claude : temperature 0‑1 (default 1.0); can use either temperature or top_p (mutually exclusive); supports top_k.

DeepSeek : Supports top_k; temperature 0‑2 (default 1.0); full penalty support.

Google Gemini : temperature 0‑2 (default ≈0.7); no penalty parameters; default top_k = 40.

Kimi : temperature 0‑1; some models lock other knobs; uses max_completion_tokens instead of max_tokens.

Llama (self‑hosted) : Typical defaults temperature=0.6, top_p=0.95, min_p=0.01; use penalties cautiously.

Practical checklist (quick reference)

Code generation : temperature 0.0‑0.3, top_p 0.1‑0.5 or top_k 10‑30, min_p 0.01‑0.03, frequency_penalty 0.0‑0.1.

Math reasoning : temperature 0.0‑0.2, top_p 0.1‑0.4, top_k 10‑20, min_p 0.01‑0.03.

Knowledge QA : temperature 0.0‑0.4, top_p 0.5‑0.8, min_p 0.01‑0.05.

Customer dialogue : temperature 0.4‑0.7, top_p 0.85‑0.95, min_p 0.01‑0.05, frequency_penalty 0.0‑0.2, presence_penalty 0.0‑0.2.

General chat : temperature 0.6‑0.8, top_p 0.9‑0.95, min_p 0.01‑0.05, frequency_penalty 0.0‑0.3, presence_penalty 0.0‑0.3.

Translation : temperature 0.3‑0.6, top_p 0.8‑0.9, min_p 0.0‑0.03, frequency_penalty 0.0‑0.1, presence_penalty 0.0‑0.1.

Summarization : temperature 0.3‑0.6, top_p 0.85‑0.95, min_p 0.0‑0.05, frequency_penalty 0.1‑0.3, presence_penalty 0.0‑0.2.

Creative writing : temperature 0.8‑1.2, top_p 0.9‑0.95, top_k 40+, frequency_penalty 0.3‑1.0, presence_penalty 0.5‑1.0.

Poetry : temperature 1.0‑1.5, top_p 0.95‑1.0, frequency_penalty 0.3‑0.6, presence_penalty 0.5‑0.8.

Understanding the knobs

Temperature mathematically rescales logits: probs = softmax(logits / temperature). Lower values ( T < 1) make high‑probability tokens dominate; higher values flatten the distribution, allowing more diverse tokens.

top_k vs top_p : top_k imposes a hard limit on the number of candidates (e.g., keep the 20 most likely tokens). top_p adapts the limit based on cumulative probability, keeping fewer tokens when the model is confident and more when it is uncertain. Use top_k for tasks that need a strict shortlist (code completion, math); use top_p for natural‑language tasks where flexibility is beneficial.

min_p filters out low‑probability tail tokens. For example, with min_p=0.05 and a top token probability of 0.9, any token below 0.9 × 0.05 = 0.045 is discarded. This is useful when you want to remove obvious noise without shrinking the candidate pool.

Step‑by‑step tuning workflow

Identify the task type (accuracy‑focused, balanced, or creative).

Set temperature to a value that matches the desired determinism.

Pick top_p for soft, context‑aware pruning or top_k for a hard shortlist.

If the output still contains junk or repeats, experiment with min_p, frequency_penalty, and presence_penalty one at a time.

Validate the configuration by running a fixed prompt 5‑10 times and measuring stability, error rate, style, and repetition.

This disciplined approach avoids the “tweak everything at once” trap and yields reproducible, task‑aligned results.

Final thoughts

Sampling parameters are engineering levers, not mystical knobs. Choose a baseline temperature based on whether you need stability or creativity, then adjust the candidate‑pool control ( top_p or top_k) and finally apply fine‑grained filters ( min_p, penalties) as needed. The workflow above, combined with the scenario‑specific ranges, lets you arrive at a stable configuration without guesswork.