Mastering Ollama Modelfile: Build and Customize Your Own LLM

Getting the Base Model's Modelfile

Use the command ollama show --modelfile <model_name> to display the Modelfile of an existing model. For example, to view the Modelfile of phi4-mini:3.8b, run: ollama show --modelfile phi4-mini:3.8b This prints the Modelfile definition used by the model.

Modelfile Structure Breakdown

The following excerpt shows the generated Modelfile for phi4-mini:3.8b. Key directives are annotated:

# Modelfile generated by "ollama show"
# To build a new Modelfile based on this, replace FROM with:
# FROM phi4-mini:3.8b

# --- Directive Overview ---

# 1. FROM directive
# Specifies the base model. Ollama starts building the custom model from this.
FROM C:\Users\heish\.ollama\models\blobs\sha256-3c168af1dea0a414299c7d9077e100ac763370e5a98b3c53801a958a47f0a5db

# 2. TEMPLATE directive
# Defines the prompt template using Go template syntax.
TEMPLATE """{{- /* start template */ -}}
{{- if or .System .Tools }}<|system|>{{ if .System }}{{ .System }}{{ end }}
{{- if .Tools }}{{ if not .System }}You are a helpful assistant with some tools.{{ end }}<|tool|>{{ .Tools }}<|/tool|><|end|>
{{- end }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
{{- if ne .Role \"system\" }}<|{{ .Role }}|>{{ .Content }}
{{- if .ToolCalls }}<|tool_call|>[{{ range .ToolCalls }}{"name":"{{ .Function.Name }}","arguments":{{ .Function.Arguments }}{{ end }}]<|/tool_call|>
{{- end }}
{{- if not $last }}<|end|>
{{- end }}
{{- if and (ne .Role \"assistant\") $last }}<|end|><|assistant|>
{{- end }}
{{- end }}
{{- end }}
"""

# 3. LICENSE directive

LICENSE """Microsoft.
Copyright (c) Microsoft Corporation.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the \"Software\"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE."""

# Additional common directives (examples)

# PARAMETER directive: set model parameters such as temperature or stop sequences
PARAMETER temperature 0.7
PARAMETER stop "<|end|>"
PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"

# SYSTEM directive: define a default system prompt
SYSTEM """You are a helpful AI assistant."""

# ADAPTER directive: apply a LoRA adapter
ADAPTER ./my-lora-adapter

Creating Your Own Modelfile

Choose a base model : Use the FROM directive to specify the model you want to extend.

Define a prompt template (optional) : Use TEMPLATE to customize how system messages, user input, history, and tool calls are formatted. If no special format is needed, you can inherit the base model's template.

Set parameters (optional) : Use PARAMETER to adjust behavior such as creativity (temperature) or stop tokens.

Add a system prompt (optional) : Use SYSTEM to establish a default role or behavior for the model.

Apply an adapter (optional) : If you have trained a LoRA adapter, reference it with ADAPTER.

Build the model : Save the Modelfile (e.g., as MyCustomModel) and run ollama create MyCustomModel -f ./MyCustomModel to compile the custom model.

Example Modelfile based on the original model:

# Based on original model
FROM C:\Users\heish\.ollama\models\blobs\sha256-3c168af1dea0a414299c7d9077e100ac763370e5a98b3c53801a958a47f0a5db

# Custom template that outputs strict JSON
TEMPLATE """{{- if .Tools }}<|system|>You are a helpful assistant with tool access. Tools: {{ .Tools }}<|end|>{{ end }}
<|user|>{{ (index .Messages 0).Content }}<|end|><|assistant|>{{ if .ToolCalls }}{"tool_calls": [{{ range $index, $tc := .ToolCalls }}{{ if $index }}, {{ end }}{"type":"function","function":{"name":"{{ $tc.Function.Name }}","arguments":{{ $tc.Function.Arguments }}}}{{ end }}]}{{ else }}No tools needed.{{ end }}<|end|>"""

# System prompt to restrict simulated behavior
SYSTEM """You are a helpful assistant that can check weather information.
When users ask about weather, use the get_weather function tool to fetch real data.
Format the response in a natural, friendly way based on the tool's output. Do not simulate tool responses unless instructed."""

By understanding these core directives and their syntax, you can start creating and customizing Ollama models to meet specific requirements.