Most Ollama cheat sheets online still tell you to quantize with ollama create -q q4_K_M and pin models with --keepalive -1. On the current release both fail, and so does the ADAPTER line they show for LoRA files. This page is the Ollama commands reference we keep next to the terminal: every CLI command and flag, the REPL, the native, OpenAI and Anthropic APIs, the environment variables and the Modelfile, each with the output you should see.
You can also grab it as a one-page PDF. If Ollama is not on the machine yet, start with the Ollama install guide for Ubuntu 26.04. Every command below was run on Ollama 0.40.14 on an Ubuntu 24.04 VM with 4 vCPUs and no GPU in November 2026, so the token rates are CPU rates and every error message is verbatim.
Ollama commands at a glance
Ollama ships 14 subcommands plus a bare ollama that opens a menu. The table maps each one to what it does and the flags you’ll reach for most. ollama --help prints the same list, and ollama COMMAND --help prints every flag for one command.
| Command | What it does | Flags worth knowing |
|---|---|---|
ollama | First-run welcome, then the launch menu (needs a terminal) | none |
ollama run MODEL [PROMPT] | Chat interactively or answer one prompt and exit | --think, --hidethinking, --format, --keepalive, --verbose, --nowordwrap, --dimensions, --truncate, --ask, --choice, --level, --insecure |
ollama pull MODEL | Download a model, or update one you already have | --insecure |
ollama list (alias ls) | Models on disk: NAME, ID, SIZE, MODIFIED | none |
ollama ps | Models in memory: SIZE, PROCESSOR, CONTEXT, RUNNER, UNTIL | none |
ollama show MODEL | Architecture, context length, quantization, capabilities, defaults | --parameters, --modelfile, --template, --system, --license, -v |
ollama stop MODEL | Unload a model now, without stopping the server | none |
ollama cp SRC DST | Copy a model under a new name (no extra disk) | none |
ollama rm MODEL [MODEL...] | Delete one or more models | none |
ollama create NAME | Build a model from a Modelfile or safetensors | -f, -q, --draft-quantize, --combine, --force |
ollama push NAME | Upload a model to ollama.com (needs sign-in) | --insecure |
ollama serve (alias start) | Run the server in the foreground | configured by environment variables |
ollama signin / signout | Link or unlink this machine and your ollama.com account | none |
ollama launch [APP] | Point Claude Code, Codex, OpenCode and 15 other tools at Ollama | --model, --config, --restore, -y |
ollama -v | Print the server and client version | --verbose, --nowordwrap also work globally |
Two commands people expect are missing. There’s no ollama update (you re-run the installer) and no ollama prune, even though a popular gist still lists one. The server already prunes unused blobs on every start unless you set OLLAMA_NOPRUNE.
Install, update and check the Ollama version
On Linux, one script installs and updates Ollama. It drops the binary in /usr/local/bin, unpacks the runners into /usr/local/lib/ollama, creates an ollama system user and starts a systemd unit on 127.0.0.1:11434. Running it again later is how you update, and it cleans out the old runner libraries first:
curl -fsSL https://ollama.com/install.sh | shA clean run on a box with no GPU ends with a CPU-only warning, which is expected:
>>> Installing ollama to /usr/local
>>> Downloading ollama-linux-amd64.tar.zst
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.That download is a 1.4 GB tarball, because it carries both CUDA 12 and CUDA 13 runners (1.3 GB and 1.1 GB unpacked) even on a machine without a GPU. The script streams it straight into tar, so it cannot resume. Our first three attempts died (one at 60%, one at 1%) and left a working ollama binary behind but no service. That half-install fails in a confusing way, covered in the errors section at the end. If your link is flaky, re-run the script until it prints Install complete. On minimal images, install zstd first (sudo apt-get install zstd), because the script stops without it.
Check the version and the API. ollama -v reports the server and the client separately when they differ, and the root URL answers with plain text, which makes it the cheapest health check you can put in a script:
ollama -v
curl -s http://localhost:11434/
curl -s http://localhost:11434/api/versionAll three agree on a healthy install:
ollama version is 0.40.14
Ollama is running
{"version":"0.40.14"}To pin a release or roll back after a bad update, pass OLLAMA_VERSION to the same script. It becomes a ?version= parameter on the download URL, and ollama.com redirects that to the matching GitHub release asset, so any published tag works:
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.40.5 sh # pin: rollback target, see https://github.com/ollama/ollama/releasesOn macOS and Windows the desktop app updates itself and puts the same ollama CLI on your path, and every command on this page works there too. The Linux-only parts are the systemd unit and the /usr/share/ollama paths. To remove Ollama from Linux completely, stop the unit and delete what the installer created:
sudo systemctl disable --now ollama
sudo rm /etc/systemd/system/ollama.service
sudo rm -rf /etc/systemd/system/ollama.service.d
sudo systemctl daemon-reload
sudo rm -r /usr/local/lib/ollama
sudo rm /usr/local/bin/ollama
sudo userdel ollama
sudo groupdel ollama
sudo rm -r /usr/share/ollamauserdel prints group ollama not removed because it has other members, because the installer added your own user to that group, which is why the groupdel line is there. The last line deletes every model the service downloaded. If you moved the store with OLLAMA_MODELS, delete that directory as well, and remove ~/.ollama if you ever ran ollama serve as your own user.
Manage local models
Model names follow name:tag, and a missing tag means latest. ollama pull downloads a model, and running it again on a model you already have checks the registry and fetches only what changed. That is how you update a model. A misspelled name fails with Error: pull model manifest: file does not exist, from both pull and run.
ollama pull qwen3.5:4b
ollama pull nomic-embed-textollama list and its alias ollama ls print the same table. A copy shares its ID with the source, which is your hint that both names point at the same weights on disk:
ollama lsHere mymodel is a copy of qwen3.5:4b, so both show d8b0f5e9760c:
NAME ID SIZE MODIFIED
mymodel:latest d8b0f5e9760c 3.3 GB Less than a second ago
nomic-embed-text:latest 0a109f422b47 274 MB 3 minutes ago
qwen3.5:4b d8b0f5e9760c 3.3 GB 3 minutes agoBefore you build anything on a model, read ollama show. It tells you the native context length, the quantization, whether the model can call tools or think, and which sampling defaults it ships with. Those defaults override Ollama’s generic ones, so they matter when two models behave differently on the same prompt:
ollama show qwen3.5:4bQwen 3.5 4B reports a 262,144-token native context, vision, tools and thinking, with thinking on by default:
Model
architecture qwen35
parameters 4.2B
context length 262144
embedding length 2560
quantization Q4_K_M
requires 0.30.0
renderer qwen3.5
parser qwen3.5
template source renderer_parser
Capabilities
completion
vision
tools
thinking
levels false, true
default true
Projector
architecture clip
parameters 333.51M
embedding length 1024
dimensions 2560
Parameters
presence_penalty 1.5
temperature 1
top_k 20
top_p 0.95
License
Apache License
Version 2.0, January 2004
...The “context length” there is what the model supports, not what Ollama loads. Ollama loads 4,096 tokens on a machine with under 24 GiB of VRAM, 32,768 at 24 to 48 GiB and 262,144 above that, unless you raise it yourself. The flags pull out one section each, which is handy in scripts:
ollama show qwen3.5:4b --parameters
ollama show qwen3.5:4b --modelfile
ollama show qwen3.5:4b --template
ollama show qwen3.5:4b --system
ollama show qwen3.5:4b --license
ollama show qwen3.5:4b -v-v appends the raw GGUF metadata, including the base model’s Hugging Face repo and license link. --system prints nothing for stock library models, because they ship without a system prompt, and --template on newer models prints only {{ .Prompt }}. Prompt formatting has moved into the built-in RENDERER and PARSER that --modelfile reveals.
Copy, delete and unload work the way you’d expect, with two catches. ollama cp on a model whose manifest lists both MLX and llama.cpp builds, such as qwen3.5:4b, keeps only the build you pulled and says so. ollama rm takes several names but stops at the first one that does not exist, so the names after a typo survive:
ollama cp qwen3.5:4b mymodel:latest
ollama rm x1 x2 x3
ollama stop qwen3.5:4bWith x2 missing, x1 is deleted and the command stops at the error, so anything listed after it is untouched:
copied 'qwen3.5:4b' to 'mymodel:latest'
Warning: discarded mlx (not pulled locally); copy contains llamacpp
deleted 'x1'
Error: model 'x2' not foundollama ps answers the question “is it on the GPU”. It lists what’s loaded, how much memory it takes, where it runs, the context it was loaded with and when it unloads. The RUNNER column is new in the 0.40 line and shows which engine serves the model: llamacpp, ggml for the older engine, or mlx on Apple Silicon.
ollama psOn the CPU test box the whole model sits in system RAM:
NAME ID SIZE PROCESSOR CONTEXT RUNNER UNTIL
qwen3.5:4b d8b0f5e9760c 3.1 GB 100% CPU 4096 llamacpp 4 minutes from nowA split such as 48%/52% CPU/GPU means the model plus its context did not fit in VRAM, and generation slows to near CPU speed. Lower the context or pick a smaller quantization before you blame the GPU. The VRAM sizing guide has the numbers per model size. On Linux the service keeps models in /usr/share/ollama/.ollama/models, and ollama serve run as your own user uses ~/.ollama/models, which is why a model pulled one way is missing the other way.
Run models from the command line
ollama run MODEL on its own opens an interactive chat. Add a prompt as the next argument, or pipe text in, and it answers once and exits, which is what you want in scripts. Qwen 3.5 thinks by default, so these examples pass --think=false to get the answer without a reasoning block. Pass the prompt as an argument and pipe a file at the same time, and Ollama appends the file to your instruction:
ollama run qwen3.5:4b --think=false "What port does PostgreSQL use? One sentence."
echo "What is the capital of France? One word." | ollama run qwen3.5:4b --think=false
cat /var/log/apt/history.log | ollama run qwen3.5:4b --think=false --nowordwrap "Which packages were installed? One line."The third line took 2m12s on the test box, and the cost scales with what you pipe in: the model reads the whole file as prompt tokens, so pipe tail -n 50 of a big file rather than all of it. Redirected output on 0.40.14 comes back as one clean line with no escape bytes; the old versions that broke lines at about 75 columns and sent ESC[nD ESC[K to redraw words are gone. --nowordwrap is still worth adding whenever the output goes to a file or another program, because it does no harm there and keeps a terminal from rewrapping your capture. This is the redirected answer, byte for byte:
PostgreSQL typically listens on port **5432** by default to handle incoming connections from clients. This specific port number is configured as the standard setting found in the server's `postgresql.conf` file and documented in the official release notes.Spinner codes still go to stderr, so don’t merge it with 2>&1 when you capture output. Thinking models add another wrinkle to scripts, because they print a “Thinking…” block before the answer. The flags that shape a one-shot run:
--think=falseor--think=true, andhigh,medium,lowon models that list those levels--hidethinkingto print only the answer--format jsonfor JSON output--verbosefor load time and tokens per second--keepalive 30mto keep the model in memory after the answer
--hidethinking only hides the trace. The model still generates it, so on slow hardware --think=false is the switch that saves time. On our CPU box the same 13 × 17 question took 24.2 seconds with --hidethinking and 0.8 seconds with --think=false, and both printed 221. Asking for a level the model doesn’t list, such as --think=high on Qwen 3.5 (levels false, true), didn’t error either. It printed a “Thinking…” block, the same as --think=true.
--format json constrains the output to valid JSON, but the model still picks the structure, so describe the shape you want in the prompt:
echo "List 3 Linux distributions with their first release year. Return JSON with a distributions array of objects with name and year." | ollama run qwen3.5:4b --think=false --format jsonValid JSON came back with the requested keys. The years are the model’s own and need checking, because the model will state them confidently even when it is wrong:
{
"distributions": [
{
"name": "Red Hat Enterprise Linux",
"year": 1993
},
{
"name": "Ubuntu",
"year": 2004
},
{
"name": "Linux Mint",
"year": 2006
}
]
}--verbose is the quickest benchmark you have. It splits prompt processing from generation, and the first run also shows how long the model took to load:
ollama run qwen3.5:4b --think=false --nowordwrap --verbose "Name the default SSH port. Reply with just the number."With the model already resident from the run before it, load time collapses to 493 microseconds and the whole call finishes in under a second:
22
total duration: 968.190644ms
load duration: 493.044µs
prompt eval count: 24 token(s)
prompt eval duration: 674.588ms
prompt eval rate: 35.58 tokens/s
eval count: 3 token(s)
eval duration: 237.829ms
eval rate: 12.61 tokens/sThat 12.61 tokens per second is on a 3-token answer, and short answers flatter the rate. Read the eval rate from a response of 50 tokens or more before you compare machines. If that number is what pushes you toward new hardware, the GPU guide for local LLMs ranks cards by measured throughput.
Keep a model loaded or unload it
Models unload after 5 minutes idle. --keepalive changes that for the run, but the CLI only accepts a Go duration, so the -1 that every older cheat sheet prints fails with Error: time: missing unit in duration "-1", with or without the equals sign. Give the negative value a unit instead:
ollama run qwen3.5:4b --keepalive -1s --think=false --nowordwrap "Say OK"
ollama run qwen3.5:4b --keepalive 1h30m --think=false --nowordwrap "Say OK"
ollama run qwen3.5:4b --keepalive 0 --think=false "Say OK"After the first command ollama ps shows the model pinned, after the second about an hour, and after the third it is gone as soon as the answer prints:
NAME ID SIZE PROCESSOR CONTEXT RUNNER UNTIL
qwen3.5:4b d8b0f5e9760c 3.1 GB 100% CPU 4096 llamacpp ForeverThe server-wide OLLAMA_KEEP_ALIVE variable is more forgiving: it accepts a duration or a bare number of seconds, and any negative value means forever. The API’s keep_alive takes the integer -1 too, but the string "-1" fails the same way the CLI does. Use 0 anywhere to unload immediately.
Get embeddings from the CLI
Run an embedding model and ollama run prints the vector as a JSON array instead of text. nomic-embed-text returns 768 dimensions, and --dimensions truncates that for a smaller vector store:
echo "the quick brown fox" | ollama run nomic-embed-text --dimensions 256 | python3 -c "import sys,json; v=json.load(sys.stdin); print(len(v), v[:3])"The vector comes back at the requested length:
256 [-0.074136734, 0.017474521, -0.21942759]Input longer than the model’s context is cut silently by default, and on 0.40.14 the CLI flag meant to stop that does not work. We piped a 100 KB file into nomic-embed-text with --truncate=false and got the same vector as without it, even though ollama run --help says the flag errors instead. The switch that does refuse is the API’s truncate field: POST /api/embed with "truncate": false answers {"error":"the input length exceeds the context length"}. That’s fine for search, but it hides data loss in a RAG pipeline, so chunk your documents before embedding, as the Ollama and pgvector RAG build does.
Ask a decision model with –ask
Decision models such as nimble do not write text. They score a question about the input you pipe in and print a label with a confidence, which suits ticket triage and routing. --ask alone gives yes or no, each --choice adds a label to pick from, and --level scores the input on a scale you define, lowest first:
ollama run nimble --ask "Is a refund being requested?" < ticket.txt
ollama run nimble --ask "Which team should handle this?" --choice billing="Payments and refunds" --choice technical="Software problems" --choice shipping="Delivery and tracking" < ticket.txt
ollama run nimble --ask "How urgent is this?" --level low --level medium --level high < ticket.txtWe fed it a short complaint about an order stuck at “label created” that ends with a refund request. The three questions came back in that order, and the level form also prints the score on the scale:
yes 100%
shipping 63%
medium 55% 1.2 / 2The first call took 12m37s on the 8 GB test box because nimble is a 9B model at Q8_0 that loads at 10 GB, and the load crawled through RAM plus swap while Qwen 3.5 was still resident; ollama ps then showed both models sharing 13.4 GB. A warm ask with both loaded still took 3m18s. On a box this size, stop what you don’t need with ollama stop first, or set OLLAMA_MAX_LOADED_MODELS=1 so the server evicts idle models before it loads the next one. A plain ollama run nimble "hello" refuses with Error: nimble answers questions about text, e.g. ollama run nimble --ask "Is a refund being requested?" < ticket.txt, because decision models only answer --ask.
Use the interactive REPL and slash commands
Inside ollama run, lines starting with a slash configure the session instead of going to the model. /? lists them, /set and /set parameter list their own options, and /? shortcuts prints the key bindings. The commands we used most:
| Slash command | Effect |
|---|---|
/set parameter num_ctx 8192 | Raise the context for this session (also temperature, seed, top_k, top_p, min_p, num_predict, repeat_penalty, repeat_last_n, num_gpu, stop) |
/set system You are terse. | Session system prompt |
/set think, /set nothink | Thinking on or off |
/set verbose, /set quiet | Timing stats after each answer, or not |
/set format json, /set noformat | JSON mode on or off |
/set nohistory, /set nowordwrap | Stop saving history, stop wrapping |
/show info, /show parameters, /show system | Inspect the model and what you changed (also license, modelfile, template) |
/save NAME | Save the session settings as a new model |
/load NAME | Switch to another model or saved session |
/clear | Forget the conversation so far |
/bye or Ctrl+D | Exit |
""" | Start and end a multi-line message |
| Ctrl+G | Write the prompt in $OLLAMA_EDITOR |
| Ctrl+C | Stop the current answer |
/show parameters splits what the model ships with from what you set, so you can see which value wins. After /set parameter num_ctx 8192 and /set parameter temperature 0.2 on Qwen 3.5:
Model defined parameters:
presence_penalty 1.5
temperature 1
top_k 20
top_p 0.95
User defined parameters:
num_ctx 8192
temperature 0.2/save is meant to turn that tuned session into a model you can call by name, and it does on llama3.2:1b and on custom models. On the library qwen3.5:4b it failed with Error: pull model manifest: file does not exist and dropped us out of the REPL, and copying the model first with ollama cp did not help. The source explains it: /save builds the new model FROM the model’s parent_model, and the library qwen3.5:4b reports model.gguf there, so Ollama tries to pull a model called model.gguf. /api/show on our box gave parent_model values of model.gguf for qwen3.5:4b, an empty string for llama3.2:1b and qwen3.5:4b for a Modelfile-built model, which matches exactly which saves worked. Until that’s fixed, use a Modelfile when session settings need to last.
/load on an embedding model fails with error: 400 Bad Request: "nomic-embed-text" does not support thinking, because the session you are leaving has thinking enabled and the target model cannot. The REPL keeps the prompt, and the old model stays loaded. Bare ollama with no arguments is a different program: a full-screen menu that starts with a “Welcome to Ollama!” page on first run and then lists the ollama launch integrations. It needs a real terminal; over a plain SSH command with no TTY it falls back to printing the standard CLI help and exits 0.
Run Ollama as a Linux service
The installer’s unit runs ollama serve as the ollama user and restarts it on failure, so you manage it with the usual systemd commands. Each loaded model runs as a llama-server child process under the unit, which systemctl status shows together with its context size:
sudo systemctl restart ollama
systemctl status ollama --no-pager
journalctl -u ollama -fWith a model loaded after the overrides below, the status shows the drop-in and the runner process with -c 8192. The runner line is trimmed at the end:
● ollama.service - Ollama Service
Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled)
Drop-In: /etc/systemd/system/ollama.service.d
└─override.conf
Active: active (running) since Thu 2026-11-05 22:09:11 UTC; 7s ago
Main PID: 4714 (ollama)
Tasks: 24 (limit: 9483)
Memory: 3.6G (peak: 3.6G)
CPU: 8.958s
CGroup: /system.slice/ollama.service
├─4714 /usr/local/bin/ollama serve
└─4794 /usr/local/lib/ollama/llama-server --model /usr/share/ollama/.ollama/models/blobs/sha256-a9d0a3e1d8c91732a3812becd31f8d4a8ff57414903f226306069f0437e2ac50 --port 45339 --host 127.0.0.1 --no-webui --offline -c 8192 -np 1 --log-verbosity 4 --no-log-prefix --no-log-timestamps --no-jinja --chat-template chatml --mmproj /usr/share/ollama/.ollama/models/blobs/sha256-e56a899b1540860385465756dd308c84417410b94dbc1c380adffd20a921ca60 --load-mode none --flash-attn auto -b 1024 -ub 1024 --context-shift --keep 4 ...Settings go in environment variables, and for the service that means a systemd drop-in. Exporting them in your shell changes nothing, because the server runs under systemd as another user. That is the most common reason OLLAMA_HOST=0.0.0.0 “does not work”. Open the override editor:
sudo systemctl edit ollamaAdd the variables under a [Service] header. This set listens on every interface, loads models with an 8K context instead of the 4K CPU default, and keeps them for 30 minutes:
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="OLLAMA_KEEP_ALIVE=30m"Reload, restart, and confirm the service actually sees the values before you test anything else:
sudo systemctl daemon-reload
sudo systemctl restart ollama
systemctl show ollama -p Environment
ss -tlnp | grep 11434The listener moves from 127.0.0.1:11434 to *:11434, and the next model loads with CONTEXT 8192 and UNTIL 29 minutes from now in ollama ps:
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin OLLAMA_HOST=0.0.0.0:11434 OLLAMA_CONTEXT_LENGTH=8192 OLLAMA_KEEP_ALIVE=30m
LISTEN 0 4096 *:11434 *:*Ollama has no authentication by default, so anyone who can reach port 11434 can pull models, delete them and run inference on your hardware. Open the port only to the hosts that need it with the UFW firewall or firewalld, or put a reverse proxy with auth in front. From another machine, point the CLI at the server with OLLAMA_HOST:
OLLAMA_HOST=10.0.1.50:11434 ollama ps
curl -s http://10.0.1.50:11434/api/versionFrom our Mac on the same LAN, both commands worked once the override was active: the version call returned {"version":"0.40.14"} and ollama ps listed the model loaded on the VM.
Move the model store with OLLAMA_MODELS
Models fill the root disk fast, and moving them is the second most common service change. The new directory must belong to the ollama user, which is the step most guides skip. We created /data/ollama/models as root, pointed OLLAMA_MODELS at it in the override, and the service crash-looped with this in the journal:
Error: mkdir /data/ollama/models/blobs: permission denied: ensure path elements are traversableCreate the directory, copy the existing models across, and hand both to the service user before you restart. Every parent directory also needs to be traversable by ollama, which rules out a path under a locked-down home directory:
sudo mkdir -p /data/ollama/models
sudo rsync -a /usr/share/ollama/.ollama/models/ /data/ollama/models/
sudo chown -R ollama:ollama /data/ollamaAdd Environment="OLLAMA_MODELS=/data/ollama/models" to the override, then run sudo systemctl daemon-reload and sudo systemctl restart ollama. ollama list should show every model again. If it comes back empty, the service started on an empty directory, so check the copy and the path. Once it looks right, get the space back with sudo rm -rf /usr/share/ollama/.ollama/models. Leave the rest of /usr/share/ollama/.ollama alone, because id_ed25519 there is the server’s identity key for ollama signin.
Set Ollama environment variables
ollama serve --help prints the variables the server reads, but only some of their defaults. We took the rest from the release’s envconfig source and checked the ones that matter on the test box. Set them in the systemd override on Linux, with launchctl setenv plus an app restart on macOS, or as user variables on Windows.
| Variable | Default | What it changes |
|---|---|---|
OLLAMA_HOST | 127.0.0.1:11434 | Bind address for the server, target address for the CLI |
OLLAMA_MODELS | ~/.ollama/models (service: /usr/share/ollama/.ollama/models) | Where models are stored |
OLLAMA_CONTEXT_LENGTH | 4K, 32K or 256K by VRAM | Context loaded when a request doesn’t set num_ctx |
OLLAMA_KEEP_ALIVE | 5m | Idle time before unload; negative means forever, 0 means at once |
OLLAMA_NUM_PARALLEL | 1 | Requests one loaded model serves at the same time |
OLLAMA_MAX_LOADED_MODELS | 0 (auto: 3 per GPU, 3 on CPU) | Models kept in memory together; set 1 on small boxes |
OLLAMA_MAX_QUEUE | 512 | Queued requests before the server refuses more |
OLLAMA_FLASH_ATTENTION | auto | Force flash attention on or off |
OLLAMA_KV_CACHE_TYPE | f16 | q8_0 or q4_0 shrink the context’s memory |
OLLAMA_NO_CLOUD | off | Disable cloud models and web search |
OLLAMA_ORIGINS | localhost, 127.0.0.1, 0.0.0.0, app and editor schemes | Extra CORS origins for browser apps |
OLLAMA_DEBUG | off | 1 for debug logs, 2 for trace |
OLLAMA_LOAD_TIMEOUT | 5m | How long a stalled model load may take |
OLLAMA_GPU_OVERHEAD | 0 | VRAM bytes to leave free per GPU |
OLLAMA_SCHED_SPREAD | off | Spread every model across all GPUs |
OLLAMA_IGPU_ENABLE | auto (CUDA iGPUs and listed ROCm targets allowed) | Allow or block integrated GPUs |
OLLAMA_NOPRUNE | off | Keep unused blobs on startup |
OLLAMA_MAX_TRANSFER_STREAMS | 4 | Parallel streams for safetensors pulls and pushes |
OLLAMA_LLM_LIBRARY | auto | Force a specific runner library |
OLLAMA_CREATE_REMOTE | off | Send ollama create through the API even to a local server |
LLAMA_ARG_FIT, LLAMA_ARG_FIT_TARGET | on, auto | llama.cpp’s automatic memory fit and its free-VRAM margin (MiB) |
CUDA_VISIBLE_DEVICES and friends | all | Pick GPUs (HIP_, ROCR_, GGML_VK_VISIBLE_DEVICES) |
Three variables belong to the client, not the server: OLLAMA_HOST again (which server the CLI talks to), OLLAMA_EDITOR for Ctrl+G and OLLAMA_NOHISTORY. Some names that still show up in tutorials, such as OLLAMA_NUM_GPU and OLLAMA_TMPDIR, aren’t in the current server’s variable list, so don’t expect them to change anything. Per-request options in the API beat the variables, and a model’s own Modelfile parameters beat Ollama’s generic defaults.
Call the Ollama REST API with curl
Everything the CLI does goes through the HTTP API on port 11434, so anything you can do in the terminal you can script with curl. Responses stream by default. Add "stream": false to get one JSON object, and "think": false on thinking models when you only want the answer:
curl -s http://localhost:11434/api/chat -d '{
"model": "qwen3.5:4b",
"messages": [
{"role": "system", "content": "You are a Linux sysadmin. Be brief."},
{"role": "user", "content": "What is DNS?"}
],
"think": false,
"stream": false
}' | python3 -m json.toolThe reply sits in message.content, and all durations are nanoseconds. 72 tokens in 7.1 seconds of eval_duration is 10.1 tokens per second on this CPU:
{
"model": "qwen3.5:4b",
"created_at": "2026-11-05T22:08:17.044001323Z",
"message": {
"role": "assistant",
"content": "Domain Name System (DNS) is the internet's phonebook. It translates human-readable domain names (e.g., `google.com`) into IP addresses that computers use to locate servers. Without it, we'd need to memorize numerical strings like `142.250.185.79` instead of meaningful website names."
},
"done": true,
"done_reason": "stop",
"total_duration": 7942943105,
"load_duration": 3616950,
"prompt_eval_count": 31,
"prompt_eval_cached_count": 0,
"prompt_eval_duration": 705294000,
"eval_count": 72,
"eval_duration": 7098118000
}/api/generate takes a single prompt instead of messages and returns the text in response, plus a context array of token IDs you can ignore. With "think": true, the reasoning arrives in a separate top-level thinking field, never inside response (1,181 to 1,430 characters across our runs of the 13 × 17 question, so the length varies per run); on /api/chat the same reasoning lands in message.thinking instead, so you never have to strip it out of the answer.
For output your code will parse, pass a JSON schema in format. Unlike "format": "json", the schema fixes the field names and types, so the response always parses into the structure you expect:
curl -s http://localhost:11434/api/chat -d '{
"model": "qwen3.5:4b",
"stream": false,
"think": false,
"messages": [{"role":"user","content":"Pick a city and report plausible weather. Output JSON only."}],
"format": {
"type": "object",
"properties": {
"city": {"type": "string"},
"temp_c": {"type": "number"},
"condition": {"type": "string"}
},
"required": ["city","temp_c","condition"]
}
}' | python3 -c "import json,sys; print(json.load(sys.stdin)['message']['content'])"The content is a JSON string that matches the schema:
{
"city": "London",
"temp_c": 12,
"condition": "Light Rain"
}Tool calling works the same way on any model whose capabilities include tools. You describe the functions, and when the model wants one it returns tool_calls with parsed arguments instead of text. Your code runs the function and sends the result back as a tool message:
curl -s http://localhost:11434/api/chat -d '{
"model": "qwen3.5:4b",
"stream": false,
"think": false,
"messages": [{"role":"user","content":"What is the weather in Nairobi?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}]
}' | python3 -c "import json,sys; print(json.dumps(json.load(sys.stdin)['message'].get('tool_calls'), indent=2))"Qwen 3.5 4B picked the tool and filled the argument:
[
{
"id": "call_o956m64c",
"function": {
"index": 0,
"name": "get_weather",
"arguments": {
"city": "Nairobi"
}
}
}
]The rest of the native API, with what each call returned on the test box:
| Endpoint | Body or use | What came back |
|---|---|---|
POST /api/generate | model, prompt, options | response, timings, context |
POST /api/embed | model, input (string or list), dimensions | 2 inputs gave 2 × 768 floats |
GET /api/tags | local models (ollama list) | size, quantization, context, runner, capabilities |
GET /api/ps | loaded models (ollama ps) | size_vram, context_length, expires_at, runner |
POST /api/show | {"model": "qwen3.5:4b"} | capabilities, details, model_info, parameters, thinking, renderer, parser |
POST /api/pull | {"model": "all-minilm", "stream": false} | {"status":"success"} |
POST /api/copy | source, destination | HTTP 200 |
DELETE /api/delete | {"model": "qwen-backup"} | HTTP 200, or 404 model 'qwen-backup' not found |
POST /api/create | model, from, system, parameters | status stream ending success |
POST /api/push | {"model": "user/name"} | not run (needs ollama signin) |
GET /api/version | health check | {"version":"0.40.14"} |
GET /api/status | cloud switch state | {"cloud":{"disabled":false,"source":"none"}} |
Per-request settings go in options and override everything else for that call, including the context the model loads with. This request reloaded Qwen with an 8K context, and ollama ps showed CONTEXT 8192 straight after:
curl -s http://localhost:11434/api/generate -d '{"model": "qwen3.5:4b", "prompt": "Say OK", "think": false, "stream": false, "options": {"num_ctx": 8192, "temperature": 0, "seed": 42, "num_predict": 20}}'A request with a model name and no prompt only loads or unloads the model, which is how the API warms a model before traffic arrives. "keep_alive": -1 pins it, and "keep_alive": 0 unloads it and returns "done_reason":"unload":
curl -s http://localhost:11434/api/generate -d '{"model": "qwen3.5:4b", "keep_alive": -1}'
curl -s http://localhost:11434/api/generate -d '{"model": "qwen3.5:4b", "keep_alive": 0}'One trap on /api/embed: sending a chat model there doesn’t return a clean “not an embedding model” error. It returns the llama.cpp server’s message, This server does not support embeddings. Start it with `--embeddings`, which reads like a flag you forgot. The real fix is to use a model whose capabilities list embedding.
Use the OpenAI and Anthropic compatible endpoints
Ollama also speaks the OpenAI and Anthropic wire formats, so most SDKs and tools work by changing the base URL. curl needs no key at all (/v1/models returned 200 without one), and the SDKs that insist on one take any string, such as ollama. The endpoints we ran:
POST /v1/chat/completions, with streaming andstream_options.include_usagePOST /v1/responsesfor the OpenAI Responses APIPOST /v1/embeddingsandGET /v1/modelsPOST /v1/messagesfor the Anthropic Messages API
On thinking models, set "reasoning_effort": "none" in Chat Completions, or "reasoning": {"effort": "none"} in Responses, when you only want the answer. Without it, a small CPU model can spend its whole token budget reasoning:
curl -s http://localhost:11434/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3.5:4b",
"reasoning_effort": "none",
"messages": [{"role": "user", "content": "What port does SSH use? One sentence."}]
}' | python3 -m json.toolThe response is shaped exactly like OpenAI’s, including cached-token usage:
{
"id": "chatcmpl-52",
"object": "chat.completion",
"created": 1793916522,
"model": "qwen3.5:4b",
"system_fingerprint": "fp_ollama",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "SSH typically uses TCP port 22."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 21,
"prompt_tokens_details": {
"cached_tokens": 0
},
"completion_tokens": 10,
"total_tokens": 31
}
}The Anthropic endpoint takes the usual headers and a thinking block, which is what lets tools built for Claude run on a local model. The Claude Code cheat sheet covers the client side:
curl -s http://localhost:11434/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: ollama" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "qwen3.5:4b",
"max_tokens": 200,
"thinking": {"type": "disabled"},
"messages": [{"role": "user", "content": "What port does HTTPS use? One sentence."}]
}' | python3 -m json.toolBack comes a standard Messages API object:
{
"id": "msg_3af47311bf846b2f9f4342f0",
"type": "message",
"role": "assistant",
"model": "qwen3.5:4b",
"content": [
{
"type": "text",
"text": "HTTPS primarily uses port 443 by default for secure web communication over TCP/IP."
}
],
"stop_reason": "end_turn",
"usage": {
"input_tokens": 21,
"cache_read_input_tokens": 0,
"output_tokens": 19
}
}In Python, the official ollama package wraps the native API, and the openai package works against /v1 unchanged. We installed ollama 0.6.3 and openai 3.46.0 into a venv and ran this script against the server:
import ollama
from openai import OpenAI
# 1. Generate (single-turn)
res = ollama.generate(model="qwen3.5:4b", prompt="Say hi in 5 words or less.", think=False)
print(res["response"])
# 2. Chat (multi-turn)
res = ollama.chat(
model="qwen3.5:4b",
messages=[{"role": "user", "content": "Reply with one word: yes"}],
think=False,
)
print(res["message"]["content"], "eval_count:", res["eval_count"])
# 3. Streaming
for chunk in ollama.generate(model="qwen3.5:4b", prompt="List 3 colors, comma separated.",
think=False, stream=True):
print(chunk["response"], end="", flush=True)
print()
# 4. Embeddings
res = ollama.embed(model="nomic-embed-text", input=["hello", "world"])
print(len(res["embeddings"]), "vectors of dim", len(res["embeddings"][0]))
# 5. Tool calling
res = ollama.chat(
model="qwen3.5:4b",
messages=[{"role": "user", "content": "Weather in Nairobi?"}],
think=False,
tools=[{
"type": "function",
"function": {
"name": "get_weather",
"description": "Current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}],
)
for call in (res["message"].tool_calls or []):
print(call["function"]["name"], call["function"]["arguments"])
# 6. OpenAI SDK against the /v1 endpoint
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
r = client.chat.completions.create(
model="qwen3.5:4b",
messages=[{"role": "user", "content": "What port does SSH use? One short sentence."}],
reasoning_effort="none",
)
print(r.choices[0].message.content)All six calls returned in 14.6 seconds on the CPU box, with the models already warm. The first answer ignored the five-word limit, which is a reminder that small models treat length rules as suggestions:
Hello there, how are you doing?
yes eval_count: 2
Red, blue, green
2 vectors of dim 768
get_weather {'city': 'Nairobi'}
SSH uses TCP port 22 by default.The tool call came back with its arguments already parsed into a Python dict, so your code can pass them straight to the function it names. The OpenAI client needed nothing beyond the base URL and a placeholder key.
Build a custom model with a Modelfile
A Modelfile bakes a system prompt and default parameters into a named model, so every client gets the same behavior without repeating options. The new model reuses the base model’s weights, so it costs no extra disk. Create a working directory and the file:
mkdir -p ~/sysadmin-bot && cd ~/sysadmin-bot
vim ModelfileThis is the exact Modelfile we built and ran:
FROM qwen3.5:4b
SYSTEM "You are a Linux systems administrator. Answer in one or two lines and include the command."
PARAMETER temperature 0.3
PARAMETER num_ctx 8192Build it, then use it like any other model:
ollama create sysadmin-bot -f Modelfile
echo "How do I check free disk space?" | ollama run sysadmin-bot --think=false --nowordwrapThe build reuses four existing layers, writes two small new ones for the system prompt and parameters, and the answer follows the baked-in instruction:
using existing layer sha256:e56a899b1540860385465756dd308c84417410b94dbc1c380adffd20a921ca60
using existing layer sha256:a9d0a3e1d8c91732a3812becd31f8d4a8ff57414903f226306069f0437e2ac50
using existing layer sha256:b507b9c2f6ca642bffcd06665ea7c91f235fd32daeefdf875a0f938db05fb315
using existing layer sha256:50cbab8a892c5f2993b8c7351a99182507472def3b1374558308605d99b86b32
creating new layer sha256:5cd361cc11ba206d9e893bed3aa7c38c50a4a441526f7ac3971475e4e48d146f
creating new layer sha256:71a9e198c4d8769980baff8bcb3bd95e5deaffe2501dae334ad86c5f4a477d03
writing manifest
success
Run `df -h` to see disk usage in human-readable format, or `du -sh /path/to/dir` to check a specific directory's size.The directives you’ll use, and how each one behaved on the current release:
| Directive | Purpose | Behavior we saw |
|---|---|---|
FROM | Base model, a GGUF file or a safetensors directory (required) | ollama create with no Modelfile fails with no Modelfile or safetensors files found |
SYSTEM | Default system prompt | Shown by ollama show --system |
PARAMETER | Default options such as num_ctx, temperature, stop | A typo fails the build: unknown parameter 'temprature' |
TEMPLATE | Go template for the prompt format | Newer models use the built-in RENDERER and PARSER instead |
MESSAGE | Seed conversation turns | Few-shot examples without a long system prompt |
LICENSE | License text | Shown by ollama show --license |
REQUIRES | Minimum Ollama version | A local build with REQUIRES 0.99.0 still ran; pulls of the model are refused on older Ollama versions |
ADAPTER | LoRA adapter (removed) | Error: LoRA adapters are no longer supported |
PARAMETER values only set defaults, so a client that sends its own options still overrides them. Ollama’s generic defaults barely matter any more, because most library models ship their own (Qwen 3.5 sets temperature 1, top_k 20, top_p 0.95 and presence_penalty 1.5), and repeat_penalty now defaults to off. Run ollama show MODEL --parameters before you tune, so you know what you’re overriding.
Quantizing during create has also changed. -q q4_K_M used to shrink an F16 GGUF in one step. Now any -q on a GGUF base fails with Error: create-time quantization is only supported for safetensors imports; quantize GGUF models before importing, and the levels for safetensors imports are int4, int8, nvfp4, mxfp4 and mxfp8. For GGUF, quantize with llama.cpp’s llama-quantize first. The GGUF vs AWQ vs GPTQ comparison covers that tool and the levels, and the llama.cpp guide shows how to build and run llama.cpp itself.
Once Modelfiles stop being enough and you want to fine-tune or build agents on top of Ollama, Local AI Engineering with Ollama (paid link) by Aymen El Amri (FAUN.dev, 2026, 258 pages) is the most current book on that path. It goes from running and understanding models through customizing and fine-tuning them to agentic apps on your own hardware, which fills the gaps left by the removed ADAPTER route. Pair it with the Ollama models cheat sheet to choose what to build on.
Sign in, run cloud models and launch integrations
Models with a :cloud tag run on Ollama’s servers but use the same commands and API, so ollama run kimi-k3:cloud works like a local model once this machine is linked to an account. Unlinked, it stops with You need to be signed in to Ollama to run Cloud models. On a headless server, ollama signin can’t open a browser, so it prints the link to open on another device:
ollama signinThe link carries this machine’s name and public key, which is how the account learns to trust it:
You need to be signed in to Ollama to run Cloud models.
If your browser did not open, navigate to:
https://ollama.com/connect?name=llm01&key=REDACTEDTo guarantee nothing leaves the box, set OLLAMA_NO_CLOUD=1 in the service override. /api/status then reports the switch and where it came from, and a cloud model fails at once instead of prompting for sign-in. After our failed cloud attempts, ollama list also showed a kimi-k3:cloud entry with a size of -, so remove those stubs with ollama rm.
{"cloud":{"disabled":true,"source":"env"}}
Error: ollama cloud is disabled: remote model details are unavailableollama launch writes the configuration that points a coding agent at your Ollama server, then starts it. ollama launch --help on the current release lists 18 integrations:
claude(Claude Code), plusclaude-desktopfor--restorechatgpt(aliasescodex-app,codex-desktop,codex-gui) andcodexopencode,copilot,cline,qwen,vscodehermes(we keep a full Hermes Agent cheat sheet),hermes-desktop,openclaw,omp,droiddsh(DeepSeek Harness),kimi,muse,pi,pool
--model picks the model, --config writes the settings without starting the tool, -y answers the confirmation prompts, and --restore puts the tool back on its own provider. Arguments after -- go to the tool itself, for example ollama launch codex -- --sandbox workspace-write. Our OpenCode and Codex CLI cheat sheets cover those tools once they’re running.
Fix common Ollama errors
Every error below came up while testing this page, word for word. Search for the text you see, and the fix is underneath.
Error: time: missing unit in duration “-1”
ollama run --keepalive parses a Go duration, so -1 has no unit and fails. Use --keepalive -1s (or -1m) to keep the model loaded forever, and ollama ps will show Forever. The API raises the same error for the string "-1", so send the integer -1 instead.
Error: unknown flag: –system
ollama run has no --system flag, and -c (unknown shorthand flag: 'c' in -c) and --num-gpu do not exist either. Set a system prompt with /set system in the REPL, a SYSTEM line in a Modelfile, or a system message in the API. Context and GPU layers go through /set parameter num_ctx and num_gpu, the API’s options, or OLLAMA_CONTEXT_LENGTH.
Error: listen tcp 127.0.0.1:11434: bind: address already in use
You ran ollama serve while the systemd service (or the desktop app) already holds the port. You don’t need a second server, so use sudo systemctl restart ollama to apply changes. If you really want a foreground server for debugging, stop the service first or give the second one another port with OLLAMA_HOST=127.0.0.1:11500 ollama serve.
Error: could not connect to ollama server, run ‘ollama serve’ to start it
The CLI cannot reach a server at OLLAMA_HOST. Check systemctl is-active ollama, then check that your shell isn’t pointing somewhere else with echo $OLLAMA_HOST. A close cousin, Warning: could not connect to a running Ollama instance from ollama -v, is what our interrupted install produced: the binary was in place, but the script died before it created the user and the unit. Re-run the installer to finish the job.
Error: Post “http://127.0.0.1:11434/v1/systemone”: EOF
An EOF from any endpoint means the server dropped the connection mid-request, and on a machine without much RAM that is usually the kernel OOM killer. Check with journalctl -u ollama | grep -i oom and sudo dmesg | grep "Out of memory". Unload what you don’t need with ollama stop, set OLLAMA_MAX_LOADED_MODELS=1, or run a smaller quantization.
Error: mkdir /data/ollama/models/blobs: permission denied
The full message ends in ensure path elements are traversable, and the service crash-loops. OLLAMA_MODELS points at a directory the ollama user can’t write. Run sudo chown -R ollama:ollama on the new location and make sure every parent directory is traversable, then restart.
Error: create-time quantization is only supported for safetensors imports
ollama create -q no longer quantizes GGUF models, whichever level you pass. Quantize the GGUF with llama.cpp before you import it, or import safetensors and use -q int4, int8, nvfp4, mxfp4 or mxfp8.
Error: LoRA adapters are no longer supported
The Modelfile has an ADAPTER line. Merge the adapter into the base weights with your training tool and import the merged model with FROM.
Error: pull model manifest: file does not exist
From pull or run, the model or tag does not exist, so check the spelling on ollama.com/library. From /save in the REPL, the model’s parent_model is a file name (model.gguf on qwen3.5:4b) that Ollama tries to pull, as described in the REPL section, so build the model with a Modelfile instead.
This server does not support embeddings
You sent a chat model to /api/embed. Use an embedding model such as nomic-embed-text or all-minilm. The reverse mistake, loading an embedding model to chat, fails with does not support generate.
Error: ollama cloud is disabled
OLLAMA_NO_CLOUD=1 is set, so :cloud models are blocked on purpose. Remove the variable and restart the service if you want them, then run ollama signin. curl -s localhost:11434/api/status tells you whether the switch came from the environment or from server.json in the server user’s .ollama directory (/usr/share/ollama/.ollama/server.json for the service).
