Run Local LLMs on Fedora 45 / 44 with Ollama (CPU)

A 4-core Fedora box with no GPU runs Gemma 3 1B at about 36 tokens per second through Ollama. The model matters far more than the CPU. On the same VM, Gemma 4 E2B crawled at 3.5 tokens per second, and Qwen 3.5 thought for four and a half minutes before writing a two sentence answer. Picking the right small model and fixing two defaults is the job when you run local LLMs with Ollama on Fedora without a GPU.

Original content from computingforgeeks.com - post 167957

Below, you install Ollama from the official script, see why Fedora’s own package falls short, benchmark four small models and call the API. Then you add a systemd drop-in that keeps a CPU box clear of the OOM killer and open the port to your LAN. Every command below was run on Fedora 45 with Ollama 0.40.11 in October 2026, and the install, benchmarks and uninstall again on Fedora 44.

Check the host before you install

RAM decides what you can run, not cores. A model has to fit in memory next to everything else on the box, and on CPU the whole model sits in system RAM, so a 3B model wants about 2.5 GB free on its own. Cores decide speed, and the default already uses all of them. The lab VMs had the following, which is a sensible floor for following along:

• Fedora 45 or Fedora 44, any edition
• 4 vCPU with AVX2 (AVX-512 helps)
• 8 GB RAM for models up to 3B
• About 15 GB of free disk for the runtime plus four models
• curl and zstd, both present on stock Fedora images

We tested Fedora 45 and Fedora 44 only. If a box is still on 43, bring it up to date first, and the Fedora 44 to 45 upgrade guide covers the last hop.

Disk is the one people underestimate. The runtime alone takes 2.3 GB because the installer ships CUDA libraries even on a box with no NVIDIA card, and the four models in this guide add another 10 GB. On a 20 GB cloud VM that left 7.7 GB free, which was enough to run them and not enough to copy the model store, as you’ll see later.

Fedora’s ollama package or the upstream script?

Use the upstream script. Fedora does package Ollama in its main repository, but the build lags far behind upstream, and the models in this guide need a current runtime. This is what dnf info --available ollama reported on each lab VM against what the official installer put down the same day:

Source	Fedora 45	Fedora 44
Fedora repository (dnf install ollama)	0.24.0-4.fc45	0.12.11-4.fc44
Official install script	0.40.11	0.40.11

We removed the upstream build from the Fedora 44 VM and installed the package to see what that gap costs. It pulled in 27 packages, including 2.4 GB of ROCm libraries, reported itself as ollama version is 0.0.0, and downloaded qwen3.5:2b and gemma4:e2b without complaint. Neither would run:

Error: 500 Internal Server Error: unable to load model: /var/lib/ollama/.ollama/models/blobs/sha256-e8bf47d40c5ba68baf8f3e503ee36710da6e8641cb0a670a508cff5b3f0cfbf7

Older models such as gemma3:1b did load, at 19 to 24 tokens per second against 27 to 36 with the upstream build on the same VM. If you already have the package, remove it with sudo dnf remove ollama before running the script, because both use port 11434 once its service is enabled, and they keep models in different places (/var/lib/ollama for the package, /usr/share/ollama for the script).

Install Ollama on Fedora with the official script

The script downloads a zstd tarball, unpacks it under /usr/local, creates an ollama system user and writes a systemd unit. It does not add a dnf repository, so the repo file changes in Fedora 45 don’t touch it and the same command works on 44 and 45:

curl -fsSL https://ollama.com/install.sh | sh

On the Fedora 45 VM the run took 3 minutes 15 seconds, almost all of it the download. The status lines are worth reading once, because they tell you where everything went:

>>> Installing ollama to /usr/local
>>> Downloading ollama-linux-amd64.tar.zst
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink '/etc/systemd/system/default.target.wants/ollama.service' → '/etc/systemd/system/ollama.service'.
WARNING: Unable to detect NVIDIA/AMD GPU. Install lspci or lshw to automatically detect and install GPU dependencies.

Ignore the GPU warning on a CPU box. It only means the script couldn’t look for a graphics card because lspci and lshw are not on the cloud image, and CPU inference worked without either. If you do have an NVIDIA GPU, sort the driver first with the NVIDIA driver guide for Fedora and Ollama will pick it up.

Next, check the version and the service. Both should answer even before you pull a model.

ollama --version
systemctl status ollama --no-pager | head -9

You want active (running) and a memory figure in the tens of megabytes, because nothing is loaded until the first request. The capture below also shows ollama list after the models from the next section were pulled:

The service listens on 127.0.0.1:11434 only, which is the right default. The generated unit hardcodes a PATH copied from the shell that ran the installer, and the script rewrites that whole file on every update. Any edit you make to /etc/systemd/system/ollama.service itself is therefore lost, so later sections put every change in a drop-in instead.

To confirm Ollama picked a CPU build that matches your processor, check the log after the first model loads:

sudo journalctl -u ollama --no-pager | grep -E "inference compute|loaded CPU backend"

The lab host is an 11th gen Intel Core i7, so Ollama loaded its Ice Lake backend with AVX-512. An older CPU gets a Haswell or plain x64 build and runs slower:

level=INFO source=types.go:50 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="7.7 GiB" available="7.2 GiB"
load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-icelake.so

If library= ever reads cuda or vulkan on a machine where you expected CPU, Ollama found a GPU you forgot about, which is good news for speed.

Pull models that fit a CPU

On CPU, stay at 3B parameters or below in a 4-bit quantization. Bigger models do run, but generation drops into single digits and the wait for each answer starts to feel broken. These four cover the current small families from Google, Alibaba and Meta:

ollama pull gemma3:1b
ollama pull qwen3.5:2b
ollama pull gemma4:e2b
ollama pull llama3.2:3b

At about 10 MB/s the pulls took 1 minute 42 seconds for Gemma 3 and 8 minutes 35 seconds for Gemma 4. Models land in /usr/share/ollama/.ollama/models, owned by the ollama user. The size on disk and the memory each one actually used once loaded are not the same thing:

Model	On disk	RAM when loaded	Good at
gemma3:1b	806 MB	878 MiB	Fast chat, summaries, shell helpers
qwen3.5:2b	2.7 GB	2,822 MiB	Coding, multilingual prompts, tool calls
llama3.2:3b	2.0 GB	2,468 MiB	Longer, cleaner English answers
gemma4:e2b	4.6 GB	4,983 MiB	Image input, when speed doesn’t matter

The RAM column comes from the llama-server process RSS, not from ollama ps. For Gemma 4 E2B, ollama ps reported 1.1 GB while the process held almost 5 GB, so size your box from ps or systemctl status ollama. Our Ollama models cheat sheet lists the bigger options for when you add a GPU.

Benchmark each model on CPU

The --verbose flag prints a timing block after every answer, so you can measure your own box instead of trusting ours. Run the same prompt against each model, once cold and once warm:

ollama run qwen3.5:2b "Explain what SELinux does in two sentences." --verbose --think=false

The figure that matters for interactive use is eval rate, the tokens generated per second. On the warm run Qwen 3.5 2B produced 63 tokens at 15.83 tokens per second, and the whole answer took about 4 seconds:

Read the answer as well as the numbers. The 2B model invented an expansion for the SELinux acronym on one run and a different one on the next, which is typical of models this small. They are fine for drafting and summarizing, but check any fact they give you. Repeating the prompt across all four models on both releases gave this:

Model	Eval rate, Fedora 45 (tok/s)	Eval rate, Fedora 44 (tok/s)	Verdict on CPU
Gemma 3 1B	35 to 37	27 to 36	Fastest; the default pick
Qwen 3.5 2B (thinking off)	14 to 16	15 to 16	Best quality per second
Llama 3.2 3B	14 to 16	14 to 16	Same speed as Qwen, older model
Gemma 4 E2B (thinking off)	3.2 to 3.6	3.5 to 3.6	Too slow for chat on CPU

Fedora 45 and Fedora 44 are the same speed within noise. Both VMs shared one Proxmox host with other workloads, so individual runs swung by up to 25%, and the low 27 on Fedora 44 came from one run that measured 35.7 when repeated. The model choice moves the number by a factor of ten, while the release moves it by nothing you’d notice.

Gemma 4 E2B is the surprise. Its runner loads a separate draft model for speculative decoding plus a vision projector, holds about 5 GB, and still generates at 3.5 tokens per second on this CPU, so keep it for image questions where you can wait. The first load of any model after a pull took 10 to 16 seconds as the file came off disk, while later cold loads took 1.5 to about 7 seconds from the page cache.

Does setting num_thread help?

No, not on a box that does nothing else. Ollama already starts the runner with one thread per core, and passing num_thread through the API only helps when you want to hold cores back for other work. On Gemma 3 1B with the same 46 token answer each time, the API reported:

• 1 thread: 9.34 tokens per second
• 2 threads: 16.6 to 17.0 tokens per second
• 4 threads: 22.0 to 27.0 tokens per second
• Default (4 threads): 27.0 to 37.8 tokens per second

Scaling is close to linear up to two threads and flattens after that, the usual sign that memory bandwidth rather than core count sets the ceiling for token generation. If you’re buying hardware for this, faster RAM and more memory channels deserve as much attention as cores, which is why a mini PC built for local AI often beats an older server with twice the cores.

Turn off thinking on Qwen 3.5

Qwen 3.5 thinks by default, and on CPU that turns a short answer into a long wait. Run without --think=false, the same SELinux prompt produced 3,878 tokens of hidden reasoning before a two sentence answer, which took 4 minutes 37 seconds at 14.2 tokens per second. With thinking off, the answer arrived in about 4 seconds and read just as well:

total duration:       4m37.791765449s
load duration:        2.726642429s
prompt eval count:    199 token(s)
prompt eval duration: 1.881285s
prompt eval rate:     105.78 tokens/s
eval count:           3878 token(s)
eval duration:        4m33.180976s
eval rate:            14.20 tokens/s

The tokens per second barely changed, which is the point. Thinking does not make the model slower, it makes it write fifty to sixty times more, so turn it off in each interface you use:

• CLI: --think=false on ollama run
• Native API: "think": false in the request body
• OpenAI-compatible API: "reasoning_effort": "none"

Editor plugins and scripts that talk to the OpenAI endpoint do not send that field unless you add it. Through /v1/chat/completions without it, a request to say hello in three words came back with 3,480 completion tokens and an 11,481 character reasoning field, so check your client’s request options before you blame the CPU.

Call the Ollama API from scripts and editors

Ollama serves its native API under /api/ and an OpenAI-compatible one under /v1/ on the same port. The native chat endpoint returns the answer plus the timing data you saw from --verbose:

curl -s http://localhost:11434/api/chat -d '{
  "model": "qwen3.5:2b",
  "messages": [{"role": "user", "content": "What is 2+2? Answer in one word."}],
  "think": false,
  "stream": false
}' | python3 -m json.tool

The durations are in nanoseconds. This was the first request after a restart, so load_duration takes 11.2 of the 11.7 seconds, and generating the answer took 74 milliseconds:

{
    "model": "qwen3.5:2b",
    "created_at": "2026-10-26T00:43:33.724092066Z",
    "message": {
        "role": "assistant",
        "content": "4"
    },
    "done": true,
    "done_reason": "stop",
    "total_duration": 11731432549,
    "load_duration": 11158212283,
    "prompt_eval_count": 24,
    "prompt_eval_cached_count": 0,
    "prompt_eval_duration": 493000000,
    "eval_count": 2,
    "eval_duration": 73694000
}

The OpenAI-compatible endpoint takes the OpenAI request schema, so any client that lets you set a base URL works against it:

curl -s http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.5:2b",
    "messages": [{"role": "user", "content": "Say hello in three words."}],
    "reasoning_effort": "none"
  }' | python3 -m json.tool

The reply is a standard chat.completion object with choices, a usage block and system_fingerprint set to fp_ollama. Four completion tokens confirm that thinking stayed off:

Point the official OpenAI Python or JavaScript client at http://localhost:11434/v1/ and pass any non-empty string as the API key, since Ollama ignores it. curl -s http://localhost:11434/v1/models lists the model names those clients will accept. For a chat UI on top of the same server, follow the Open WebUI with Ollama guide, and keep the Ollama commands cheat sheet open for the rest of the CLI.

Keep one model in RAM with a systemd drop-in

By default Ollama keeps each model loaded for five minutes after its last request, and on CPU it allows up to three at once, all sharing system RAM. In the first benchmark pass we ran the models back to back, and the earlier ones were still holding memory when Gemma 4 loaded with 2 GiB free, so the kernel OOM killer took out the runner and the client got a bare EOF:

Error: Post "http://127.0.0.1:11434/api/generate": EOF

The journal said what happened, and the service restarted on its own three seconds later:

msg="system memory" total="7.7 GiB" free="2.0 GiB" free_swap="7.7 GiB"
systemd[1]: ollama.service: The kernel OOM killer killed some processes in this unit.
systemd[1]: ollama.service: Failed with result 'oom-kill'.

Limiting Ollama to one loaded model fixes that for good on a single-user box, and a longer keep-alive saves the reload for a model you come back to. Create the drop-in directory and open an override file in it, which survives updates because the installer only rewrites the main unit:

sudo mkdir -p /etc/systemd/system/ollama.service.d
sudo vim /etc/systemd/system/ollama.service.d/override.conf

Add these lines. OLLAMA_NO_CLOUD switches off Ollama’s cloud features (remote inference and web search), so a prompt can’t be sent to a model on ollama.com by mistake:

[Service]
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_KEEP_ALIVE=15m"
Environment="OLLAMA_NO_CLOUD=1"

OLLAMA_NUM_PARALLEL already defaults to 1 on this release, so there’s no need to set it. Reload systemd and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Ollama prints its effective settings on startup, so the journal is the place to confirm the drop-in took:

sudo journalctl -u ollama -n 40 --no-pager | grep -oE "OLLAMA_(KEEP_ALIVE|MAX_LOADED_MODELS|NO_CLOUD|NUM_PARALLEL):[^ ]*"

Each variable should show the value from the file.

OLLAMA_KEEP_ALIVE:15m0s
OLLAMA_MAX_LOADED_MODELS:1
OLLAMA_NO_CLOUD:true
OLLAMA_NUM_PARALLEL:1

To see the limit working, run Gemma 3 and then Qwen 3.5, and check ollama ps after each. After the second run only Qwen is listed, with 14 minutes left on its timer, because loading it evicted Gemma.

Expose Ollama to your LAN with firewalld

Ollama has no authentication at all, so anyone who can reach the port can use your CPU and read your prompts. Only open it when another machine on a network you trust needs it, and only to that network. First make it listen on every interface by adding one line to the same override file:

sudo vim /etc/systemd/system/ollama.service.d/override.conf

The full file now reads as follows.

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_KEEP_ALIVE=15m"
Environment="OLLAMA_NO_CLOUD=1"

Check the firewall before you restart, because Fedora editions start from very different defaults and the wrong one leaves 11434 open to everyone:

• Cloud images: no firewalld installed, so the port is open
• Workstation, KDE and the Atomic desktops: the FedoraWorkstation zone allows TCP and UDP 1025 to 65535
• Server: the FedoraServer zone allows only ssh, DHCPv6 client and Cockpit on 9090

If firewalld is missing, install and start it. On a Cloud image it starts in the public zone, which allows ssh, mDNS and DHCPv6 client traffic, so an SSH session on port 22 survives, but add any other port you rely on before you enable it:

sudo dnf install -y firewalld
sudo systemctl enable --now firewalld

Next, check which zone your interface uses. The rich rule only works in the zone that actually holds the interface:

sudo firewall-cmd --get-active-zones

If it says FedoraWorkstation, switch the default to public before you add any rule. We set FedoraWorkstation as the default on the lab VM and the second VM reached Ollama with no rule at all, because of that 1025 to 65535 range. The switch also closes that range for everything else, so desktop tools such as KDE Connect or Steam local streaming will need their own rules afterwards. --set-default-zone changes both the runtime and permanent configuration and moves interfaces that have no zone of their own, so run --get-active-zones again afterwards and confirm the interface moved:

sudo firewall-cmd --set-default-zone=public

On Server, keep FedoraServer, because switching would drop Cockpit on port 9090. Either way, the ZONE line reads the default zone, which holds the interface unless you pinned that connection to another zone, so compare it with the --get-active-zones output. We tested this with FedoraServer as the default: the LAN client was refused before the rule and answered after it. Set your own subnet in the variable:

LAN_SUBNET=10.0.1.0/24
ZONE=$(sudo firewall-cmd --get-default-zone)
sudo firewall-cmd --permanent --zone="$ZONE" --add-rich-rule="rule family=ipv4 source address=${LAN_SUBNET} port port=11434 protocol=tcp accept"
sudo firewall-cmd --reload
sudo systemctl daemon-reload
sudo systemctl restart ollama

The firewalld guide for Fedora covers zones in more depth if your box has several interfaces. Then test from another machine on the LAN, using the Ollama host’s address:

curl -s http://10.0.1.50:11434/api/version

A version string means the listener and the firewall both let you in. A curl: (7) error points back at one of the two, and the troubleshooting section below shows which:

{"version":"0.40.11"}

Never forward 11434 from the internet. If you need remote access, put Ollama behind a reverse proxy with authentication or reach the box over a VPN.

Move the model store out of /usr/share

Models pile up fast, and /usr/share/ollama sits on the root filesystem. OLLAMA_MODELS points Ollama somewhere else, but setting it alone makes your existing models vanish from ollama list, because Ollama looks only in the new path. Move the store rather than copying it. We tried rsync first on the 20 GB VM and it filled the disk to 97% before we stopped it, while mv within the same filesystem finished in 28 milliseconds. For a second disk, format and mount it at /srv/ollama first. Across filesystems mv copies and then deletes, so it takes as long as the copy and needs the full size free on the new disk. We only ran the same-filesystem case.

sudo systemctl stop ollama
sudo mkdir -p /srv/ollama
sudo mv /usr/share/ollama/.ollama/models /srv/ollama/models
sudo chown -R ollama:ollama /srv/ollama

Add the path as a second drop-in so it stays separate from the tuning file:

sudo vim /etc/systemd/system/ollama.service.d/models.conf

It holds a single variable. Keeping it in its own file means you can drop the path later without touching the tuning settings.

[Service]
Environment="OLLAMA_MODELS=/srv/ollama/models"

Reload, start the service and list the models again:

sudo systemctl daemon-reload
sudo systemctl start ollama
ollama list

All four models came back. SELinux stayed enforcing and logged no denials, because the installer’s unit runs as unconfined_service_t, so a model directory labelled var_t or usr_t works without a semanage fcontext rule. Confirm on your own box with sudo ausearch -m avc -ts recent, and see the SELinux survival guide if you ever confine the service.

Update Ollama

Ollama ships new releases often, sometimes several in a week, and new model families usually need a current runtime. Update by running the install script again:

curl -fsSL https://ollama.com/install.sh | sh

The script removes the old libraries first, so an update re-downloads the full 2.3 GB runtime even if you deleted the CUDA folders to save space. It also rewrites ollama.service, while both drop-in files in /etc/systemd/system/ollama.service.d/ came through untouched and the journal showed the same settings afterwards. The first lines of the update look like this:

>>> Cleaning up old version at /usr/local/lib/ollama
>>> Installing ollama to /usr/local
>>> Downloading ollama-linux-amd64.tar.zst

Downloaded models are kept, and ollama list showed the same four afterwards. Since nothing under /usr/local belongs to an RPM, dnf upgrade never updates Ollama for you, so put the script in whatever runbook you use for the rest of the box.

Troubleshoot Ollama errors on Fedora

These are the errors the lab produced, with the cause behind each one.

Error: Post “http://127.0.0.1:11434/api/generate”: EOF

The runner died mid-request, and on CPU that is almost always the OOM killer. Check with sudo journalctl -u ollama | grep -i oom. Set OLLAMA_MAX_LOADED_MODELS=1 as shown above, or pick a smaller model, because RAM is the hard limit.

Error: pull model manifest: file does not exist

The model name or tag is wrong. We got it from ollama pull gemma-3:1b, where the real name is gemma3:1b. Copy the exact name and tag from the model’s page on ollama.com.

Error: could not connect to ollama server, run ‘ollama serve’ to start it

The CLI couldn’t reach port 11434. We hit it twice: once while the service was restarting after the OOM kill, and once from a pull fired the instant after systemctl restart ollama. Wait a few seconds and retry. Don’t start ollama serve by hand while the systemd service is enabled, because only one of them can hold the port. If it persists, read systemctl status ollama.

curl: (7) Failed to connect to port 11434 from another machine

Either Ollama still listens on 127.0.0.1 only, or firewalld is dropping the connection. ss -ltn | grep 11434 on the host should show *:11434 once OLLAMA_HOST is set, and sudo firewall-cmd --list-rich-rules --zone=$(sudo firewall-cmd --get-default-zone) should list your subnet.

Remove Ollama completely

Because the script installs outside dnf, dnf remove will not touch it. These are the steps from the official Linux docs plus the drop-in folder the docs do not mention, run on both VMs:

sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service
sudo rm -rf /etc/systemd/system/ollama.service.d
sudo rm -r $(which ollama | tr 'bin' 'lib')
sudo rm $(which ollama)
sudo userdel ollama
sudo groupdel ollama
sudo rm -r /usr/share/ollama
sudo systemctl daemon-reload

The tr trick turns /usr/local/bin/ollama into /usr/local/lib/ollama, which holds the 2.3 GB runtime. userdel prints one warning, and that is why the separate groupdel is there:

userdel: group ollama not removed because it has other members.

The other member is you, since the installer added your login to the ollama group. If you moved the models to /srv/ollama, delete that directory too. Disk use on the Fedora 45 VM dropped from 12 GB to 1.3 GB, systemctl cat ollama found no unit and which ollama came back empty, so nothing was left behind.