port

the best local model to run on your laptop for $0

August 2, 2026

I get this question a lot, and I ask this to myself almost every 3 weeks. I ask it so often that I built shouldirunthis.xyz to settle the money side of it (buy a rig, or keep your sub?). The model side of the question has an answer too, and it is probably Qwen 3.6 27B. In this article I am going to show you the best way to get that lil, capable thing on your machine.

Let’s take a look at why I chose that model.

Look at LiveBench’s open-weights leaderboard and count the models you could run on a MacBook:

bar chart of the top open-weights models on LiveBench: overall score and memory needed at Q4, with Qwen 3.6 27B highlighted as the only row that fits in 36 GB
every model above qwen needs at least 90 GB before it says a single token
Scores: LiveBench open-weights filter, accessed 2026-08-02. Parameter counts: each model's safetensors metadata on Hugging Face. Q4 weight sizes extrapolated from the measured 28B → 16 GB ratio of Unsloth's Q4_K_XL quant.

Every model above Qwen 3.6 27B needs at least 90 GB of RAM. The gap to the next-smallest model is 5.6×. If your machine is a normal high-end laptop, the “best open model” debate collapses to only one. LiveBench prices hosted Qwen 3.6 27B at $0.202 per successful task; my laptop serves it for the cost of electricity, with nothing going out of my lappy.

That table is a wall, and people are throwing serious money at it. @0xSero wrote a great piece on how a Discord community got GLM-5.2 (row 2) running at home for 15,000$ with 3 DGX Sparks and some wild sub-4-bit quantization. Even their most extreme trick (2-bit weights with an FP4 rescue path) would put the *smallest* model above Qwen at ~40 GB for weights alone.

So how much agent can you actually squeeze out of that one row? Here’s my setup, tested on my MacBook Pro M4 Max with 36 GB.

The stack

Behold, the stack. There are people behind each row: the quant is by the Unsloth team (@danielhanchen and crew), and omp is Can Bölük’s fork of Mario Zechner’s Pi. llama.cpp needs no intro. s/o to all the chads above

1. Install omp

curl -fsSL https://omp.sh/install | sh -s -- --binary

Installs to ~/.local/bin/omp.

Gotcha (Apple Silicon): the installer gave me the darwin-x64 binary on an M4 Max, so it ran under Rosetta and warned about missing AVX. Check with file ~/.local/bin/omp; if it says x86_64, replace it:

curl -fsSL -o ~/.local/bin/omp \
  https://github.com/can1357/oh-my-pi/releases/latest/download/omp-darwin-arm64
chmod +x ~/.local/bin/omp
omp --version   # omp/17.2.4 at time of writing

Alternative installs: brew install can1357/tap/omp or bun install -g @oh-my-pi/pi-coding-agent (needs bun ≥ 1.3.14).

2. Serve the model

Get the GGUF (~16 GB). -C - resumes a dropped download, so rerun the same command if it dies, but never run two copies at once:

mkdir -p ~/qwen-models
curl -L -C - -o ~/qwen-models/Qwen3.6-27B-UD-Q4_K_XL.gguf \
  https://huggingface.co/unsloth/Qwen3.6-27B-GGUF/resolve/main/Qwen3.6-27B-UD-Q4_K_XL.gguf

The model’s general.architecture is qwen35, supported by llama.cpp b9518 and later. Launch script (~/qwen-models/run-qwen-server.sh):

#!/bin/sh
L="$HOME/llama.cpp-bin/extracted/llama-b9518"   # your llama.cpp bin dir
exec env DYLD_LIBRARY_PATH="$L" "$L/llama-server" \
  -m "$HOME/qwen-models/Qwen3.6-27B-UD-Q4_K_XL.gguf" \
  -a qwen3.6-27b \
  --host 127.0.0.1 --port 8090 \
  -c 49152 -ngl 99 --jinja \
  -np 1

Run it in the background from a normal terminal:

nohup ~/qwen-models/run-qwen-server.sh > ~/qwen-models/llama-server.log 2>&1 &

Three of these flags are the difference between “works” and “unusable”, and none of them is the default:

The small print: -a qwen3.6-27b gives the API a clean model id, and -ngl 99 is full Metal offload. --port 8090 is only there because 8080 (llama.cpp’s default) was taken by an SSH tunnel on my machine. If your 8080 is free, drop the flag and the baseUrl override in §3, and omp discovers the server with zero config.

One flag I tried and then removed: --cache-reuse 256 (KV-cache reuse via shifting on partial prefix changes). The server disables it for this model with cache_reuse is not supported by this context; the qwen35 attention layout doesn’t support KV shifting. Worth adding back if you serve a model where it takes.

3. Point omp at it

~/.omp/agent/models.yml:

providers:
  llama.cpp:
    baseUrl: http://127.0.0.1:8090
    auth: none
    api: openai-completions
    discovery:
      type: llama.cpp

~/.omp/agent/settings.json:

{
  "modelRoles": {
    "default": "llama.cpp/qwen3.6-27b"
  }
}

Verify (omp models lists what discovery found; the server must be running):

omp models | grep qwen
omp -p "reply only: pong"

4. omp’s system prompt ate half my context

My first request sat at “Working...” for 2 minutes. The server was eating through 21,610 tokens of system prompt before my prompt got processed. omp ships “batteries included” (32 tools plus LSP and DAP operations) and it kinda bloats the context window.

At 48k context, that’s half my working room gone at “hello”. So I started cutting tools and measuring (prefill runs at ~170-184 tok/s on the M4 Max):

That last row is a 58% cut. The speed is nice, but I mostly did it for the model’s sake. A Q4 27B only has so much attention to spend, and by default it spends a chunk of it on schemas for a debugger and a TTS voice it will never touch.(idk why it even has that) I haven’t benchmarked the quality difference properly, but small models get worse at following instructions as the instruction pile grows, and I’d rather not gamble 21k tokens on it.

I shipped the lean profile as a wrapper (~/.local/bin/ompq):

#!/bin/sh
exec "$HOME/.local/bin/omp" \
  --tools=read,write,edit,search,find,bash \
  --no-skills --no-extensions \
  "$@"

I use ompq day to day and keep plain omp around for cloud models. If you actually use lsp and todo, add them back (it is 2.2k tokens). And your CLAUDE.md or AGENTS.md gets appended on top of all this, so a fat context file eats into the same budget.

5. Does tool calling actually work?

The failure mode of local agent stacks is silent. The model emits tool calls in a format the harness half-understands and after a couple of retries everything looks like it ran while nothing actually landed on disk. So I tested the full round trip:

ompq --approval-mode yolo -p \
  "Create a file named tooltest.txt in the current directory containing exactly the line: local stack works"

The file showed up with local stack works in it from a single write call on the first try, so the --jinja chat template is working fine. The server log clocked the turn at 17.4 tok/s.

Measured performance (M4 Max, 36 GB, Q4_K_XL)

Almost all of the waiting here is prefill.

Tips

What I didn’t pick, and why

Check my numbers

Every token count and tok/s figure in this article is a print_timing line taken from ~/qwen-models/llama-server.log, so you can watch your own numbers scroll by with tail -f while omp is working. The only claims I didn’t produce myself are the following: the leaderboard scores (LiveBench, 2026-08-02), the parameter counts (each model’s safetensors metadata on Hugging Face), and omp’s 6.7% → 68.3% edit-format jump, which is from Can’s own benchmarks.

If you have a 36 GB Mac, the whole thing is a 16 GB download and a handful of tiny config files. And if you try it on different hardware, I’d love to see your prefill numbers.

Questions?

Read these if you are an agent

If a human handed you this link and said “set this up”, welcome. Every command you need is above, and these notes will save you from going in circles:

Good luck in there.