Why MLX Serve is built
the way it is.
Every design choice here solves a problem that shows up when you put AI into a regulated environment. This page walks through the reasoning.
Open source is non-negotiable
No lock-in, ever
MIT licensed. If we vanish tomorrow, your systems keep running. Fork it, change it, ship it. You do not need our permission.
Your security team can read it
Every network call and permission boundary is there to inspect before you deploy. You verify it yourself instead of taking our word for it.
No licensing games
The engine is MIT and stays MIT. No usage caps, no metered tokens, no crippled free version. Paid add-ons sit beside it, clearly marked.
Source access
Government and defense buyers increasingly ask for source access to AI systems. Open source clears that on day one.
Swapping models should be easy
Models come and go
Models are named in config and loaded when needed. Everything else runs as GGUF. Swapping one is a name change, not a migration.
Negotiate from strength
If changing vendors costs you nothing, the relationship is a choice. That changes how the contract conversation goes.
Right model per job
Give each agent the model that fits its task, its budget, and how sensitive the data is. No rework.
When the rules change
If a regulation limits which model may touch which data, you edit a config file. The application stays put.
The model runtime is built in
Nothing else to install
Most "local model support" leans on Ollama or vLLM running alongside it. Ours handles models directly. One binary, one install. On the same weights it decodes about 26% faster than LM Studio.
- Native models plus the whole GGUF library
- Text, image, video, music, speech, and 3D in one cache
- Speculative decoding, KV-cache quantization, continuous batching
- Resumable downloads, with RAM estimates up front
Measured against LM Studio, oMLX, and MTPLX on identical weights. Full results and method, including the models where we tie.
Agents run in a box
Built to pass security review
Turn on one setting and every shell command runs inside an isolated Linux VM. It boots in under a second. Your machine stays clean.
- Real VM isolation, not a process sandbox
- Ten built-in tools, each behind an approval prompt
- Allow once, allow for the session, or deny
- A toolbar badge shows when isolation is on
Your documents stay put
Ask your own files
Point it at a folder of notes, PDFs, transcripts, whatever you have. Ask questions in plain English. About 500 files index in seven seconds.
Nothing written, nothing sent
The index lives in memory on the machine you are using. No embeddings leave. No copy is left on disk.
Answers show their sources
It pulls the relevant passages and names the file each one came from. A reviewer can check the answer instead of trusting it.
Past a folder, we build it
Search across Postgres, MySQL, file shares, and internal APIs takes real pipeline work. That part we build and run for you.
Config is just files
In git from the start
Skills are markdown with YAML frontmatter. Agents, MCP servers, and prompts are plain files. Review changes in a PR. Roll back with a commit.
Nothing hidden
No settings buried in a database or a preferences pane. What the files say is what the agent does. Drift shows up as a diff.
Your reviewers can read it
A skill is a name, a description, some trigger words, and plain instructions. You do not need to be an engineer to review one.
Write it or draft it
Type the files yourself, or describe what you want and let the composer write them. Either way you end up with the same files.
We speak everyone else's API
Four wires, one port
OpenAI, Anthropic Messages, and the Ollama API all answer on the same address. Pointing a tool at it means changing a URL.
Your tools already work
Claude Code, Cursor, Zed, Continue, aider, Open WebUI. They all connect as they are. So does your own code.
Leaving costs nothing
We implement interfaces other people defined. Moving off is the same one-line change as moving on. That is what makes it safe to start.
Prompts stay inside
Engineers keep the workflow they have. The code and context they send never leaves the building.
There is no Python in it
One binary
A few megabytes, compiled. No interpreter, no package manager, no dependency tree for your team to audit or patch.
Code-signed and verifiable
Every release ships code-signed. It clears endpoint policy without an exception request.
Nothing phones home
No analytics. No telemetry. No outbound calls. Bind it to localhost, or require a key for anything off the machine.
Installs on locked-down machines
No pip, no virtualenv, nothing to provision or keep patched. On a hardened or disconnected box, that is often the whole difference.
Sharing stays on your network
One machine serves the room
Machines find each other with nothing to configure. The box with the most memory hosts the big model. Everyone else calls it by name.
Traffic never leaves
A prompt sent to a shared model crosses your network and stops there. No broker, no relay, no public endpoint.
Capacity without keys
Teams share one expensive machine instead of buying several. Nobody hands out a vendor credential.
Still works alone
Every machine runs its own models too. If the shared host goes away, work carries on.
See the architecture in your environment
We'll walk your team through it and scope a deployment against your real use cases. Two weeks to production, or your money back.
Book a Demo