Back to The Lab

Currently building · Open source

Unhosted — AI that lives where you do.

Frontier-class inference on hardware you own. Three trust modes — local, trusted, public — pooled into one inference cluster.

Period
2024 – Present · pre-alpha
Role
Founder · Lead designer · Engineer
Tags
Open sourceDistributed systemsLocal AIApache 2.0

Unhosted is the open-source project I started to answer one question: when frontier AI runs as well on your laptop as it does in a hyperscaler, who owns the inference?

Pool the computers you already own — and optionally your friends', and optionally a public swarm of strangers' GPUs — into a single inference cluster. One endpoint. Mac, Linux, Windows. CUDA, Metal, ROCm.

Three trust modes. You decide the radius. Local and trusted are free forever; public is the safety net you pay for in USDC only when you opt in. You can use Unhosted for the rest of your life and never spend a dollar.

github.com/unhosted-aiunhosted-coreunhosted-payments

github.com/unhosted-ai · CLI v0.0.34

The radius is the product.

Most local-AI tools force the user to choose between "private and limited" and "powerful and surveilled." Unhosted makes the trust radius itself the primary control.

pre-alphabuilt in publicApache 2.0

The trust radius

  • 1 · Local

    Devices you own

    Your laptop, gaming PC, home server. No internet required. Free forever. The whole product works here without ever crossing the network boundary.

  • 2 · Trusted

    Friends, family, team

    Your roommate's PC, your homelab, your team. End-to-end encrypted, no public exposure, no payment. Pair once over the internet, then routes treat them like local.

  • 3 · Public

    Strangers' GPUs · USDC

    Opt-in safety net. A swarm of strangers renting idle GPUs for USDC per token. Used only when your circle can't fulfill the request, with a per-month spend cap you set.

What's honest about today

The product is pre-alpha and shipping in public. Status is named explicitly, not hidden behind a polished feature page. This table is the truth about what works today.

CapabilityStatusNotes
Single-machine inferenceshippedv0.0.1. Wraps llama.cpp llama-server. Smoke-tested on M-series.
LAN cluster (request routing)shippedv0.0.2. Round-robin across local + peers; verified end-to-end.
mDNS peer discovery + pairingshippedv0.0.3. One-click pair in the app sidebar; hot-reload routing.
Model management (unhosted pull)shippedKnown short names + direct GGUF URL support.
Public-mode policy + signed receiptsshippedunhosted-payments-core 0.0.2 + daemon integration in v0.0.39. Cross-language wire compatibility (Rust ↔ TypeScript) verified.
Wallet-js payer helpersshipped@unhosted-ai/wallet-js 0.0.1. Browser + Node.
VRAM-pooling via RPC-enabled llama.cppshippedDistributed via the homebrew-unhosted tap because upstream Homebrew omits -DGGML_RPC=ON.
MCP server (first plugin)scaffoldedExposes local capabilities (memory, web fetch, VRAM-pool status) as MCP tools so Claude Desktop and IDE clients can call into the daemon.
First payment rail (Lightning leading candidate)pendingSlice 3 of the transactional-public-mode ADR.
Solidity escrow on BasependingSlice 5 of the transactional-public-mode ADR.

Repos in the org

unhosted-core

RustView on GitHub →

The daemon. Wraps llama.cpp's llama-server on each node and presents a single OpenAI-compatible inference endpoint across the whole cluster.

What it does: mDNS peer discovery on the LAN, one-click pairing over the internet for trusted nodes, round-robin routing across local + peers with hot-reload, model management, VRAM-pooling via RPC-enabled llama.cpp, public-mode policy + signed-receipt verification.

Why it lives in its own crate: the daemon's release cadence is hardware-bound (Apple silicon updates, CUDA driver versions, ROCm releases). Compliance and rails work for the public mode happens on a different clock — that's unhosted-payments.

unhosted-payments

Rust + TypeScriptView on GitHub →

The settlement layer. When peers rent out idle compute to strangers in public mode, this is where the money moves — policy-driven, multi-rail, country-aware.

Why a separate repo: three reasons. Cadence — compliance work on a different clock than daemon engineering. Surface — payments touches more languages than the daemon's Rust. Trust boundary — sensitive code (key handling, settlement state) is reviewable in isolation; a reader auditing payments shouldn't have to skim 50k lines of daemon code to find the parts that move money.

Shape: Rust core crate with PeerPaymentPolicy, PaymentRail, KycTier, Country, SignedReceipt, verify_receipt, sign_receipt. TypeScript wallet-js for payer-side browser/Node helpers, cross-verified by an integration test. Solidity escrow on Base pending.

unhosted-plugins

TypeScriptView on GitHub →

Extensions that talk to a running Unhosted daemon. Each plugin lives in its own top-level directory; the only thing that makes it a 'plugin' is that it talks to the documented HTTP API at 127.0.0.1:7777.

First plugin (scaffolded): mcp-server/ — exposes Unhosted's local capabilities (memory, web fetch, VRAM-pool status) as MCP tools so MCP-aware clients like Claude Desktop or IDE extensions can call into the daemon.

Why a separate repo: the core daemon stays small, and plugin work doesn't block daemon releases (or vice versa).

homebrew-unhosted

RubyView on GitHub →

Homebrew tap for Unhosted dependencies that aren't shipped by upstream Homebrew the way Unhosted needs them.

Today: one formula, llama-cpp-rpc — llama.cpp built with -DGGML_RPC=ON, which Unhosted's VRAM-pooling feature requires. Upstream brew install llama.cpp omits the flag. The tap fills the gap.

Design moves I'm proud of

  • LESSON 01

    "Status: pre-alpha" as the first line of the README.

    Most pre-alpha projects pretend otherwise. Naming it up front earns the right to ship in public without misleading anyone. The 'What's honest' status table runs the same posture — a binary works / doesn't per capability.

  • LESSON 02

    Three trust modes as a visual primitive, not a settings page.

    Most local-AI tools force the user to choose between 'private and limited' and 'powerful and surveilled.' The radius diagram says the truth: three concentric circles, and you can stop at the inner one for the rest of your life if you want.

  • LESSON 03

    "You can use Unhosted forever and never spend a dollar."

    A revenue model can sit under a free product if the free product is genuinely complete on its own. Public mode is the safety net for the cases your circle can't fulfill, not the upsell the rest of the product pushes you toward.

  • LESSON 04

    Separate repos by cadence + trust boundary, not by tech.

    unhosted-payments isn't separate because it's a different language — it's separate because a blocked KYC review shouldn't pause a VRAM-pool release, and because a reader auditing money-moving code shouldn't have to skim 50k lines of daemon code first.

  • LESSON 05

    The CLI is the source of truth; the GUI is a wrapper.

    Every operation the desktop app can do is also expressible as unhosted <verb>. That makes the product scriptable by default and forces the design of each verb to be sharp enough to type.

If frontier AI ends up running on the hardware you already own, the design problem is no longer about chat windows. It's about where the inference happens, who the trust boundary is around, and how legibly the system tells you those things.

View the org on GitHub →
Upcoming