Background: Why This Project#
This repo is the full companion guide for the talk “How to Deploy an LLM Locally on a Mac” — you don’t need to watch the video; just follow the steps and you’ll have a working local model on your Mac. Many developers own an Apple Silicon Mac and want AI-assisted coding, but run into three walls: pricey API subscriptions, confusing model choices, and no idea how to make a local model actually work. This project clears that path end-to-end, with a visible finish line: your local model draws a duck riding a bicycle 🦆🚲.
Why It Matters#
- 🔒 Privacy is the #1 reason: your data never leaves your computer — private code, business documents, customer data, personal conversations are never uploaded, never used for training, never leaked. For trade secrets, compliance, or privacy-sensitive scenarios, local inference is the way to go
- 💰 Predictable cost: deploy once, run in unified memory, no per-token billing
- 🛠 Make local models actually work: not just a web chat — plug them into harnesses like Claude Code for real coding workflows
- 🧭 Lower the mental barrier: from “what are you actually deploying” to hardware sizing, model selection, and a terminology cheat sheet — one document covers it all
Quick Start (Apple Silicon, ≥16GB unified memory)#
- Install oMLX:
brew install omlx - Start & pick a model:
omlx start, then choose one from the menu-bar app (e.g. Qwen3.6-35B MLX 3bit) - Plug into Claude Code:
omlx launch claude→ ask it to “draw a duck riding a bicycle”
Tech stack: Apple MLX framework + oMLX runtime + Claude Code. The repo also ships the talk slides and a “duck gallery” comparing what each model actually drew.
Video Walkthrough#
Open source: MESevenJourney/mac-llm-deploy (bilingual README)
