A beginner's guide to self-hosted AI setup.

Hendo

— 1 min read
Share

I wanted to create this blog to chronicle my journey. Here are my requirements:

  1. No Cloud.
  2. Self-hosted.
  3. Open-source/Open-weight.

So far, I purchased a GMKtec Evo-X2 with 128GB of memory. I was eyeing the Mac Mini first, but they stopped shipping 128gb after everything went crazy.

From a high level, my plan is to install Linux, install llama.cpp as the front end, install pi.dev as the local coding agent.

llama.app - Official home for llama.cpp
Official website for the llama.cpp project
llama.cpp
Pi Coding Agent
A terminal-based coding agent

Then start trying to connect home-assistant MCP and other apps to create hass dashboards.