AI Survival
A fully offline, portable AI assistant that runs a local LLM from an external HDD/USB — no internet required after setup.
Overview
AI Survival packages a complete local LLM stack onto an external HDD or USB drive. Plug it into any machine, run a single setup script, and chat with a quantized language model through a browser-based UI — fully air-gapped. It supports two deployment modes: Docker Compose (cross-platform, recommended) and a Python-native local mode for machines without Docker.
Key Features
- One-command startup (
setup.shthenstart.sh) across Linux, macOS, and Windows - Two deployment modes: Docker Compose and local Python — no internet required after initial prep
- Interactive model downloader with resume support (TinyLlama 1.1B → Llama 3.1 8B)
- NVIDIA CUDA GPU acceleration via
docker-compose.gpu.ymloverride validate.shpre-deployment checklist covering 30 checks across 6 categories- Offline Docker installer pre-caching and
save-images.shfor fully air-gapped targets
Technical Decisions
Chose llama.cpp over Ollama for direct inference control and a built-in OpenAI-compatible HTTP API, with Open WebUI as the frontend since it natively speaks that API. Docker Compose handles service orchestration and volume-mounted GGUF models, while docker save/load enables true air-gapped deployment by exporting image tars to the HDD. Q4_K_M quantization was selected as the default — best quality-to-size ratio for resource-constrained portable hardware.