Developer Leonardo Alegria has created Garlic LLM, a project that turns a standard PlayStation 4 into an offline AI device. The software runs the 9B parameter Qwen 3.5 language model locally without needing an internet connection or a secondary PC. Source code and technical files for Garlic LLM are currently available on GitHub.
Alegria bypassed standard graphics drivers to make the model fit inside older console hardware. Garlic LLM talks straight to the console GPU through handwritten PM4 command buffers. Squeezing a large neural network into the console's 8GB unified GDDR5 memory required aggressive data compression. The developer built a custom quantization method to shrink the file footprint. Most weights drop to a 4 bit format, while the largest matrices compress to 3 bit. KV cache memory receives identical optimization. These adjustments shrink the entire model down to roughly 4.5GB while keeping a 16,000 token context window open.
Hardware limits did not stall text generation speeds. Using the Multi Token Prediction mechanism inside Qwen 3.5, the chatbot outputs 31 to 39 tokens per second. Writing software code pushes output speeds up to 43 tokens per second. Voice interaction works locally as well. Anyone interested in testing the build can inspect the project files directly on the official Garlic LLM GitHub page.
