Inferencing LLMs on Smartphones
A few days ago, I installed PocketPal on my smartphone – a privacy-friendly AI assistant for your pocket. The app allows running small language models directly on the smartphone. Is it just a gimmick or a first step into a new era of AI usage that prioritizes privacy and offline functionality?