Sraars: > We won’t be able to afford them. NPU is literally in every new processor. As time advances they will get more powerful and be able to handle a lot of the simple tasks people throw at them. Homeprivacy: > Absolutely. I think Locally hosted LLM with Cloud PII redaction for any inference that has to hit a frontier model is not only prudent but necessary for those that want to maintain privacy going forward. The tradeoff of privacy for productivity doesn’t have to happen, but most people doin’t know that. There’s always going to be room for both. Some things a small model can do quite easily, like semantic searching eg “find me the invoice where I bought a shoe” should **never** leave the system. You might not have the word “shoe” in the file, but maybe “sneaker” instead. A small language model is perfect for this. You can use a sliding window that clears as it goes since it doesn’t need to reference those documents again. This can all be done easily on a small NPU. There are plenty of other examples where you need huge context windows, especially when dealing with massive datasets. In those cases, it makes more sense to use the cloud, especially for data that isn’t sensitive. Contrary to popular belief, public LLMs such as ChatGPT and Gemini **do not** train on active conversations in real-time. Training on this type of data **would** lead to model collapse. For example, a model trained on low-quality data from someone who thinks that an LLM is intelligent and now their boyfriend would not be useful to anyone.

Homeprivacy: > AI will centralize our lives under its umbrella in exchange for productivity. As for this, there’s absolutely evidence to the contrary, in terms of local LLM usage, and processor NPU capabilities. Certain queries do not require large models and most certainly can be done decentralized on the user’s device. This saves a round trip, and doesn’t disclose any data.

Homeprivacy: > all of them are making moves to own the AI memory layer for you and your family. What is the AI memory layer?