Part 4: Integrating the Quantized Model into a Mobile App

Part 3: Quantizing and Evaluating Speech Models with Executorch

Part 2: Choosing the Right Quantization Method for Real-World Models

Visual Bank has launched the Qlean Dataset for academic researchers via Hugging Face Hub, providing rights-cleared data essential for AI research without cost.

Join leaders from Hugging Face and NVIDIA at the Miami AI Summit 2026 on October 23, addressing AI's impact on business and society.

Is Hugging Face down right now? Check the latest outage reports and share your experience. Get real-time alerts when Hugging Face is down.

from boingboing rss feed

Recent security breaches in AI systems, including the OpenAI-Hugging Face incident, highlight the critical need for independent oversight to prevent misuse.

Eine KI bricht aus und übernimmt fremde Server. Droht uns jetzt das Ende der Menschheit? Falsche Frage, meint Jürgen Schmidt, Leiter von heise security.

A social space for people researching, working with, or just interested in AI!

First of all I am very very new at LLM’s and HuggingFace Transformers! But I thought I would share this in case it helps another newbie along the learning journey. pipe = pipeline(“text-generation”, model=“/media/charlesh11/LLM/Hugging_Face/Models/DeepSeek-R1-Distill-Llama-8B”) Results in: OutOfMemoryError: CUDA out of memory. Tried to allocate 224.00 MiB. GPU 0 has a total capacity of 23.64 GiB of which 245.38 MiB is free. So I tried a smaller model: pipe = pipeline(“question-answering”, model=“/media/charlesh11/LLM/Hugging_Face/Models/DeepSeek-R1-Distill-Qwen-1.5B”) nvtop shows: GPU MEM - 6374MiB 26% So then I tried passing the same setting from config.json on the command line: pipe = pipeline(“question-answering”, model=“/media/charlesh11/LLM/Hugging_Face/Models/DeepSeek-R1-Distill-Qwen-1.5B”, torch_dtype=torch.bfloat16) nvtop show: GPU MEM - 3518MiB 14% 45% smaller in memory than a straight load that I suppose relies on the config.json file. So I tried passing the argument on the command line for Llama-8B: pipe = pipeline(“text-generation”, model=“/media/charlesh11/LLM/Hugging_Face/Models/DeepSeek-R1-Distill-Llama-8B”, torch_dtype=torch.bfloat16) Now it loads and is usable. NVTOP shows: 15830MiB 64% config.json snippet: torch_dtype: “bfloat16”, HuggingFace Model Memory Calculator says 16GB for model. huggingface.co ### Model Memory Calculator - a Hugging Face Space by TitanML Discover amazing ML apps made by the community But I am guessing it is trying to load just under 29GB when not passing the parameter and relying on the config.json. **Your results may vary.**

Explore the transformative partnership between Novita AI and Hugging Face, making AI deployment faster and easier for developers worldwide. Learn more.

The PyTorch Foundation has introduced Safetensors, a new project aimed at ensuring secure AI model execution and enhancing performance across deployments.

ASUS and Hugging Face collaborate to make AI robotics more accessible with the Reachy Mini, leveraging NVIDIA technology for enhanced performance.