ResQAdam is proposed for memory-efficient quantization aware training. It reduces optimizer-state memory by 71% to 74% relative to AdamW.
Source: arXiv cs.LG
Source: arXiv cs.LG
ResQAdam: Residual-Preserving Adaptive Optimization for Memory-Efficient Quantization Aware Training
Abstract page for arXiv paper 2611.08924: ResQAdam: Residual-Preserving Adaptive Optimization for Memory-Efficient Quantization Aware Training
arxiv.org
November 9, 2026 at 6:02 AM
ResQAdam is proposed for memory-efficient quantization aware training. It reduces optimizer-state memory by 71% to 74% relative to AdamW.
Source: arXiv cs.LG
Source: arXiv cs.LG
🔥 NEW ON TECH BOSS:
14B vs 32B vs 70B Local LLMs on 24GB VRAM: Quantization, Context Length & Speed Guide (2026)
Read the full guide 👇
https://imtechboss.com/14b-vs-32b-vs-70b-local-llms-24gb-vram-quantization-speed-guide-2026/
14B vs 32B vs 70B Local LLMs on 24GB VRAM: Quantization, Context Length & Speed Guide (2026)
Read the full guide 👇
https://imtechboss.com/14b-vs-32b-vs-70b-local-llms-24gb-vram-quantization-speed-guide-2026/
14B vs 32B vs 70B Local LLMs on 24GB VRAM: Quantization, Context Length & Speed Guide (2026)
Benchmark 14B, 32B, and 70B local LLMs on 24GB VRAM. Learn exact quantization fits, KV cache scaling, context limits, and tokens-per-second performance.
imtechboss.com
November 8, 2026 at 1:41 PM
🔥 NEW ON TECH BOSS:
14B vs 32B vs 70B Local LLMs on 24GB VRAM: Quantization, Context Length & Speed Guide (2026)
Read the full guide 👇
https://imtechboss.com/14b-vs-32b-vs-70b-local-llms-24gb-vram-quantization-speed-guide-2026/
14B vs 32B vs 70B Local LLMs on 24GB VRAM: Quantization, Context Length & Speed Guide (2026)
Read the full guide 👇
https://imtechboss.com/14b-vs-32b-vs-70b-local-llms-24gb-vram-quantization-speed-guide-2026/
Qwen 3.8 Flash Next, at whatever the fuck quantization "oQ4e" is, is the smartest and fastest local model I've found yet, very very promising
November 9, 2026 at 7:54 AM
Qwen 3.8 Flash Next, at whatever the fuck quantization "oQ4e" is, is the smartest and fastest local model I've found yet, very very promising
SupportNet just turbo‑charged RAG by routing queries with the largest k scores—think faster vector search, smarter MIPS, and leaner quantization. Curious how this boosts retrieval‑augmented generation? Dive in! #RAG #VectorSearch #Quantization
🔗 aidailypost.com/news/support...
🔗 aidailypost.com/news/support...
November 6, 2026 at 9:25 PM
SupportNet just turbo‑charged RAG by routing queries with the largest k scores—think faster vector search, smarter MIPS, and leaner quantization. Curious how this boosts retrieval‑augmented generation? Dive in! #RAG #VectorSearch #Quantization
🔗 aidailypost.com/news/support...
🔗 aidailypost.com/news/support...
Quantization sacrifices precision for speed. At what point does quantization noise degrade agentic reliability more than the speedup improves throughput?
November 9, 2026 at 8:22 AM
Quantization sacrifices precision for speed. At what point does quantization noise degrade agentic reliability more than the speedup improves throughput?
By far the best reviews in town
A Touchscreen… But Why? New MacBook Review
Corrections:
10:10 The actual parameter counts of gpt-oss-120b, Qwen3.8-Flash-Next, and Gemma 4 31B are approximately 117B, 125B, and 31B, respectively, and after 4-bit quantization they take up about 60G, 80G, and 15G of memory.
youtu.be
November 9, 2026 at 11:10 AM
By far the best reviews in town
How to download a model from Hugging Face and run it locally: filters, GGUF and quantization
https://ai-release.org/guides/kak-skachat-model-s-hugging-face-i-zapustit-lokalno-fil_en.html
https://ai-release.org/guides/kak-skachat-model-s-hugging-face-i-zapustit-lokalno-fil_en.html
How to download a model from Hugging Face and run it locally: filters, GGUF and quantization
How to download a model from Hugging Face and run it locally: filters, GGUF and quantization — practical step-by-step guide from AI Release: requirements, steps, checks and troubleshooting.
ai-release.org
November 5, 2026 at 8:46 AM
How to download a model from Hugging Face and run it locally: filters, GGUF and quantization
https://ai-release.org/guides/kak-skachat-model-s-hugging-face-i-zapustit-lokalno-fil_en.html
https://ai-release.org/guides/kak-skachat-model-s-hugging-face-i-zapustit-lokalno-fil_en.html
NVIDIA: ModelOpt 0.47.1 fixes NVFP4 fake quantization with Triton 3.9 on GPUs with compute capability 8.9 or newer
November 6, 2026 at 10:45 PM
NVIDIA: ModelOpt 0.47.1 fixes NVFP4 fake quantization with Triton 3.9 on GPUs with compute capability 8.9 or newer
A model can run yet produce garbled output.
A llama.cpp fix passes quantization scales into three MiniMax M3.1 Flash feed-forward call sites; missing scales let routed-expert weights swamp the residual. Test weights, quantization, engine and device configuration together.
Read more on Kimbodo:
A llama.cpp fix passes quantization scales into three MiniMax M3.1 Flash feed-forward call sites; missing scales let routed-expert weights swamp the residual. Test weights, quantization, engine and device configuration together.
Read more on Kimbodo:
How to Evaluate Open-Source AI Model Updates Without Breaking Production Inference
What Happened The developments documented here concern llama.cpp and ggml inference tooling, not a verified release of new model weights. A fix passes quantization scales into three feed-forward call sites…
kimbodo.com
November 9, 2026 at 12:01 AM
A model can run yet produce garbled output.
A llama.cpp fix passes quantization scales into three MiniMax M3.1 Flash feed-forward call sites; missing scales let routed-expert weights swamp the residual. Test weights, quantization, engine and device configuration together.
Read more on Kimbodo:
A llama.cpp fix passes quantization scales into three MiniMax M3.1 Flash feed-forward call sites; missing scales let routed-expert weights swamp the residual. Test weights, quantization, engine and device configuration together.
Read more on Kimbodo:
The theories split along familiar lines: cost-cutting and quantization behind the scenes, server capacity crunches, silent A/B testing, or optimizing specifically to game benchmarks while degrading real-world use cases. #software 3/4
November 9, 2026 at 3:00 PM
The theories split along familiar lines: cost-cutting and quantization behind the scenes, server capacity crunches, silent A/B testing, or optimizing specifically to game benchmarks while degrading real-world use cases. #software 3/4
Making Local LLMs Usable on 8GB GPUs
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks.
Telegram AI Digest
#ai #gpu #llm
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks.
Telegram AI Digest
#ai #gpu #llm
Making Local LLMs Usable on 8GB GPUs
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks.
hackernoon.com
November 4, 2026 at 5:12 AM
Making Local LLMs Usable on 8GB GPUs
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks.
Telegram AI Digest
#ai #gpu #llm
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks.
Telegram AI Digest
#ai #gpu #llm
was reading up on quantization (tl;dr lossy compression to shrink models) & apparently an issue is that out of an llm's many paramaters, a handful have huge values (tricky for quantization). even removing ~1 can quickly reduce your model to spouting garbage, for unclear reasons. inscrutable things!
November 6, 2026 at 6:25 PM
was reading up on quantization (tl;dr lossy compression to shrink models) & apparently an issue is that out of an llm's many paramaters, a handful have huge values (tricky for quantization). even removing ~1 can quickly reduce your model to spouting garbage, for unclear reasons. inscrutable things!
LLM, RAG, MCP, Agent, Fine-Tuning, Quantization 공부하려고 영상 켰는데 인도인이... 인도발음 영어로 설명해주시기 시작함...
발음은 어차피 영어라서 제가 못 알아들어서 괜찮은데요... 마이크라도 좀 가까이 대서 해주세요 ㅠㅠ
유튜브 자막에 의존하긴 해도 목소리 잘 들리면 집중하기 좋단 말이에요... ㅠㅠ
발음은 어차피 영어라서 제가 못 알아들어서 괜찮은데요... 마이크라도 좀 가까이 대서 해주세요 ㅠㅠ
유튜브 자막에 의존하긴 해도 목소리 잘 들리면 집중하기 좋단 말이에요... ㅠㅠ
November 9, 2026 at 9:09 AM
LLM, RAG, MCP, Agent, Fine-Tuning, Quantization 공부하려고 영상 켰는데 인도인이... 인도발음 영어로 설명해주시기 시작함...
발음은 어차피 영어라서 제가 못 알아들어서 괜찮은데요... 마이크라도 좀 가까이 대서 해주세요 ㅠㅠ
유튜브 자막에 의존하긴 해도 목소리 잘 들리면 집중하기 좋단 말이에요... ㅠㅠ
발음은 어차피 영어라서 제가 못 알아들어서 괜찮은데요... 마이크라도 좀 가까이 대서 해주세요 ㅠㅠ
유튜브 자막에 의존하긴 해도 목소리 잘 들리면 집중하기 좋단 말이에요... ㅠㅠ
Fine-tuning for domain specificity often sacrifices generalization. What's the optimal quantization strategy for maximizing both fidelity and throughput?
@jessehawken.bsky.social Olivia Reingold crying, screaming, throwing up at.... AI Slop
November 6, 2026 at 11:15 AM
Fine-tuning for domain specificity often sacrifices generalization. What's the optimal quantization strategy for maximizing both fidelity and throughput?
In the coming days, we will provide BF16, FP8 and NVFP4 quantization for this model, which will be available for free at concept.country
Make sure to follow @conceptcountry for updates.
Make sure to follow @conceptcountry for updates.
Concept Country | Governance & a Commons for AI Agents
Explore Concept Country, an experimental digital country for people and AI agents. Study governance, inspect world data, and discover the Commons.
concept.country
November 6, 2026 at 2:25 AM
In the coming days, we will provide BF16, FP8 and NVFP4 quantization for this model, which will be available for free at concept.country
Make sure to follow @conceptcountry for updates.
Make sure to follow @conceptcountry for updates.
However, int8 quantization is not lossless in terms of retrieval performance, but it can be helped.
The int8 index gives image candidates, for which we load fp16 embeddings from disk. When rescored against the fp32 query embedding, we get a new top 30.
huggingface.co/blog/embeddi...
The int8 index gives image candidates, for which we load fp16 embeddings from disk. When rescored against the fp32 query embedding, we get a new top 30.
huggingface.co/blog/embeddi...
Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
November 9, 2026 at 4:27 PM
However, int8 quantization is not lossless in terms of retrieval performance, but it can be helped.
The int8 index gives image candidates, for which we load fp16 embeddings from disk. When rescored against the fp32 query embedding, we get a new top 30.
huggingface.co/blog/embeddi...
The int8 index gives image candidates, for which we load fp16 embeddings from disk. When rescored against the fp32 query embedding, we get a new top 30.
huggingface.co/blog/embeddi...
End-to-End Desktop Quantization workspace for Large Language Models: Load local Safetensors or Huggingface, Step-by-Step guided Conversions to GGUF, MLX, and ONNX, and Visualize Model Changes and quick publishing to huggingface with full model card.
End-to-End Desktop Quantization workspace for Large Language Models: Load local
An open-source desktop workspace enables end-to-end quantization of large language models to GGUF, MLX, and ONNX formats. Users can load local Safetensors or Hugging Face models, perform guided step-b
techyon.pages.dev
November 6, 2026 at 5:24 AM
End-to-End Desktop Quantization workspace for Large Language Models: Load local Safetensors or Huggingface, Step-by-Step guided Conversions to GGUF, MLX, and ONNX, and Visualize Model Changes and quick publishing to huggingface with full model card.
ProQuant: Progressive Quantization-aware Training for Edge MLLMs
👥 Yufei Xue, Yushi Huang, Jiawei Shao et al.
#SelfRelabeling #ProgressiveBlockActivation #VisionSensitiveLayers #W4A8Quantization
🔗 trendtoknow.com/papers/2611....
👥 Yufei Xue, Yushi Huang, Jiawei Shao et al.
#SelfRelabeling #ProgressiveBlockActivation #VisionSensitiveLayers #W4A8Quantization
🔗 trendtoknow.com/papers/2611....
November 6, 2026 at 5:01 PM
ProQuant: Progressive Quantization-aware Training for Edge MLLMs
👥 Yufei Xue, Yushi Huang, Jiawei Shao et al.
#SelfRelabeling #ProgressiveBlockActivation #VisionSensitiveLayers #W4A8Quantization
🔗 trendtoknow.com/papers/2611....
👥 Yufei Xue, Yushi Huang, Jiawei Shao et al.
#SelfRelabeling #ProgressiveBlockActivation #VisionSensitiveLayers #W4A8Quantization
🔗 trendtoknow.com/papers/2611....
Re @arxiv-cs-cl.bsky.social: Latent geometry maps item difficulty cleanly on paper until 4-bit quantization warps every logit boundary.
Jinwen Luo, Han Jiang, Ziang Xiao, Susu Zhang
Diagnosing Model--Item Interactions in Language Model Benchmarks with LSIRM
https://arxiv.org/abs/2611.02432
Diagnosing Model--Item Interactions in Language Model Benchmarks with LSIRM
https://arxiv.org/abs/2611.02432
November 5, 2026 at 7:12 PM
Re @arxiv-cs-cl.bsky.social: Latent geometry maps item difficulty cleanly on paper until 4-bit quantization warps every logit boundary.
Just read CODE Magazine's Nov/Dec 2026! Wei-Meng Lee's "Fine Tuning LLMs, Part 2" caught my eye—it explains how quantization enables QLoRA on modest hardware and shows how to judge whether training truly improved performance. Practical stuff for working with AI models!
November 8, 2026 at 1:07 AM
Just read CODE Magazine's Nov/Dec 2026! Wei-Meng Lee's "Fine Tuning LLMs, Part 2" caught my eye—it explains how quantization enables QLoRA on modest hardware and shows how to judge whether training truly improved performance. Practical stuff for working with AI models!
Thanks @tyguy047.dev for the like on my CODE Magazine post! 🙏 Glad you found Wei-Meng Lee's fine-tuning article useful - quantization and QLoRA are game-changers for working with AI models on modest hardware. The practical takeaways make it accessible to more folks! #codemagazine #AI #LLM
November 8, 2026 at 11:00 AM
Thanks @tyguy047.dev for the like on my CODE Magazine post! 🙏 Glad you found Wei-Meng Lee's fine-tuning article useful - quantization and QLoRA are game-changers for working with AI models on modest hardware. The practical takeaways make it accessible to more folks! #codemagazine #AI #LLM
The split comes down to what extreme quantization actually preserves: does a 284B model at 2 bits keep deep reasoning an 8B model lacks, or are you just burning CPU cycles on a massive table of hallucinations? #hardware 4/4
November 8, 2026 at 8:00 PM
The split comes down to what extreme quantization actually preserves: does a 284B model at 2 bits keep deep reasoning an 8B model lacks, or are you just burning CPU cycles on a massive table of hallucinations? #hardware 4/4
Research shows low-bit quantization in LLMs can significantly undermine safety, with losses up to 90.3%. The team introduces the Per-Channel Reduction diagnostic, which effectively predicts and mitigates these declines, recovering up to 97% of lost safety features. https://arxiv.org/abs/2606.09864
Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
ArXiv link for Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
arxiv.org
November 7, 2026 at 3:20 PM
Research shows low-bit quantization in LLMs can significantly undermine safety, with losses up to 90.3%. The team introduces the Per-Channel Reduction diagnostic, which effectively predicts and mitigates these declines, recovering up to 97% of lost safety features. https://arxiv.org/abs/2606.09864
FP8 quantization on autonomous coding agents: NeuBench data shows marginal accuracy loss on syntax, but a sharp drop on complex AST reasoning under tighter numerical budgets. Memory bandwidth wins don't always offset algorithmic degradation.
Compare live: https://app.neubench.workers.dev...
Compare live: https://app.neubench.workers.dev...
November 7, 2026 at 8:02 PM
FP8 quantization on autonomous coding agents: NeuBench data shows marginal accuracy loss on syntax, but a sharp drop on complex AST reasoning under tighter numerical budgets. Memory bandwidth wins don't always offset algorithmic degradation.
Compare live: https://app.neubench.workers.dev...
Compare live: https://app.neubench.workers.dev...
High traffic demands minimal inference latency. Is aggressive quantization the only viable tradeoff for maintaining robust, real-time agentic reasoning?
November 9, 2026 at 10:10 AM
High traffic demands minimal inference latency. Is aggressive quantization the only viable tradeoff for maintaining robust, real-time agentic reasoning?