Post-training quantization reduces the memory burden of large model deployment by casting weights to fewer bits. A new method decomposes original trained weights into a quantized matrix, a low-rank component, and a sparse matrix to compensate for error.
Source: arXiv cs.LG
Source: arXiv cs.LG
Quantization of Large Models with Weight Outliers and Sparsity
Abstract page for arXiv paper 2611.00286: Quantization of Large Models with Weight Outliers and Sparsity
arxiv.org
November 3, 2026 at 11:20 AM
Post-training quantization reduces the memory burden of large model deployment by casting weights to fewer bits. A new method decomposes original trained weights into a quantized matrix, a low-rank component, and a sparse matrix to compensate for error.
Source: arXiv cs.LG
Source: arXiv cs.LG
BFloat16 mixed precision training on TFloat32 tensor cores, followed by quantization-aware training (from BFloat16 to fake Int8) and then post-training quantization to w8a8 with nearest-power-of-two weight snapping is the darkest magic I've ever attempted to conjure. Not sure how or whether it works
November 3, 2026 at 9:55 AM
BFloat16 mixed precision training on TFloat32 tensor cores, followed by quantization-aware training (from BFloat16 to fake Int8) and then post-training quantization to w8a8 with nearest-power-of-two weight snapping is the darkest magic I've ever attempted to conjure. Not sure how or whether it works
Today in #PHY131 #QuantumForEveryone: The midterm exam, which covers established quantum technologies involving quantization of light, quantization of energy in atoms, particle-wave duality, quantization of energy in solids, tunneling, and quantization of angular momentum.
November 3, 2026 at 8:52 PM
Today in #PHY131 #QuantumForEveryone: The midterm exam, which covers established quantum technologies involving quantization of light, quantization of energy in atoms, particle-wave duality, quantization of energy in solids, tunneling, and quantization of angular momentum.
DGM Is Dead; Long Live DGM-Star! A Quantization-Theoretic Approach to Gamma Rate Discretization https://www.biorxiv.org/content/10.64898/2026.10.30.763052v1
November 1, 2026 at 2:32 AM
DGM Is Dead; Long Live DGM-Star! A Quantization-Theoretic Approach to Gamma Rate Discretization https://www.biorxiv.org/content/10.64898/2026.10.30.763052v1
Quantization of accounts
November 3, 2026 at 7:00 AM
Quantization of accounts
It's not that you can't make music, you just haven't tried a midi controller with scale quantization yet
October 28, 2026 at 6:34 PM
It's not that you can't make music, you just haven't tried a midi controller with scale quantization yet
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 6:17 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Sokolovskii et al. (2026-10 preprint, bioRxiv)
DGM Is Dead; Long Live DGM-Star! A Quantization-Theoretic Approach to Gamma Rate Discretization
doi.org/10.64898/202...
DGM Is Dead; Long Live DGM-Star! A Quantization-Theoretic Approach to Gamma Rate Discretization
doi.org/10.64898/202...
DGM Is Dead; Long Live DGM-Star! A Quantization-Theoretic Approach to Gamma Rate Discretization
Modeling substitution rate heterogeneity as a discretized Gamma distribution is standard in phylogenetic inference. Ferretti et al. (2026) have recently demonstrated that the most commonly used equipr...
www.biorxiv.org
November 2, 2026 at 12:45 AM
Sokolovskii et al. (2026-10 preprint, bioRxiv)
DGM Is Dead; Long Live DGM-Star! A Quantization-Theoretic Approach to Gamma Rate Discretization
doi.org/10.64898/202...
DGM Is Dead; Long Live DGM-Star! A Quantization-Theoretic Approach to Gamma Rate Discretization
doi.org/10.64898/202...
Fastest month in AI engineering history. Context windows doubled, quantization dropped to sub-2-bit, and evals are constantly shifting.
Track the drift across 260+ models: https://neubench.netlify.app
#AI #OpenSource #MachineLearning
Track the drift across 260+ models: https://neubench.netlify.app
#AI #OpenSource #MachineLearning
November 2, 2026 at 12:05 PM
Fastest month in AI engineering history. Context windows doubled, quantization dropped to sub-2-bit, and evals are constantly shifting.
Track the drift across 260+ models: https://neubench.netlify.app
#AI #OpenSource #MachineLearning
Track the drift across 260+ models: https://neubench.netlify.app
#AI #OpenSource #MachineLearning
MXCode introduces a framework that bridges ultra-low-bit vector quantization with MicroXcaling native FP4 computation.
Source: arXiv cs.LG
Source: arXiv cs.LG
MXCode: Vector Quantization for MX-Native Inference at Scale
Abstract page for arXiv paper 2611.00403: MXCode: Vector Quantization for MX-Native Inference at Scale
arxiv.org
November 3, 2026 at 10:40 AM
MXCode introduces a framework that bridges ultra-low-bit vector quantization with MicroXcaling native FP4 computation.
Source: arXiv cs.LG
Source: arXiv cs.LG
Calibration of Large Model Quantization based on HiFloat4 Format introduces three approaches, HiGPTQ, HiAutoRound, and HiFlatQuant, to enhance model quantization based on the HiF4 format.
Source: arXiv
Source: arXiv
Calibration of Large Model Quantization based on HiFloat4 Format
Abstract page for arXiv paper 2610.41112: Calibration of Large Model Quantization based on HiFloat4 Format
arxiv.org
November 2, 2026 at 9:20 AM
Calibration of Large Model Quantization based on HiFloat4 Format introduces three approaches, HiGPTQ, HiAutoRound, and HiFlatQuant, to enhance model quantization based on the HiF4 format.
Source: arXiv
Source: arXiv
FP8 quantization vs dense weights for coding agents: does the accuracy drop on complex AST manipulation outweigh the inference speedup?
We ran the numbers across local & cloud setups. Compare 260+ models on https://neubench.netlify.app
#AI #MachineLearning #DevTools
We ran the numbers across local & cloud setups. Compare 260+ models on https://neubench.netlify.app
#AI #MachineLearning #DevTools
November 1, 2026 at 6:03 AM
FP8 quantization vs dense weights for coding agents: does the accuracy drop on complex AST manipulation outweigh the inference speedup?
We ran the numbers across local & cloud setups. Compare 260+ models on https://neubench.netlify.app
#AI #MachineLearning #DevTools
We ran the numbers across local & cloud setups. Compare 260+ models on https://neubench.netlify.app
#AI #MachineLearning #DevTools
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 6:40 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 6:14 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Researchers propose QbD-VLA for task-guided mixed-precision quantization of vision-language-action models. Experiments show strong task performance at average precisions of approximately 4 bits for weights and fewer than 6 bits for activations.
Source: arXiv cs.RO
Source: arXiv cs.RO
Quantize by Doing: Task-Guided Mixed-Precision Quantization of Vision-Language-Action Models
Abstract page for arXiv paper 2610.38661: Quantize by Doing: Task-Guided Mixed-Precision Quantization of Vision-Language-Action Models
arxiv.org
October 30, 2026 at 8:00 AM
Researchers propose QbD-VLA for task-guided mixed-precision quantization of vision-language-action models. Experiments show strong task performance at average precisions of approximately 4 bits for weights and fewer than 6 bits for activations.
Source: arXiv cs.RO
Source: arXiv cs.RO
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 7:21 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 7:05 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Imajev-4B finally runs local in GGUF, and the Q4_K_M is just 2.52 GB.
mindchain/imajev-4b-GGUF
https://huggingface.co/mindchain/imajev-4b-GGUF
#AI #LocalLLM #OpenSource #GGUF #Quantization
mindchain/imajev-4b-GGUF
https://huggingface.co/mindchain/imajev-4b-GGUF
#AI #LocalLLM #OpenSource #GGUF #Quantization
November 3, 2026 at 6:43 AM
Imajev-4B finally runs local in GGUF, and the Q4_K_M is just 2.52 GB.
mindchain/imajev-4b-GGUF
https://huggingface.co/mindchain/imajev-4b-GGUF
#AI #LocalLLM #OpenSource #GGUF #Quantization
mindchain/imajev-4b-GGUF
https://huggingface.co/mindchain/imajev-4b-GGUF
#AI #LocalLLM #OpenSource #GGUF #Quantization
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 7:36 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 6:48 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 6:27 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 6:23 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Making Local LLMs Usable on 8GB GPUs
hackernoon.com
November 2, 2026 at 6:20 PM
Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks. #localllm
Interesting shifts happening in Samsung's mobile AI stack. Curious to see how their on-device quantization handles edge constraints vs standard open architectures.
Compare 260+ models on https://neubench.netlify.app
#Samsung #EdgeAI #MachineLearning
Compare 260+ models on https://neubench.netlify.app
#Samsung #EdgeAI #MachineLearning
October 31, 2026 at 2:03 AM
Interesting shifts happening in Samsung's mobile AI stack. Curious to see how their on-device quantization handles edge constraints vs standard open architectures.
Compare 260+ models on https://neubench.netlify.app
#Samsung #EdgeAI #MachineLearning
Compare 260+ models on https://neubench.netlify.app
#Samsung #EdgeAI #MachineLearning