llama.cpp nightly b12284 merged an FP8 KV cache (#28392) and CUDA OCP FP8 (#28413). FP8 is half the bits of FP16, so more context fits the same VRAM. PR author's own bench, RTX PRO 4500 Blackwell, Qwen3.6-27B NVFP4: 37.6 t/s tg256, 34.1 t/s at 32K. Not our measurement.
November 12, 2026 at 8:32 AM
llama.cpp nightly b12284 merged an FP8 KV cache (#28392) and CUDA OCP FP8 (#28413). FP8 is half the bits of FP16, so more context fits the same VRAM. PR author's own bench, RTX PRO 4500 Blackwell, Qwen3.6-27B NVFP4: 37.6 t/s tg256, 34.1 t/s at 32K. Not our measurement.
Debería poder recuperarse. Es raro que no lo haga con algo tan tonto. Test Q8 y si sigue igual ya me da curiosidad. Uso FP8 que es algo peor que Q8, vLLM multiplica fp8xfp8 por defecto, lo parcheo para que haga bf16xbf16; solo store necesita fp8, osea que a veces puede ser el engine.
November 13, 2026 at 11:11 AM
Debería poder recuperarse. Es raro que no lo haga con algo tan tonto. Test Q8 y si sigue igual ya me da curiosidad. Uso FP8 que es algo peor que Q8, vLLM multiplica fp8xfp8 por defecto, lo parcheo para que haga bf16xbf16; solo store necesita fp8, osea que a veces puede ser el engine.
Just added to the catalog:
DeepGEMM-Ascend: Fast GEMM Kernels for Huawei Ascend NPUs
https://osrepos.com/repo/deepseek-ai-deepgemm-ascend?utm_source=bluesky
DeepGEMM-Ascend ports DeepSeek's DeepGEMM library to Huawei Ascend NPUs, with API-compatible kernels for BF16, FP8 and FP4…
#opensource
DeepGEMM-Ascend: Fast GEMM Kernels for Huawei Ascend NPUs
https://osrepos.com/repo/deepseek-ai-deepgemm-ascend?utm_source=bluesky
DeepGEMM-Ascend ports DeepSeek's DeepGEMM library to Huawei Ascend NPUs, with API-compatible kernels for BF16, FP8 and FP4…
#opensource
DeepGEMM-Ascend: Fast GEMM Kernels for Huawei Ascend NPUs
DeepGEMM-Ascend ports DeepSeek's DeepGEMM library to Huawei Ascend NPUs, with API-compatible kernels for BF16, FP8 and FP4 GEMM, MQA logits, and MegaMoE. It targets teams optimizing inference and training workloads on Ascend 950 hardware.
osrepos.com
November 14, 2026 at 12:20 AM
Just added to the catalog:
DeepGEMM-Ascend: Fast GEMM Kernels for Huawei Ascend NPUs
https://osrepos.com/repo/deepseek-ai-deepgemm-ascend?utm_source=bluesky
DeepGEMM-Ascend ports DeepSeek's DeepGEMM library to Huawei Ascend NPUs, with API-compatible kernels for BF16, FP8 and FP4…
#opensource
DeepGEMM-Ascend: Fast GEMM Kernels for Huawei Ascend NPUs
https://osrepos.com/repo/deepseek-ai-deepgemm-ascend?utm_source=bluesky
DeepGEMM-Ascend ports DeepSeek's DeepGEMM library to Huawei Ascend NPUs, with API-compatible kernels for BF16, FP8 and FP4…
#opensource
llama.cpp v1.0.0
Hardens the server against SSRF and DNS rebinding, requires opt-in for tools on non-loopback binds, and adds CUDA FP8, Vulkan Intel flash attention and Hexagon Q5 quantization.
Hardens the server against SSRF and DNS rebinding, requires opt-in for tools on non-loopback binds, and adds CUDA FP8, Vulkan Intel flash attention and Hexagon Q5 quantization.
llama.cpp v1.0.0
Hardens the server against SSRF and DNS rebinding, requires opt-in for tools on non-loopback binds, and adds CUDA FP8, Vulkan Intel flash attention and Hexagon Q5 quantization.
whatsnew.fyi
November 13, 2026 at 12:32 PM
llama.cpp v1.0.0
Hardens the server against SSRF and DNS rebinding, requires opt-in for tools on non-loopback binds, and adds CUDA FP8, Vulkan Intel flash attention and Hexagon Q5 quantization.
Hardens the server against SSRF and DNS rebinding, requires opt-in for tools on non-loopback binds, and adds CUDA FP8, Vulkan Intel flash attention and Hexagon Q5 quantization.
Fused MoE kernels get a boost on NVIDIA H200: kernel tweaks (WGMMA layout fixes), smarter FP8 routing, and Triton JIT specialization cuts TTFT ~9% and throughput up to 10%. 🚀⚡ #MoE #LLM #GPU #CUDA #vLLM
Mixture-of-Experts kernel engineering on Amazon SageMaker AI | Amazon Web Services
AWS engineers share kernel optimizations for fused Mixture-of-Experts (MoE) models on Amazon SageMaker AI: Swap-AB for WGMMA utilization, Triton backend routing for FP8 quantization, and eliminating Triton JIT recompilation. Together they deliver up to 10% higher throughput and up to 9% faster Time to First Token on H200 GPUs.
aws.amazon.com
November 13, 2026 at 12:39 AM
📰 Reddit r/LocalLLaMA
llama.cpp: Add FP8 KV-cache
https://www.reddit.com/r/LocalLLaMA/comments/1xt9o3t/llamacpp_add_fp8_kvcache/
#IA #AI #ML
llama.cpp: Add FP8 KV-cache
https://www.reddit.com/r/LocalLLaMA/comments/1xt9o3t/llamacpp_add_fp8_kvcache/
#IA #AI #ML
November 11, 2026 at 7:01 AM
📰 Reddit r/LocalLLaMA
llama.cpp: Add FP8 KV-cache
https://www.reddit.com/r/LocalLLaMA/comments/1xt9o3t/llamacpp_add_fp8_kvcache/
#IA #AI #ML
llama.cpp: Add FP8 KV-cache
https://www.reddit.com/r/LocalLLaMA/comments/1xt9o3t/llamacpp_add_fp8_kvcache/
#IA #AI #ML
ggml v1.0.0 - 폐기 예정 API 제거, FP8 지원과 백엔드 성능 개선
llama.cpp의 텐서 연산 라이브러리입니다. CUDA에 OCP FP8 지원과 배치 크기 2~24용 양자화 행렬 곱셈을 추가하고, Metal에서는 MoE 라우팅 연산을 하나로 합쳤습니다.
공통 스케줄러는 읽지 않는 입력의 업로드를 생략하고, MoE 캐시는 캐시 미스의 일부를 CPU에서 계산한 뒤 결과를 비동기로 복사할 수 있습니다.
llama.cpp의 텐서 연산 라이브러리입니다. CUDA에 OCP FP8 지원과 배치 크기 2~24용 양자화 행렬 곱셈을 추가하고, Metal에서는 MoE 라우팅 연산을 하나로 합쳤습니다.
공통 스케줄러는 읽지 않는 입력의 업로드를 생략하고, MoE 캐시는 캐시 미스의 일부를 CPU에서 계산한 뒤 결과를 비동기로 복사할 수 있습니다.
ggml v1.0.0 - 폐기 예정 API 제거, FP8 지원과 백엔드 성능 개선
ggml v1.0.0 - 폐기 예정 API 제거, FP8 지원과 백엔드 성능 개선
llama.cpp의 텐서 연산 라이브러리입니다. CUDA에 OCP FP8 지원과 배치 크기 2~24용 양자화 행렬 곱셈을 추가하고, Metal에서는 MoE 라우팅 연산을 하나로 합쳤습니다.
공통 스케줄러는 읽지 않는 입력의 업로드를 생략하고, MoE 캐시는 캐시 미스의 일부를 CPU에서 계산한 뒤 결과를 비동기로 복사할 수 있습니다.
news.hada.io
November 15, 2026 at 12:41 AM
ggml v1.0.0 - 폐기 예정 API 제거, FP8 지원과 백엔드 성능 개선
llama.cpp의 텐서 연산 라이브러리입니다. CUDA에 OCP FP8 지원과 배치 크기 2~24용 양자화 행렬 곱셈을 추가하고, Metal에서는 MoE 라우팅 연산을 하나로 합쳤습니다.
공통 스케줄러는 읽지 않는 입력의 업로드를 생략하고, MoE 캐시는 캐시 미스의 일부를 CPU에서 계산한 뒤 결과를 비동기로 복사할 수 있습니다.
llama.cpp의 텐서 연산 라이브러리입니다. CUDA에 OCP FP8 지원과 배치 크기 2~24용 양자화 행렬 곱셈을 추가하고, Metal에서는 MoE 라우팅 연산을 하나로 합쳤습니다.
공통 스케줄러는 읽지 않는 입력의 업로드를 생략하고, MoE 캐시는 캐시 미스의 일부를 CPU에서 계산한 뒤 결과를 비동기로 복사할 수 있습니다.
llama.cpp b12284 adds CUDA OCP FP8 support
November 11, 2026 at 9:30 AM
llama.cpp b12284 adds CUDA OCP FP8 support
China's AI stockpiles & $1M gray-market servers buy 6-12 months. Beyond that, bottlenecks hit: Huawei's Ascend lags in FP8, and clustering chips can't match top Nvidia silicon. A hard structural crunch is approaching.
#AI #Semiconductors #China
#AI #Semiconductors #China
November 13, 2026 at 7:34 PM
China's AI stockpiles & $1M gray-market servers buy 6-12 months. Beyond that, bottlenecks hit: Huawei's Ascend lags in FP8, and clustering chips can't match top Nvidia silicon. A hard structural crunch is approaching.
#AI #Semiconductors #China
#AI #Semiconductors #China
ggml v1.0.0 - 폐기 예정 API 제거, FP8 지원과 백엔드 성능 개선
ggml 1.0.0 은 폐기 예정 공개 API와 기존 DIAG_MASK 연산을 제거하고, YaRN 설정과 FP8 지원을 확장한 릴리스임 RPC 프로토콜 v9 은 호환되지 않는 상대의 연결을 거부하며, 로딩 속도와 연결 실패 처리를 개선함 CUDA와 Metal 은 행 수가 적은 행렬 곱셈과 MoE 연산을 최적화하고, ...
ggml 1.0.0 은 폐기 예정 공개 API와 기존 DIAG_MASK 연산을 제거하고, YaRN 설정과 FP8 지원을 확장한 릴리스임 RPC 프로토콜 v9 은 호환되지 않는 상대의 연결을 거부하며, 로딩 속도와 연결 실패 처리를 개선함 CUDA와 Metal 은 행 수가 적은 행렬 곱셈과 MoE 연산을 최적화하고, ...
ggml v1.0.0 - 폐기 예정 API 제거, FP8 지원과 백엔드 성능 개선
ggml 1.0.0 은 폐기 예정 공개 API와 기존 DIAG_MASK 연산을 제거하고, YaRN 설정과 FP8 지원을 확장한 릴리스임 RPC 프로토콜 v9 은 호환되지 않는 상대의 연결을 거부하며, 로딩 속도와 연결 실패 처리를 개선함 CUDA와 Metal 은 행 수가 적은 행렬 곱셈과 MoE 연산을 최적화하고, ...
news.hada.io
November 14, 2026 at 9:00 AM
ggml v1.0.0 - 폐기 예정 API 제거, FP8 지원과 백엔드 성능 개선
ggml 1.0.0 은 폐기 예정 공개 API와 기존 DIAG_MASK 연산을 제거하고, YaRN 설정과 FP8 지원을 확장한 릴리스임 RPC 프로토콜 v9 은 호환되지 않는 상대의 연결을 거부하며, 로딩 속도와 연결 실패 처리를 개선함 CUDA와 Metal 은 행 수가 적은 행렬 곱셈과 MoE 연산을 최적화하고, ...
ggml 1.0.0 은 폐기 예정 공개 API와 기존 DIAG_MASK 연산을 제거하고, YaRN 설정과 FP8 지원을 확장한 릴리스임 RPC 프로토콜 v9 은 호환되지 않는 상대의 연결을 거부하며, 로딩 속도와 연결 실패 처리를 개선함 CUDA와 Metal 은 행 수가 적은 행렬 곱셈과 MoE 연산을 최적화하고, ...
Nemotron-4 340B is 680GB of weights in bf16; Nvidia's recipe is fp8 across 8x H100, ~$16-24/hr rented, before a token. The license allows commercial use, so it's capex vs opex. Where do you put the breakeven against per-token pricing?
November 12, 2026 at 11:02 PM
Nemotron-4 340B is 680GB of weights in bf16; Nvidia's recipe is fp8 across 8x H100, ~$16-24/hr rented, before a token. The license allows commercial use, so it's capex vs opex. Where do you put the breakeven against per-token pricing?
llama.cpp tagged v1.0.0 on November 13 after a three-file version-bump pull request, with a changelog that includes FP8 KV-cache support, a host-memory MoE expert cache and SSRF and DNS-rebinding fixes.
llama.cpp reaches version 1.0.0: FP8 KV cache, a host-memory expert cache and server hardening
llama.cpp tagged v1.0.0 on November 13 after a three-file version-bump pull request, with a changelog that includes FP8 KV-cache support, a host-memory MoE expert cache and SSRF and DNS-rebinding fixes.
groundtruth.day
November 13, 2026 at 2:07 PM
llama.cpp tagged v1.0.0 on November 13 after a three-file version-bump pull request, with a changelog that includes FP8 KV-cache support, a host-memory MoE expert cache and SSRF and DNS-rebinding fixes.
Ggml v1.0.0
https://github.com/ggml-org/ggml/releases/tag/v1.0.0
https://news.ycombinator.com/item?id=50493062
https://github.com/ggml-org/ggml/releases/tag/v1.0.0
https://news.ycombinator.com/item?id=50493062
Release v1.0.0 · ggml-org/ggml
Overview
ggml 1.0.0 prunes the deprecated public API and the legacy DIAG_MASK ops, adds configurable YaRN truncation and FP8 support, and brings broad performance work across backends: few-row quan...
github.com
November 14, 2026 at 5:30 AM
llama.cpp 1.0.0 is out.
The release adds FP8 KV-cache support and an OpenAI-compatible speech-generation endpoint.
https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0
The release adds FP8 KV-cache support and an OpenAI-compatible speech-generation endpoint.
https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0
November 13, 2026 at 12:27 PM
llama.cpp 1.0.0 is out.
The release adds FP8 KV-cache support and an OpenAI-compatible speech-generation endpoint.
https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0
The release adds FP8 KV-cache support and an OpenAI-compatible speech-generation endpoint.
https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0
The source, dated, on The Deploy Log:
https://deployedbyai.com/log/6c6a007a883d/llama-cpp-cuda-ocp-fp8/?k=br
Follow for one line every time AI actually ships, as it ships, 24/7.
https://deployedbyai.com/log/6c6a007a883d/llama-cpp-cuda-ocp-fp8/?k=br
Follow for one line every time AI actually ships, as it ships, 24/7.
llama.cpp CUDA OCP FP8
llama.cpp, 11 Nov 2026. Sourced to github.com.
deployedbyai.com
November 11, 2026 at 9:30 AM
The source, dated, on The Deploy Log:
https://deployedbyai.com/log/6c6a007a883d/llama-cpp-cuda-ocp-fp8/?k=br
Follow for one line every time AI actually ships, as it ships, 24/7.
https://deployedbyai.com/log/6c6a007a883d/llama-cpp-cuda-ocp-fp8/?k=br
Follow for one line every time AI actually ships, as it ships, 24/7.
llama.cpp v1.0.0リリースとFP8キャッシュ・UI刷新でローカル推論が成熟
ggml-org開発チームが2026年11月13日、ローカルLLM実行エンジン「llama.cpp」のメジャーバージョン1.0.0を公開した。
https://648news.com/local_ai/2026/11/14-llama-cpp-v1-release-fp8-ui-refresh
#ローカルLLM
ggml-org開発チームが2026年11月13日、ローカルLLM実行エンジン「llama.cpp」のメジャーバージョン1.0.0を公開した。
https://648news.com/local_ai/2026/11/14-llama-cpp-v1-release-fp8-ui-refresh
#ローカルLLM
llama.cpp v1.0.0リリースとFP8キャッシュ・UI刷新でローカル推論が成熟
ggml-org開発チームが2026年11月13日、ローカルLLM実行エンジン「llama.cpp」のメジャーバージョン1.0.0を公開した。非推奨APIの整理とFP8形式のKVキャッシュ対応により、PC上のメモリ効率が大幅に向上する。サーバーのセキュリティ強化とリデザインされたWeb UIの同梱で、自作PCやmacOSでモデルを動かす愛好家向けの実用性が一段階引き上げられた。これにより既存のワークフローへの移行もスムーズに進む見込みだ。コミュニティからはローカル推論における重要な節目だと評価されている。
648news.com
November 13, 2026 at 9:44 PM
llama.cpp v1.0.0リリースとFP8キャッシュ・UI刷新でローカル推論が成熟
ggml-org開発チームが2026年11月13日、ローカルLLM実行エンジン「llama.cpp」のメジャーバージョン1.0.0を公開した。
https://648news.com/local_ai/2026/11/14-llama-cpp-v1-release-fp8-ui-refresh
#ローカルLLM
ggml-org開発チームが2026年11月13日、ローカルLLM実行エンジン「llama.cpp」のメジャーバージョン1.0.0を公開した。
https://648news.com/local_ai/2026/11/14-llama-cpp-v1-release-fp8-ui-refresh
#ローカルLLM
🔌 Arm, Intel and NVIDIA Advance FP8 Standards for Machine Learning
The semiconductor industry is transitioning from high-precision formats to lower-precision 8-bit floating-point data types.
🔗 Read the full story on GazeWire
#Semiconductors #Hardware #Tsmc #GazeWire
The semiconductor industry is transitioning from high-precision formats to lower-precision 8-bit floating-point data types.
🔗 Read the full story on GazeWire
#Semiconductors #Hardware #Tsmc #GazeWire
🔌 Arm, Intel and NVIDIA Advance FP8 Standards for Machine Learning
The semiconductor industry is transitioning from high-precision formats to lower-precision 8-bit floating-point data types.
gazewire.com
November 12, 2026 at 8:53 AM
🔌 Arm, Intel and NVIDIA Advance FP8 Standards for Machine Learning
The semiconductor industry is transitioning from high-precision formats to lower-precision 8-bit floating-point data types.
🔗 Read the full story on GazeWire
#Semiconductors #Hardware #Tsmc #GazeWire
The semiconductor industry is transitioning from high-precision formats to lower-precision 8-bit floating-point data types.
🔗 Read the full story on GazeWire
#Semiconductors #Hardware #Tsmc #GazeWire
Ggml v1.0.0 | Discussion
Release v1.0.0 · ggml-org/ggml
Overview
ggml 1.0.0 prunes the deprecated public API and the legacy DIAG_MASK ops, adds configurable YaRN truncation and FP8 support, and brings broad performance work across backends: few-row quan...
github.com
November 14, 2026 at 3:20 AM
Ggml v1.0.0 | Discussion
Self-hosting an LLM? Our new guide shows how to deploy GLM-4.5-Air-FP8 on a dedicated NVIDIA GPU server with SGLang: hardware, Ubuntu setup, OpenAI-compatible API, systemd & security.
www.leoservers.com/tutorials/ho...
#LLM #GPU #AI
www.leoservers.com/tutorials/ho...
#LLM #GPU #AI
November 13, 2026 at 6:28 AM
Self-hosting an LLM? Our new guide shows how to deploy GLM-4.5-Air-FP8 on a dedicated NVIDIA GPU server with SGLang: hardware, Ubuntu setup, OpenAI-compatible API, systemd & security.
www.leoservers.com/tutorials/ho...
#LLM #GPU #AI
www.leoservers.com/tutorials/ho...
#LLM #GPU #AI
2x Radeon AI PRO R9700, Qwen3.6 27B, one user (Puget Systems): stock vLLM FP8 TP=2 15.9 tok/s, llama.cpp Q4_K_M 23.4, tuned vLLM 32.0, tuned + MTP 62.75. Now 230 builds on RigBase.
getrigbase.com/gpus/radeon-...
getrigbase.com/gpus/radeon-...
November 13, 2026 at 7:22 AM
2x Radeon AI PRO R9700, Qwen3.6 27B, one user (Puget Systems): stock vLLM FP8 TP=2 15.9 tok/s, llama.cpp Q4_K_M 23.4, tuned vLLM 32.0, tuned + MTP 62.75. Now 230 builds on RigBase.
getrigbase.com/gpus/radeon-...
getrigbase.com/gpus/radeon-...
ggml 1.0.0、古いAPIを削って性能と対応先を広げる節目のリリース
ggml の v1.0.0 は、単なる番号の繰り上げではありません。非推奨だった公開APIや古い演算を整理しつつ、YaRN の扱いを調整できるようにし、FP8…
#HackerNews #AI #テック
ggml の v1.0.0 は、単なる番号の繰り上げではありません。非推奨だった公開APIや古い演算を整理しつつ、YaRN の扱いを調整できるようにし、FP8…
#HackerNews #AI #テック
ggml 1.0.0、古いAPIを削って性能と対応先を広げる節目のリリース
ggml の v1.0.0 は、単なる番号の繰り上げではありません。非推奨だった公開APIや古い演算を整理しつつ、YaRN の扱いを調整できるようにし、FP8…
papoo.work
November 15, 2026 at 12:08 AM
ggml 1.0.0、古いAPIを削って性能と対応先を広げる節目のリリース
ggml の v1.0.0 は、単なる番号の繰り上げではありません。非推奨だった公開APIや古い演算を整理しつつ、YaRN の扱いを調整できるようにし、FP8…
#HackerNews #AI #テック
ggml の v1.0.0 は、単なる番号の繰り上げではありません。非推奨だった公開APIや古い演算を整理しつつ、YaRN の扱いを調整できるようにし、FP8…
#HackerNews #AI #テック