ooooo also, there’s a DSpark SD module so it’s fucking fast too
ooooo also, there’s a DSpark SD module so it’s fucking fast too
怎么用:权重 Apache 2.0 协议,BF16、FP8、NVFP4 三种精度都在 Hugging Face 上,保留了 Qwen3.8-27B 的架构和 25.6 万 token 上下文。官方推荐用 SGLang + DSpark 推测解码来跑,模型默认先思考再回答,推理过程放在 reasoning_content 字段里。
怎么用:权重 Apache 2.0 协议,BF16、FP8、NVFP4 三种精度都在 Hugging Face 上,保留了 Qwen3.8-27B 的架构和 25.6 万 token 上下文。官方推荐用 SGLang + DSpark 推测解码来跑,模型默认先思考再回答,推理过程放在 reasoning_content 字段里。
Sparse Attention is the technique of being selective about what tokens can attend to what others (normally, every token attends to every one behind it, at least in decoder-only models (which is ~everything these days))
Sparse Attention is the technique of being selective about what tokens can attend to what others (normally, every token attends to every one behind it, at least in decoder-only models (which is ~everything these days))
getrigbase.com/builds/4x-dg...
getrigbase.com/builds/4x-dg...
핵심 요약
• NVIDIA가 기술 블로그 튜토리얼에서 DGX Spark 64GB에 TensorRT Edge-LLM 0.11.1을 설치해 Nemotron 3.5 Lightning(전체 30B, 활성 3B, NVFP4)을 실행하는 절차를 공개했습니다.
• trtedge chat 명령 하나로 약 21GB 모델 다운로드, 엔진 빌드, 채팅이 이어지고, trtedge serve로 Open…
핵심 요약
• NVIDIA가 기술 블로그 튜토리얼에서 DGX Spark 64GB에 TensorRT Edge-LLM 0.11.1을 설치해 Nemotron 3.5 Lightning(전체 30B, 활성 3B, NVFP4)을 실행하는 절차를 공개했습니다.
• trtedge chat 명령 하나로 약 21GB 모델 다운로드, 엔진 빌드, 채팅이 이어지고, trtedge serve로 Open…
Credits: github.com/deepseek-ai/DeepInfer
#deepinfer #deepseek #python
Credits: github.com/deepseek-ai/DeepInfer
#deepinfer #deepseek #python
Up to 2.43x speedup (BF16, GSM8K).
Like & download:
https://modelscope.cn/models/XHToken/Spark-X2.6-1.7B-DSpark-GGUF
https://huggingface.co/XHToken/Spark-X2.6-1.7B-DSpark-GGUF
Up to 2.43x speedup (BF16, GSM8K).
Like & download:
https://modelscope.cn/models/XHToken/Spark-X2.6-1.7B-DSpark-GGUF
https://huggingface.co/XHToken/Spark-X2.6-1.7B-DSpark-GGUF
Average BF16 speedup: 2.14x RTX 4090 / 2.50x M5 Pro.
❤️ Like & download!
ModelScope: https://modelscope.cn/models/XHToken/Spark-X2.6-4B-DSpark-GGUF
https://huggingface.co/XHToken/Spark-X2.6-4B-DSpark-GGUF
Average BF16 speedup: 2.14x RTX 4090 / 2.50x M5 Pro.
❤️ Like & download!
ModelScope: https://modelscope.cn/models/XHToken/Spark-X2.6-4B-DSpark-GGUF
https://huggingface.co/XHToken/Spark-X2.6-4B-DSpark-GGUF
SparkLLM's new DSpark draft model, announced on X, advertises 2.50x on an M5 Pro. That's the full-precision build, ending at 77 tok/s. The 4-bit build gains much less and still lands at 115. I'd pick on final speed.
SparkLLM's new DSpark draft model, announced on X, advertises 2.50x on an M5 Pro. That's the full-precision build, ending at 77 tok/s. The 4-bit build gains much less and still lands at 115. I'd pick on final speed.
✨ MiniCPM OCR 1.0
- 1.3B
- Apache 2.0
- 55% higher processing efficiency with DSpark
- Handles complex layouts, tables & formulas
✨ MiniCPM OCR 1.0
- 1.3B
- Apache 2.0
- 55% higher processing efficiency with DSpark
- Handles complex layouts, tables & formulas
https://deployedbyai.com/log/14de670b93fa/minicpm-ocr-1-0-dspark/?k=br
https://deployedbyai.com/log/14de670b93fa/minicpm-ocr-1-0-dspark/?k=br