lightseekorg / tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
https://github.com/lightseekorg/tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
https://github.com/lightseekorg/tokenspeed
November 1, 2026 at 4:15 AM
lightseekorg / tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
https://github.com/lightseekorg/tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
https://github.com/lightseekorg/tokenspeed
模型是一个原生多模态的混合专家模型，总参数 2.8T，每次推理激活 104B，从 896 个专家里挑 16 个，上下文 1,048,576 个 token，权重以 MXFP4 存放，来自量化感知训练，一共 96 个文件，约 1.56TB。Moonshot 推荐用 vLLM、SGLang 或者 TokenSpeed 来跑。
November 5, 2026 at 2:21 AM
模型是一个原生多模态的混合专家模型，总参数 2.8T，每次推理激活 104B，从 896 个专家里挑 16 个，上下文 1,048,576 个 token，权重以 MXFP4 存放，来自量化感知训练，一共 96 个文件，约 1.56TB。Moonshot 推荐用 vLLM、SGLang 或者 TokenSpeed 来跑。
lightseekorg/tokenspeed: TokenSpeed is a speed-of-light LLM inference engine. #devopsish github.com/lightseek...
October 17, 2026 at 8:02 PM
lightseekorg/tokenspeed: TokenSpeed is a speed-of-light LLM inference engine. #devopsish github.com/lightseek...
📦 lightseekorg / tokenspeed
⭐ 2,295 (+25)
🗒 Python
TokenSpeed is a speed-of-light LLM inference engine.
⭐ 2,295 (+25)
🗒 Python
TokenSpeed is a speed-of-light LLM inference engine.
GitHub - lightseekorg/tokenspeed: TokenSpeed is a speed-of-light LLM inference engine.
TokenSpeed is a speed-of-light LLM inference engine. - lightseekorg/tokenspeed
github.com
October 15, 2026 at 12:17 PM
📦 lightseekorg / tokenspeed
⭐ 2,295 (+25)
🗒 Python
TokenSpeed is a speed-of-light LLM inference engine.
⭐ 2,295 (+25)
🗒 Python
TokenSpeed is a speed-of-light LLM inference engine.
📰 Qwen en X
685 output tokens/s per user on @Alibaba_Qwen Qwen3.8-Flash-Next. Running TokenSpeed on 4× NVIDIA GB300 with NVFP4. A look at the kernel and serving optimizations behind the results: 👉 ⭐
https://x.com/Alibaba_Qwen/status/2110214931559899630
#IA #AI #ML
685 output tokens/s per user on @Alibaba_Qwen Qwen3.8-Flash-Next. Running TokenSpeed on 4× NVIDIA GB300 with NVFP4. A look at the kernel and serving optimizations behind the results: 👉 ⭐
https://x.com/Alibaba_Qwen/status/2110214931559899630
#IA #AI #ML
October 14, 2026 at 11:00 AM
📰 Qwen en X
685 output tokens/s per user on @Alibaba_Qwen Qwen3.8-Flash-Next. Running TokenSpeed on 4× NVIDIA GB300 with NVFP4. A look at the kernel and serving optimizations behind the results: 👉 ⭐
https://x.com/Alibaba_Qwen/status/2110214931559899630
#IA #AI #ML
685 output tokens/s per user on @Alibaba_Qwen Qwen3.8-Flash-Next. Running TokenSpeed on 4× NVIDIA GB300 with NVFP4. A look at the kernel and serving optimizations behind the results: 👉 ⭐
https://x.com/Alibaba_Qwen/status/2110214931559899630
#IA #AI #ML
tokenspeed: a speed-of-light llm inference engine for agentic workloads | lightseek foundation https://lightseek.org/blog/lightseek-tokenspeed.html
September 1, 2026 at 10:29 AM
tokenspeed: a speed-of-light llm inference engine for agentic workloads | lightseek foundation https://lightseek.org/blog/lightseek-tokenspeed.html
PyTorch expands with 10 projects, boosting ML tooling.
PyTorch Ecosystem adds 10 projects, enhancing training, inference, and domain-specific tools like Perforated and RLinf. These innovations enable diverse, production-ready ML stacks, improving real-time data efficiency…
Read more on Kimbodo:
PyTorch Ecosystem adds 10 projects, enhancing training, inference, and domain-specific tools like Perforated and RLinf. These innovations enable diverse, production-ready ML stacks, improving real-time data efficiency…
Read more on Kimbodo:
How PyTorch’s 10 New Projects Change Production ML — and What Data Science Teams Using Python and R Should Do Next
What Happened The PyTorch Ecosystem Landscape added ten projects that expand training, inference, routing, dataset, visualization and domain-specific tooling: Perforated, AReaL, TorchJD, RLinf, Miles, SMG, FiftyOne, TokenSpeed, VisualTorch, and TorchSurv.…
kimbodo.com
August 31, 2026 at 8:21 PM
PyTorch expands with 10 projects, boosting ML tooling.
PyTorch Ecosystem adds 10 projects, enhancing training, inference, and domain-specific tools like Perforated and RLinf. These innovations enable diverse, production-ready ML stacks, improving real-time data efficiency…
Read more on Kimbodo:
PyTorch Ecosystem adds 10 projects, enhancing training, inference, and domain-specific tools like Perforated and RLinf. These innovations enable diverse, production-ready ML stacks, improving real-time data efficiency…
Read more on Kimbodo:
Qwen3.8-27B hits 1M downloads in 2 days, tops Hugging Face trend. 27B dense VLM, 262K context extendable to 1M. Supports image/video, default reasoning mode with adjustable depth. Weights on Transformers, works with vLLM, SGLang, TokenSpeed, Apache 2.0.
#Qwen
aidisruption.ai/p/deepseek-h...
#Qwen
aidisruption.ai/p/deepseek-h...
DeepSeek Harness Plugin Recommendations, Round 2
Discover 8 practical DeepSeek Harness plugins for workflow control, file annotation, browser automation, and custom tool creation. Boost your Agent efficiency today.
aidisruption.ai
August 18, 2026 at 7:42 AM
Qwen3.8-27B hits 1M downloads in 2 days, tops Hugging Face trend. 27B dense VLM, 262K context extendable to 1M. Supports image/video, default reasoning mode with adjustable depth. Weights on Transformers, works with vLLM, SGLang, TokenSpeed, Apache 2.0.
#Qwen
aidisruption.ai/p/deepseek-h...
#Qwen
aidisruption.ai/p/deepseek-h...
necroing but i found this visualization for anyone not willing to spin up an actual LLM to get what the numbers mean mikeveerman.github.io/tokenspeed...
May 11, 2026 at 10:40 AM
necroing but i found this visualization for anyone not willing to spin up an actual LLM to get what the numbers mean mikeveerman.github.io/tokenspeed...
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
papoo.work
May 28, 2026 at 10:31 AM
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
papoo.work
May 24, 2026 at 8:45 PM
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
papoo.work
May 23, 2026 at 2:54 PM
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
https://papoo.work/doc/4944bb87c9b6b1e9
#llm #tokens #benchmark #ai #developer-tools
On NVIDIA B200 hardware, TokenSpeed outperformed TensorRT-LLM by 9% for minimum latency and 11% for throughput.
Its multi-head latent attention kernel nearly halves decoding latency on speculative decoding workloads.
Its multi-head latent attention kernel nearly halves decoding latency on speculative decoding workloads.
Data Points: How Anthropic aligns its models
Hermes, aka the new OpenClaw. DCI, aka the new RAG. NLAs, or how to peek inside Claude. TokenSpeed, aka the new TensorRT.
www.deeplearning.ai
May 21, 2026 at 1:59 AM
On NVIDIA B200 hardware, TokenSpeed outperformed TensorRT-LLM by 9% for minimum latency and 11% for throughput.
Its multi-head latent attention kernel nearly halves decoding latency on speculative decoding workloads.
Its multi-head latent attention kernel nearly halves decoding latency on speculative decoding workloads.
6. TokenSpeed Inference Engine
The LightSeek Foundation released TokenSpeed, an inference engine licensed under the MIT License for coding agents that can handle contexts exceeding 50,000 tokens.
The LightSeek Foundation released TokenSpeed, an inference engine licensed under the MIT License for coding agents that can handle contexts exceeding 50,000 tokens.
May 21, 2026 at 1:59 AM
6. TokenSpeed Inference Engine
The LightSeek Foundation released TokenSpeed, an inference engine licensed under the MIT License for coding agents that can handle contexts exceeding 50,000 tokens.
The LightSeek Foundation released TokenSpeed, an inference engine licensed under the MIT License for coding agents that can handle contexts exceeding 50,000 tokens.
Really cool tool to visualize what token/s really looks like at different speeds, starting from 0.05 tok/s to 2000 tok/s.
After 800 tok/s you can't really tell the difference it all feels the same!
mikeveerman.github.io/tokenspeed/?...
After 800 tok/s you can't really tell the difference it all feels the same!
mikeveerman.github.io/tokenspeed/?...
tokenspeed — feel LLM tokens-per-second
mikeveerman.github.io
June 11, 2026 at 11:44 PM
Really cool tool to visualize what token/s really looks like at different speeds, starting from 0.05 tok/s to 2000 tok/s.
After 800 tok/s you can't really tell the difference it all feels the same!
mikeveerman.github.io/tokenspeed/?...
After 800 tok/s you can't really tell the difference it all feels the same!
mikeveerman.github.io/tokenspeed/?...
How fast is N tokens per second really? https://mikeveerman.github.io/tokenspeed/ (http://news.ycombinator.com/item?id=48174920)
May 20, 2026 at 9:03 PM
How fast is N tokens per second really? https://mikeveerman.github.io/tokenspeed/ (http://news.ycombinator.com/item?id=48174920)
How fast is N tokens per second really? | Discussion
tokenspeed — feel LLM tokens-per-second
mikeveerman.github.io
May 20, 2026 at 6:00 PM
How fast is N tokens per second really? | Discussion
tokenspeed-smg-grpc-proto 0.4.10.post20260621 SMG gRPC proto definitions for vLLM, TRT-LLM, MLX, TokenSpeed, and SGLang
Origin | Interest | Match
Origin | Interest | Match
Client Challenge
pypi.org
June 21, 2026 at 2:02 AM
LLMの「30 tokens/s」はどれくらい速いのかを体感できる「tokenspeed」の面白さ https://papoo.work/doc/4944bb87c9b6b1e9 #llm #tokens #benchmark #ai #developer-tools
Origin | Interest | Match
Origin | Interest | Match
Awakari App
awakari.com
May 23, 2026 at 3:02 PM
阿里最近开源了千问旗舰模型Qwen3.8-2.4T-A95B，首次开放Max级别权重，总结如下：
一、技术特性与开源生态
1参数规模创新：
◦总参数2.4万亿（激活参数950亿/Token），原生支持26万Token上下文（可扩展至101万）
◦基于Qwen3.5架构升级，强化编程、科研及长周期Agent任务能力
2开源部署方案：
◦提供SGLang/vLLM/TokenSpeed推理引擎支持，需按GPU配置调整并行策略
◦Unsloth AI通过1-bit量化技术将模型体积从4.9TB压缩至397GB（降91%），410GB内存设备即可本地运行
一、技术特性与开源生态
1参数规模创新：
◦总参数2.4万亿（激活参数950亿/Token），原生支持26万Token上下文（可扩展至101万）
◦基于Qwen3.5架构升级，强化编程、科研及长周期Agent任务能力
2开源部署方案：
◦提供SGLang/vLLM/TokenSpeed推理引擎支持，需按GPU配置调整并行策略
◦Unsloth AI通过1-bit量化技术将模型体积从4.9TB压缩至397GB（降91%），410GB内存设备即可本地运行
August 14, 2026 at 10:50 AM
阿里最近开源了千问旗舰模型Qwen3.8-2.4T-A95B，首次开放Max级别权重，总结如下：
一、技术特性与开源生态
1参数规模创新：
◦总参数2.4万亿（激活参数950亿/Token），原生支持26万Token上下文（可扩展至101万）
◦基于Qwen3.5架构升级，强化编程、科研及长周期Agent任务能力
2开源部署方案：
◦提供SGLang/vLLM/TokenSpeed推理引擎支持，需按GPU配置调整并行策略
◦Unsloth AI通过1-bit量化技术将模型体积从4.9TB压缩至397GB（降91%），410GB内存设备即可本地运行
一、技术特性与开源生态
1参数规模创新：
◦总参数2.4万亿（激活参数950亿/Token），原生支持26万Token上下文（可扩展至101万）
◦基于Qwen3.5架构升级，强化编程、科研及长周期Agent任务能力
2开源部署方案：
◦提供SGLang/vLLM/TokenSpeed推理引擎支持，需按GPU配置调整并行策略
◦Unsloth AI通过1-bit量化技术将模型体积从4.9TB压缩至397GB（降91%），410GB内存设备即可本地运行
One runtime, multiple GPU architectures, and zero vendor-specific model code.
In this blog post, the TokenSpeed team at Lightseek introduces TokenSpeed-Kernel, a portable, high-performance kernel system built for modern LLM inference.
Link to blog in comments section 👇
In this blog post, the TokenSpeed team at Lightseek introduces TokenSpeed-Kernel, a portable, high-performance kernel system built for modern LLM inference.
Link to blog in comments section 👇
June 25, 2026 at 3:30 PM
One runtime, multiple GPU architectures, and zero vendor-specific model code.
In this blog post, the TokenSpeed team at Lightseek introduces TokenSpeed-Kernel, a portable, high-performance kernel system built for modern LLM inference.
Link to blog in comments section 👇
In this blog post, the TokenSpeed team at Lightseek introduces TokenSpeed-Kernel, a portable, high-performance kernel system built for modern LLM inference.
Link to blog in comments section 👇
The speed-of-light optimization for Qwen3.5 on the TokenSpeed inference engine is a significant milestone, achieving a record-breaking 580 tokens per second (tps) for agentic workloads on NVIDIA GPUs. Check out our latest community blog to learn more 👉 bit.ly/4uGUvIS
May 27, 2026 at 4:02 PM
The speed-of-light optimization for Qwen3.5 on the TokenSpeed inference engine is a significant milestone, achieving a record-breaking 580 tokens per second (tps) for agentic workloads on NVIDIA GPUs. Check out our latest community blog to learn more 👉 bit.ly/4uGUvIS
How fast is N tokens per second really? https://mikeveerman.github.io/tokenspeed/ (http://news.ycombinator.com/item?id=48174920)
May 20, 2026 at 11:03 PM
How fast is N tokens per second really? https://mikeveerman.github.io/tokenspeed/ (http://news.ycombinator.com/item?id=48174920)
TokenSpeed-Kernel just smashed benchmarks on AMD's GPT-OSS 120B using Gluon kernels. 🚀 If you love bleeding-edge LLM performance, you'll want to see the numbers. Dive in! #TokenSpeed #AMD #GPTOSS
🔗 aidailypost.com/news/tokensp...
🔗 aidailypost.com/news/tokensp...
June 25, 2026 at 4:59 PM
TokenSpeed-Kernel just smashed benchmarks on AMD's GPT-OSS 120B using Gluon kernels. 🚀 If you love bleeding-edge LLM performance, you'll want to see the numbers. Dive in! #TokenSpeed #AMD #GPTOSS
🔗 aidailypost.com/news/tokensp...
🔗 aidailypost.com/news/tokensp...