Have you tried this one with vLLM (which supports structured gen with various options for structured gen)? huggingface.co/neuralmagic/...
neuralmagic/SmolLM-1.7B-Instruct-quantized.w8a16 · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
November 11, 2024 at 11:40 AM
Have you tried this one with vLLM (which supports structured gen with various options for structured gen)? huggingface.co/neuralmagic/...
Red Hat Announces Definitive Agreement to Acquire Neural Magic https://buff.ly/3Z389bR
#aiml #ml #redhat #neuralmagic
#aiml #ml #redhat #neuralmagic
Red Hat Announces Definitive Agreement to Acquire Neural Magic
Red Hat announced that it has signed a definitive agreement to acquire Neural Magic, a pioneer in software and algorithms that accelerate generative AI (gen AI) inference workloads.
buff.ly
November 12, 2024 at 3:39 PM
Red Hat Announces Definitive Agreement to Acquire Neural Magic https://buff.ly/3Z389bR
#aiml #ml #redhat #neuralmagic
#aiml #ml #redhat #neuralmagic
You should be able to run vLLM locally. Neural Magic also has several other quantized models that you can try. huggingface.co/collections/...
INT4 LLMs for vLLM - a neuralmagic Collection
Accurate INT4 quantized models by Neural Magic, ready for use with vLLM!
huggingface.co
November 11, 2024 at 11:46 AM
You should be able to run vLLM locally. Neural Magic also has several other quantized models that you can try. huggingface.co/collections/...
Today, RedHat completed the acquisition of @NeuralMagic, a pioneer in software and algorithms that accelerate #GenAI inference workloads. Read how we are accelerating our vision for #AI’s future: red.ht/408kJ8K.
Red Hat Completes Acquisition of Neural Magic to Fuel Optimized Generative AI Innovation Across the Hybrid Cloud
Red Hat has completed its acquisition of Neural Magic, a pioneer in software and algorithms that accelerate generative AI (gen AI) inference workloads, furthering its vision of high-performing AI work...
red.ht
January 14, 2025 at 12:13 PM
Today, RedHat completed the acquisition of @NeuralMagic, a pioneer in software and algorithms that accelerate #GenAI inference workloads. Read how we are accelerating our vision for #AI’s future: red.ht/408kJ8K.
Today, Red Hat completed the acquisition of @NeuralMagic, a pioneer in software and algorithms that accelerate #GenAI inference workloads. Read how Red Hat is accelerating their vision for #AI’s future: red.ht/408kJ8K
Red Hat Completes Acquisition of Neural Magic to Fuel Optimized Generative AI Innovation Across the Hybrid Cloud
Red Hat has completed its acquisition of Neural Magic, a pioneer in software and algorithms that accelerate generative AI (gen AI) inference workloads, furthering its vision of high-performing AI workloads...
red.ht
January 13, 2025 at 3:39 PM
Today, Red Hat completed the acquisition of @NeuralMagic, a pioneer in software and algorithms that accelerate #GenAI inference workloads. Read how Red Hat is accelerating their vision for #AI’s future: red.ht/408kJ8K
Opinion: quantized models punching above their weight class
#neuralmagic #redhat #ai #artificial-intelligence #agentic-system #llm
#neuralmagic #redhat #ai #artificial-intelligence #agentic-system #llm
Quantized DeepSeek-R1 Models: Deployment-Ready Reasoning Models - Neural Magic
State-of-the-art, open-source quantized reasoning models built on the DeepSeek-R1-Distill suite are now available.
neuralmagic.com
February 28, 2025 at 10:33 AM
Opinion: quantized models punching above their weight class
#neuralmagic #redhat #ai #artificial-intelligence #agentic-system #llm
#neuralmagic #redhat #ai #artificial-intelligence #agentic-system #llm
DeepSparse is a CPU inference runtime that takes advantage of sparsity to accelerate neural network inference. Coupled with SparseML, our optimization library for pruning and quantizing your models, DeepSparse delivers exceptional inference performance on CPU hardware.
GitHub - neuralmagic/deepsparse: Sparsity-aware deep learning inference runtime for CPUs
Sparsity-aware deep learning inference runtime for CPUs - GitHub - neuralmagic/deepsparse: Sparsity-aware deep learning inference runtime for CPUs
github.com
November 27, 2023 at 4:12 AM
DeepSparse is a CPU inference runtime that takes advantage of sparsity to accelerate neural network inference. Coupled with SparseML, our optimization library for pruning and quantizing your models, DeepSparse delivers exceptional inference performance on CPU hardware.
Enhancing DeepSeek Models with MLA and FP8 Optimizations in VLLM Article URL: https://neuralmagic...
https://neuralmagic.com/blog/enhancing-deepseek-models-with-mla-and-fp8-optimizations-in-vllm/
Event Attributes
https://neuralmagic.com/blog/enhancing-deepseek-models-with-mla-and-fp8-optimizations-in-vllm/
Event Attributes
Enhancing DeepSeek Models with MLA and FP8 Optimizations in vLLM - Neural Magic
Enhanced Performance: DeepSeek models see up to 3x throughput and 10x memory capacity improvements with MLA and FP8 kernel optimizations in vLLM v0.7.1.
neuralmagic.com
February 24, 2025 at 10:12 AM
Enhancing DeepSeek Models with MLA and FP8 Optimizations in VLLM Article URL: https://neuralmagic...
https://neuralmagic.com/blog/enhancing-deepseek-models-with-mla-and-fp8-optimizations-in-vllm/
Event Attributes
https://neuralmagic.com/blog/enhancing-deepseek-models-with-mla-and-fp8-optimizations-in-vllm/
Event Attributes
Enhancing DeepSeek Models with MLA and FP8 Optimizations in VLLM Article URL: https://neuralmagic...
https://neuralmagic.com/blog/enhancing-deepseek-models-with-mla-and-fp8-optimizations-in-vllm/
Event Attributes
https://neuralmagic.com/blog/enhancing-deepseek-models-with-mla-and-fp8-optimizations-in-vllm/
Event Attributes
Enhancing DeepSeek Models with MLA and FP8 Optimizations in vLLM - Neural Magic
Enhanced Performance: DeepSeek models see up to 3x throughput and 10x memory capacity improvements with MLA and FP8 kernel optimizations in vLLM v0.7.1.
neuralmagic.com
February 24, 2025 at 9:36 AM
Enhancing DeepSeek Models with MLA and FP8 Optimizations in VLLM Article URL: https://neuralmagic...
https://neuralmagic.com/blog/enhancing-deepseek-models-with-mla-and-fp8-optimizations-in-vllm/
Event Attributes
https://neuralmagic.com/blog/enhancing-deepseek-models-with-mla-and-fp8-optimizations-in-vllm/
Event Attributes
Acquisitions: #Chainalysis > #Alterya; #Moody’s > #CapeAnalytics; #RedHat > #NeuralMagic; #Spiideo > #Signality; #SuperDial > #MajorBoost; #Symbotic > #Walmart’s robotics division; #ZenaTech > #WeddleSurvey
January 17, 2025 at 8:24 PM
Acquisitions: #Chainalysis > #Alterya; #Moody’s > #CapeAnalytics; #RedHat > #NeuralMagic; #Spiideo > #Signality; #SuperDial > #MajorBoost; #Symbotic > #Walmart’s robotics division; #ZenaTech > #WeddleSurvey
📦 neuralmagic / deepsparse
⭐ 1,899 (+64)
🗒 Python
Sparsity-aware deep learning inference runtime for CPUs
⭐ 1,899 (+64)
🗒 Python
Sparsity-aware deep learning inference runtime for CPUs
GitHub - neuralmagic/deepsparse: Sparsity-aware deep learning inference runtime for CPUs
Sparsity-aware deep learning inference runtime for CPUs - GitHub - neuralmagic/deepsparse: Sparsity-aware deep learning inference runtime for CPUs
github.com
October 21, 2023 at 10:50 PM
📦 neuralmagic / deepsparse
⭐ 1,899 (+64)
🗒 Python
Sparsity-aware deep learning inference runtime for CPUs
⭐ 1,899 (+64)
🗒 Python
Sparsity-aware deep learning inference runtime for CPUs
Red Hat adquire startup de otimização de IA Neural Magic
#RedHat #NeuralMagic #IA #InteligenciaArtificial #Software #Tecnologia #Inovação #Aquisição #Processadores #OpenSource
#RedHat #NeuralMagic #IA #InteligenciaArtificial #Software #Tecnologia #Inovação #Aquisição #Processadores #OpenSource
Red Hat adquire startup de otimização de IA Neural Magic - Tecnocrata
…
tecnocrata.com.br
November 12, 2024 at 1:45 PM
Red Hat adquire startup de otimização de IA Neural Magic
#RedHat #NeuralMagic #IA #InteligenciaArtificial #Software #Tecnologia #Inovação #Aquisição #Processadores #OpenSource
#RedHat #NeuralMagic #IA #InteligenciaArtificial #Software #Tecnologia #Inovação #Aquisição #Processadores #OpenSource
Congratulations to @neuralmagic and @ZapataComputing recognized by @BostInno as "Startups to Watch in 2020" - we definitely agree. https://www.americaninno.com/boston/20-Startups-to-Watch-in-2020-Bos
November 17, 2024 at 11:21 AM
Congratulations to @neuralmagic and @ZapataComputing recognized by @BostInno as "Startups to Watch in 2020" - we definitely agree. https://www.americaninno.com/boston/20-Startups-to-Watch-in-2020-Bos