We’re on a journey to advance and democratize artificial intelligence through open source and open science.

アメリカのAIスタートアップであるZyphraが、自然言語処理モデルの「Zamba2-7B」をリリースしました。Zyphraは、Zamba2-7BがGoogleのGemmaやMetaのLlama 3シリーズを上回るパフォーマンスを発揮するとアピールしています。

Train, tune, and infer Bamba model. Contribute to foundation-model-stack/bamba development by creating an account on GitHub.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Another question, Alanturner2: > `cache_position = len(sequence)` but the `class Mamba2Model(Mamba2PreTrainedModel)` initializes the cache position as: `cache_position = torch.arange(0, self.config.conv_kernel, device=inputs_embeds.device)` And the cache postion is required to be `Optional[torch.LongTensor]`, therefore I am wondering why ``cache_position = len(sequence)` ` should be sufficient

Zyphra released Zamba2-VL, an open-source vision-language model family that uses a Mamba2-transformer hybrid architecture to target lower-latency multimodal inference for documents, OCR, counting and edge AI tasks.

Official Hugging Face namespace: nvidia; Model ID: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16; Pipeline: text-generation; Downloads: 336; Tags: transformers, safetensors, nemotron_h, text-generation, nvidia, pytorch, nemotron-3…

Linear Transformers have gained attention as efficient alternatives to standard Transformers, but their performance in retrieval and long-context tasks has been limited. To address these limitations, ...

Official Hugging Face namespace: nvidia; Model ID: nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-FP8; Pipeline: text-generation; Downloads: 43; Tags: transformers, safetensors, nemotron_h_puzzle, text-generation, nvidia, pytorch, nemotron-3…

NVIDIA has released Nemotron-Nano-3-30B-A3B-NVFP4, a production checkpoint that runs a 30B parameter reasoning model in 4 bit NVFP4 format while keeping accuracy close to its BF16 baseline. The model combines a hybrid Mamba2 Transformer Mixture of Experts architecture with a Quantization Aware Distillation (QAD) recipe designed specifically for NVFP4 deployment. Overall, it is an ultra-efficient NVFP4 precision version of Nemotron-3-Nano that delivers up to 4x higher throughput on Blackwell B200.

Technology Innovation Institute (TII), Abu Dhabi, has released Falcon-H1R-7B, a 7B parameter reasoning specialized model that matches or exceeds many 14B to 47B reasoning models in math, code and general benchmarks, while staying compact and efficient. It builds on Falcon H1 7B Base and is available on Hugging Face under the Falcon-H1R collection. Falcon-H1R-7B is interesting because it combines 3 design choices in 1 system, a hybrid Transformer along with Mamba2 backbone, a very long context that reaches 256k tokens in standard vLLM deployments, and a training recipe that mixes supervised long form reasoning with reinforcement learning using GRPO.