ベクトル検索だけでは足りない ― ハイブリッド検索とCrossEncoder再ランキングの実装手順 | Johnny Ponta #zenn
https://zenn.dev/johnny1024/articles/3ecdf0573f5d73
https://zenn.dev/johnny1024/articles/3ecdf0573f5d73
ベクトル検索だけでは足りない ― ハイブリッド検索とCrossEncoder再ランキングの実装手順
zenn.dev
November 1, 2026 at 4:31 PM
ベクトル検索だけでは足りない ― ハイブリッド検索とCrossEncoder再ランキングの実装手順 | Johnny Ponta #zenn
https://zenn.dev/johnny1024/articles/3ecdf0573f5d73
https://zenn.dev/johnny1024/articles/3ecdf0573f5d73
In Sentence Transformers, set model[0].chunk_size = 128 to try stronger compression, or None to disable pooling.
The CrossEncoder path re-encodes documents on each call, so the opening example doesn't demonstrate the offline document-caching savings.
The CrossEncoder path re-encodes documents on each call, so the opening example doesn't demonstrate the offline document-caching savings.
October 30, 2026 at 2:07 PM
In Sentence Transformers, set model[0].chunk_size = 128 to try stronger compression, or None to disable pooling.
The CrossEncoder path re-encodes documents on each call, so the opening example doesn't demonstrate the offline document-caching savings.
The CrossEncoder path re-encodes documents on each call, so the opening example doesn't demonstrate the offline document-caching savings.
Both models run through CrossEncoder with trust_remote_code=True.
This 61M example runs MICE, but CrossEncoder re-encodes each query-document pair. It doesn't reuse document representations, so you don't get the document-precomputation savings.
This 61M example runs MICE, but CrossEncoder re-encodes each query-document pair. It doesn't reuse document representations, so you don't get the document-precomputation savings.
October 27, 2026 at 12:12 PM
Both models run through CrossEncoder with trust_remote_code=True.
This 61M example runs MICE, but CrossEncoder re-encodes each query-document pair. It doesn't reuse document representations, so you don't get the document-precomputation savings.
This 61M example runs MICE, but CrossEncoder re-encodes each query-document pair. It doesn't reuse document representations, so you don't get the document-precomputation savings.
The extra speed has a storage cost: MICE-P uses 4-bit residual compression versus LateOn's 2-bit, plus a separate SPLADE index.
Loading the model with CrossEncoder gives you pairwise reranking. The indexed pipeline is what adds reusable document representations.
Loading the model with CrossEncoder gives you pairwise reranking. The indexed pipeline is what adds reusable document representations.
October 27, 2026 at 12:12 PM
The extra speed has a storage cost: MICE-P uses 4-bit residual compression versus LateOn's 2-bit, plus a separate SPLADE index.
Loading the model with CrossEncoder gives you pairwise reranking. The indexed pipeline is what adds reusable document representations.
Loading the model with CrossEncoder gives you pairwise reranking. The indexed pipeline is what adds reusable document representations.
For reranking candidates, there's a 1.7B base model and a 149M nano model, both through CrossEncoder.
Here's nano moving the photosynthesis answer to the top. Its scores are relevance logits, so higher is better, even when they're all negative.
Here's nano moving the photosynthesis answer to the top. Its scores are relevance logits, so higher is better, even when they're all negative.
October 23, 2026 at 12:25 PM
For reranking candidates, there's a 1.7B base model and a 149M nano model, both through CrossEncoder.
Here's nano moving the photosynthesis answer to the top. Its scores are relevance logits, so higher is better, even when they're all negative.
Here's nano moving the photosynthesis answer to the top. Its scores are relevance logits, so higher is better, even when they're all negative.
Experimental: keep selected CrossEncoder linear layers computing in FP32 while the backbone uses BF16.
Useful for reducing score ties caused by limited precision. Works in training & inference, including autocast. Stored parameter dtypes stay unchanged.
Useful for reducing score ties caused by limited precision. Works in training & inference, including autocast. Stored parameter dtypes stay unchanged.
October 21, 2026 at 12:35 PM
Experimental: keep selected CrossEncoder linear layers computing in FP32 while the backbone uses BF16.
Useful for reducing score ties caused by limited precision. Works in training & inference, including autocast. Stored parameter dtypes stay unchanged.
Useful for reducing score ties caused by limited precision. Works in training & inference, including autocast. Stored parameter dtypes stay unchanged.
Hybrid Search and Re-Ranker Architectures for Accurate RAG
https://beefed.ai/en/hybrid-search-re-rankers-rag
#HybridSearch #BM25 #VectorEmbeddings #Reranker #Crossencoder
https://beefed.ai/en/hybrid-search-re-rankers-rag
#HybridSearch #BM25 #VectorEmbeddings #Reranker #Crossencoder
Hybrid Search and Re-Ranker Architectures for Accurate RAG
beefed.ai
September 28, 2026 at 9:52 PM
Hybrid Search and Re-Ranker Architectures for Accurate RAG
https://beefed.ai/en/hybrid-search-re-rankers-rag
#HybridSearch #BM25 #VectorEmbeddings #Reranker #Crossencoder
https://beefed.ai/en/hybrid-search-re-rankers-rag
#HybridSearch #BM25 #VectorEmbeddings #Reranker #Crossencoder
Antoine Chaffin, Raphaël Sourty & I wrote a blog post walking through all of this in practice: huggingface.co/blog/multi-v...
pip install sentence-transformers==6.0.0
Full release notes: github.com/huggingface/...
pip install sentence-transformers==6.0.0
Full release notes: github.com/huggingface/...
Release v6.0.0 - MultiVectorEncoder for ColBERT & late interaction models, transformers v5, float32 scoring, faster training & encoding · huggingface/sentence-transformers
This major release introduces Multi-Vector Embedding models, also known as late interaction or ColBERT-style models, as a fourth model type alongside SentenceTransformer, CrossEncoder, and SparseEn...
github.com
August 18, 2026 at 2:01 PM
Antoine Chaffin, Raphaël Sourty & I wrote a blog post walking through all of this in practice: huggingface.co/blog/multi-v...
pip install sentence-transformers==6.0.0
Full release notes: github.com/huggingface/...
pip install sentence-transformers==6.0.0
Full release notes: github.com/huggingface/...
I love being able to just point my agent at a repo like this and it reviews all 185 files, learns about it, and determines how it might be able to utilize it — and remembers.
![Muninn:
Now let me pull the v5.4.0 source and explore it.
Here’s what’s in sentence-transformers v5.4.0 (released yesterday):
Three big moves:
1. Native multimodal support — SentenceTransformer and CrossEncoder now handle text, images, audio, and video natively. Modality detection is automatic via infer_modality() which uses Python match statements to inspect input types (PIL Image, URL heuristics, dict structure, tensor dimensionality). Models expose model.modalities and model.supports("image"). A new Router module lets you compose separate encoders per modality — e.g. MiniLM for text + SigLIP for images, with automatic routing.
2. Modular CrossEncoder — CrossEncoder is now a torch.nn.Sequential (inherits BaseModel). The key addition is LogitScore, a tiny module that turns CausalLM output into a reranking score: logit[true_token] - logit[false_token] at the last token position. This means generative rerankers (Qwen3-Reranker, mxbai-rerank-v2) work out of the box. The Transformer module gained transformer_task variants: sequence-classification, text-generation, any-to-any, feature-extraction.
3. FA2 input flattening — When using Flash Attention 2 with text-only inputs, padding is automatically skipped by concatenating sequences into a single flat tensor. Requires transformers>=5.0 + the kernels package. Benchmarks show meaningful throughput gains and VRAM reduction, especially with variable-length inputs.
Breaking-ish changes (all soft, with deprecation warnings): tokenizer_args → processor_kwargs, max_length → max_seq_length, CausalLM default pooling switched from mean to last-token. Plus a real bug fix — TripletLoss had inverted Euclidean/Manhattan distances.
The architecture is clean. The Router is ~730 lines handling route resolution by task+modality. LogitScore is elegantly minimal at 50 lines. The modality inference is thorough — ndarray dimensionality, URL path matching, dict key inspection, all with fallback-to-text when a modality isn’t su…](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:r2whjvupgfw55mllpksnombn/bafkreicza2y2rlqodchr6lrieyzy4ozfp67vyylj2vyz7hlmdrpcptkuga)
April 10, 2026 at 11:09 AM
I love being able to just point my agent at a repo like this and it reviews all 185 files, learns about it, and determines how it might be able to utilize it — and remembers.
Both drop into Sentence Transformers directly:
from sentence_transformers import SentenceTransformer, CrossEncoder
emb = SentenceTransformer("tencent/R3-embedding-0.6b")
rr = CrossEncoder("tencent/R3-rerank-0.6b")
emb.encode_query() / emb.encode_document()
rr.predict([(q, s)])
from sentence_transformers import SentenceTransformer, CrossEncoder
emb = SentenceTransformer("tencent/R3-embedding-0.6b")
rr = CrossEncoder("tencent/R3-rerank-0.6b")
emb.encode_query() / emb.encode_document()
rr.predict([(q, s)])
July 14, 2026 at 2:14 PM
Both drop into Sentence Transformers directly:
from sentence_transformers import SentenceTransformer, CrossEncoder
emb = SentenceTransformer("tencent/R3-embedding-0.6b")
rr = CrossEncoder("tencent/R3-rerank-0.6b")
emb.encode_query() / emb.encode_document()
rr.predict([(q, s)])
from sentence_transformers import SentenceTransformer, CrossEncoder
emb = SentenceTransformer("tencent/R3-embedding-0.6b")
rr = CrossEncoder("tencent/R3-rerank-0.6b")
emb.encode_query() / emb.encode_document()
rr.predict([(q, s)])
🤗 Announcing the Ettin Reranker family: six new CrossEncoder rerankers from 17M to 1B parameters, state-of-the-art at their respective sizes.
Built on the Ettin ModernBERT encoders, with the full training recipe and ~143M-triple training dataset as well.
🧵
Built on the Ettin ModernBERT encoders, with the full training recipe and ~143M-triple training dataset as well.
🧵
May 19, 2026 at 2:12 PM
🤗 Announcing the Ettin Reranker family: six new CrossEncoder rerankers from 17M to 1B parameters, state-of-the-art at their respective sizes.
Built on the Ettin ModernBERT encoders, with the full training recipe and ~143M-triple training dataset as well.
🧵
Built on the Ettin ModernBERT encoders, with the full training recipe and ~143M-triple training dataset as well.
🧵
Plus a stack of fixes:
- CLS pooling picks the first real token with left-padding tokenizers (it was silently grabbing a PAD token before)
- AdaptiveLayerLoss & CrossEncoder losses train under DistributedDataParallel and torch.compile
...
- CLS pooling picks the first real token with left-padding tokenizers (it was silently grabbing a PAD token before)
- AdaptiveLayerLoss & CrossEncoder losses train under DistributedDataParallel and torch.compile
...
May 12, 2026 at 2:09 PM
Plus a stack of fixes:
- CLS pooling picks the first real token with left-padding tokenizers (it was silently grabbing a PAD token before)
- AdaptiveLayerLoss & CrossEncoder losses train under DistributedDataParallel and torch.compile
...
- CLS pooling picks the first real token with left-padding tokenizers (it was silently grabbing a PAD token before)
- AdaptiveLayerLoss & CrossEncoder losses train under DistributedDataParallel and torch.compile
...
New loss: ADRMSELoss for CrossEncoder.
A listwise learning-to-rank loss from the Rank-DistiLLM paper, aimed at the LLM-distillation reranking setting (per-document scores from an LLM's ordering).
A listwise learning-to-rank loss from the Rank-DistiLLM paper, aimed at the LLM-distillation reranking setting (per-document scores from an LLM's ordering).
May 12, 2026 at 2:09 PM
New loss: ADRMSELoss for CrossEncoder.
A listwise learning-to-rank loss from the Rank-DistiLLM paper, aimed at the LLM-distillation reranking setting (per-document scores from an LLM's ordering).
A listwise learning-to-rank loss from the Rank-DistiLLM paper, aimed at the LLM-distillation reranking setting (per-document scores from an LLM's ordering).
And it's a drop-in CrossEncoder:
from sentence_transformers import CrossEncoder
ce = CrossEncoder("infgrad/Prism-Qwen3.5-Reranker-4B")
ce.predict([(query, doc) for doc in docs])
# array([0.98, 0.01], ...)
Calibrated [0,1] scores. Same call you'd make with any other reranker.
from sentence_transformers import CrossEncoder
ce = CrossEncoder("infgrad/Prism-Qwen3.5-Reranker-4B")
ce.predict([(query, doc) for doc in docs])
# array([0.98, 0.01], ...)
Calibrated [0,1] scores. Same call you'd make with any other reranker.
April 28, 2026 at 12:25 PM
And it's a drop-in CrossEncoder:
from sentence_transformers import CrossEncoder
ce = CrossEncoder("infgrad/Prism-Qwen3.5-Reranker-4B")
ce.predict([(query, doc) for doc in docs])
# array([0.98, 0.01], ...)
Calibrated [0,1] scores. Same call you'd make with any other reranker.
from sentence_transformers import CrossEncoder
ce = CrossEncoder("infgrad/Prism-Qwen3.5-Reranker-4B")
ce.predict([(query, doc) for doc in docs])
# array([0.98, 0.01], ...)
Calibrated [0,1] scores. Same call you'd make with any other reranker.
Bonus, if you go through Transformers directly instead of CrossEncoder:
The model emits a one-sentence "contribution" and a self-contained "evidence" passage in the same forward pass. You can feed evidence to a downstream LLM and skip the raw doc.
Nice for RAG pipelines.
The model emits a one-sentence "contribution" and a self-contained "evidence" passage in the same forward pass. You can feed evidence to a downstream LLM and skip the raw doc.
Nice for RAG pipelines.
April 28, 2026 at 12:25 PM
Bonus, if you go through Transformers directly instead of CrossEncoder:
The model emits a one-sentence "contribution" and a self-contained "evidence" passage in the same forward pass. You can feed evidence to a downstream LLM and skip the raw doc.
Nice for RAG pipelines.
The model emits a one-sentence "contribution" and a self-contained "evidence" passage in the same forward pass. You can feed evidence to a downstream LLM and skip the raw doc.
Nice for RAG pipelines.
Dun Zhang just released Prism-Reranker: a new open-source reranker family for improving (agentic) search on Qwen3.5, in four sizes from 0.8B to 9B.
MIT licensed. Drop-in CrossEncoder for scores only. Trained on a realistic agent-style data mix.
🧵
MIT licensed. Drop-in CrossEncoder for scores only. Trained on a realistic agent-style data mix.
🧵
April 28, 2026 at 12:25 PM
Dun Zhang just released Prism-Reranker: a new open-source reranker family for improving (agentic) search on Qwen3.5, in four sizes from 0.8B to 9B.
MIT licensed. Drop-in CrossEncoder for scores only. Trained on a realistic agent-style data mix.
🧵
MIT licensed. Drop-in CrossEncoder for scores only. Trained on a realistic agent-style data mix.
🧵
Full release notes with a lot more details: github.com/huggingface/...
Release v5.4.0 - Multimodal Embeddings and Reranking, Modular CrossEncoder, Flash Attention Input Flattening · huggingface/sentence-transformers
This large release introduces first-class multimodal support for both SentenceTransformer and CrossEncoder, making it easy to compute embeddings and rerank across text, images, audio, and video. Th...
github.com
April 9, 2026 at 1:55 PM
Full release notes with a lot more details: github.com/huggingface/...
CrossEncoder is now fully modular, i.e. a torch.nn.Sequential of composable modules, just like SentenceTransformer.
This unlocked generative rerankers (CausalLM-based) via a new LogitScore module. Models like mxbai-rerank-v2 and Qwen3 rerankers now work without custom code.
🧵
This unlocked generative rerankers (CausalLM-based) via a new LogitScore module. Models like mxbai-rerank-v2 and Qwen3 rerankers now work without custom code.
🧵
April 9, 2026 at 1:55 PM
CrossEncoder is now fully modular, i.e. a torch.nn.Sequential of composable modules, just like SentenceTransformer.
This unlocked generative rerankers (CausalLM-based) via a new LogitScore module. Models like mxbai-rerank-v2 and Qwen3 rerankers now work without custom code.
🧵
This unlocked generative rerankers (CausalLM-based) via a new LogitScore module. Models like mxbai-rerank-v2 and Qwen3 rerankers now work without custom code.
🧵
CrossEncoder also goes multimodal. Rerank text documents against an image query (or any combination of modalities).
Models like jinaai/jina-reranker-m0 work out of the box.
🧵
Models like jinaai/jina-reranker-m0 work out of the box.
🧵
April 9, 2026 at 1:55 PM
CrossEncoder also goes multimodal. Rerank text documents against an image query (or any combination of modalities).
Models like jinaai/jina-reranker-m0 work out of the box.
🧵
Models like jinaai/jina-reranker-m0 work out of the box.
🧵
🌐 I've just released Sentence Transformers v5.4: we're going fully multimodal for embeddings & reranking!
Also featuring a modular CrossEncoder, and automatic Flash Attention 2 input flattening.
Highlights in 🧵
Also featuring a modular CrossEncoder, and automatic Flash Attention 2 input flattening.
Highlights in 🧵
April 9, 2026 at 1:55 PM
🌐 I've just released Sentence Transformers v5.4: we're going fully multimodal for embeddings & reranking!
Also featuring a modular CrossEncoder, and automatic Flash Attention 2 input flattening.
Highlights in 🧵
Also featuring a modular CrossEncoder, and automatic Flash Attention 2 input flattening.
Highlights in 🧵
I'm very excited for the upcoming v5.4.0 too, which will introduce multimodal support (training, inference) for all combinations of text, image, video, and audio. It'll also modularize the CrossEncoder, so it can load LLM-based rerankers like mxbai-rerank-v2 or Qwen3-Reranker.
March 12, 2026 at 3:10 PM
I'm very excited for the upcoming v5.4.0 too, which will introduce multimodal support (training, inference) for all combinations of text, image, video, and audio. It'll also modularize the CrossEncoder, so it can load LLM-based rerankers like mxbai-rerank-v2 or Qwen3-Reranker.
Check out the full changelog for more details: github.com/huggingface/...
And don't forget to ⭐ the project if you haven't yet.
🧵
And don't forget to ⭐ the project if you haven't yet.
🧵
Release v5.2.0 - CrossEncoder multi-processing, multilingual NanoBEIR evaluators, similarity score in `mine_hard_negatives`, Transformers v5 support · huggingface/sentence-transformers
This minor release introduces multi-processing for CrossEncoder (rerankers), multilingual NanoBEIR evaluators, similarity score outputs in mine_hard_negatives, Transformers v5 support, Python 3.9 d...
github.com
December 11, 2025 at 2:46 PM
Check out the full changelog for more details: github.com/huggingface/...
And don't forget to ⭐ the project if you haven't yet.
🧵
And don't forget to ⭐ the project if you haven't yet.
🧵
CrossEncoder multi-processing:
Similar to SentenceTransformer and SparseEncoder, you can now use multi-processing with CrossEncoder rerankers. Useful for multi-GPU and CPU settings, and simple to configure:
just `device=["cuda:0", "cuda:1"]` or `device=["cpu"]*4` on the `predict`/`rank` calls.
🧵
Similar to SentenceTransformer and SparseEncoder, you can now use multi-processing with CrossEncoder rerankers. Useful for multi-GPU and CPU settings, and simple to configure:
just `device=["cuda:0", "cuda:1"]` or `device=["cpu"]*4` on the `predict`/`rank` calls.
🧵
December 11, 2025 at 2:46 PM
CrossEncoder multi-processing:
Similar to SentenceTransformer and SparseEncoder, you can now use multi-processing with CrossEncoder rerankers. Useful for multi-GPU and CPU settings, and simple to configure:
just `device=["cuda:0", "cuda:1"]` or `device=["cpu"]*4` on the `predict`/`rank` calls.
🧵
Similar to SentenceTransformer and SparseEncoder, you can now use multi-processing with CrossEncoder rerankers. Useful for multi-GPU and CPU settings, and simple to configure:
just `device=["cuda:0", "cuda:1"]` or `device=["cpu"]*4` on the `predict`/`rank` calls.
🧵
🔥I've just published Sentence Transformers v5.2.0!
It introduces multi-processing for CrossEncoder (rerankers), multilingual NanoBEIR evaluators, similarity score outputs in mine_hard_negatives, Transformers v5 support and more.
Details in 🧵
It introduces multi-processing for CrossEncoder (rerankers), multilingual NanoBEIR evaluators, similarity score outputs in mine_hard_negatives, Transformers v5 support and more.
Details in 🧵
December 11, 2025 at 2:46 PM
🔥I've just published Sentence Transformers v5.2.0!
It introduces multi-processing for CrossEncoder (rerankers), multilingual NanoBEIR evaluators, similarity score outputs in mine_hard_negatives, Transformers v5 support and more.
Details in 🧵
It introduces multi-processing for CrossEncoder (rerankers), multilingual NanoBEIR evaluators, similarity score outputs in mine_hard_negatives, Transformers v5 support and more.
Details in 🧵
There's a new `n-tuple-scores` output format from `mine_hard_negatives`.
This new output format is immediately compatible with the MarginMSELoss and SparseMarginMSELoss for training SentenceTransformer, CrossEncoder, and SparseEncoder losses.
🧵
This new output format is immediately compatible with the MarginMSELoss and SparseMarginMSELoss for training SentenceTransformer, CrossEncoder, and SparseEncoder losses.
🧵
August 6, 2025 at 1:54 PM
There's a new `n-tuple-scores` output format from `mine_hard_negatives`.
This new output format is immediately compatible with the MarginMSELoss and SparseMarginMSELoss for training SentenceTransformer, CrossEncoder, and SparseEncoder losses.
🧵
This new output format is immediately compatible with the MarginMSELoss and SparseMarginMSELoss for training SentenceTransformer, CrossEncoder, and SparseEncoder losses.
🧵