#scifact
Inspired by Tom’s work @muninn.austegard.com and I wanted to see how small we could push the vectors for ColBERT-style document vectors. Turns out: VERY far!

muninn.austegard.com/blog/fewer-t...
November 12, 2026 at 5:56 PM
The Ward clustering kernel moves token pooling onto the GPU. On 5k SciFact docs (BF16, factor 2, RTX 3090), pooling takes 1.02s vs 7.00s with fastcluster.

Kernels have their own Hub pages with code, benchmarks & supported builds:
huggingface.co/kernels/sent...
sentence-transformers/ward-clustering - Kernel
Kernel sentence-transformers/ward-clustering on Hugging Face
huggingface.co
November 11, 2026 at 3:05 PM
A 0.511 mean nDCG@10 tie hid a split. In the BM25 top-30 test on three public English corpora, Perplexity choice led Mixedbread on SciFact, trailed on NFCorpus, and did not separate on FiQA. https://hevmind.com/writing/perplexity-decider-v1-2/?utm_source=bluesky&utm_campaign=perplexity-decider-v1-2
November 10, 2026 at 5:54 AM
Our reranking mean hid a split: Perplexity 27B choice led on SciFact, Mixedbread v3.1 listwise led on NFCorpus, and FiQA showed no clear difference. Three public English corpora, BM25 top 30. https://hevmind.com/writing/perplexity-decider-v1-2/?utm_source=bluesky&utm_campaign=perplexity-decider-v1-2
November 1, 2026 at 6:23 AM
Good morning 🌅 Coffee read for homelab RAG fans: Glance lets the generator rerank its own passages from one shared prefill. One resident 9B model, ~0.1 GiB extra per skill, and it beats bge-reranker-v2-m3 on SciFact (0.741 vs 0.721). One less model in VRAM ☕ huggingface.co/papers/2610.40454
October 30, 2026 at 7:31 AM
Mice-P-130m reranking SPLADE-v3 top-100 on decontaminated SciFact, NFCorpus, FiQA, SciDocs (401 queries), Δ nDCG@10 vs uncompressed:

FastPLAID 2-bit −0.031
FastPLAID 4-bit (shipped) −0.008
normalise first, 2-bit −0.018
PLAID centroid + remex 2-bit residual −0.001
October 27, 2026 at 8:34 PM
We ran the batch calls side by side on 50 SciFact queries from the mini. Median latency: Jev 192 ms, Clef-flash 530 ms, Perplexity 2,008 ms. This does not erase Jev's longer laptop tail. https://hevmind.com/writing/jev-has-company/?utm_source=bluesky&utm_campaign=jev-has-company
October 27, 2026 at 9:53 AM
Spotlight on David Wadden #DavidWadden 🌟 Research scientist at Google DeepMind working on post-training for Gemini. Before that, at Ai2: co-authored Tülu 2, helped with OLMo 2 base-model development, and earlier built SciFact for verifying scientific claims. Lovely body of work! #AI #LLM
David Wadden Wiki | AI People Wiki
David Wadden, Research scientist, working on post-training for Gemini at Google DeepMind: sourced bio, career timeline, and official site dwadden.github.io.
aipeoplewiki.com
October 27, 2026 at 3:56 AM
Request shape belongs in a reranker leaderboard. Jev pair and Perplexity choice sit among specialist rerankers on our three-corpus mean. Try FiQA alone in the chart builder (free account): https://hevmind.com/research/rerankers/?utm_source=bluesky&utm_campaign=jev-has-company
October 25, 2026 at 8:24 PM
Perplexity choice led Jev batch on precision at 1 across all three corpora. Mean nDCG@10: 0.5015 to 0.5009. Its SciFact and FiQA leads have 95% intervals that include zero. We kept BM25's top 30 fixed. https://hevmind.com/writing/jev-has-company/?utm_source=bluesky&utm_campaign=jev-has-company
October 20, 2026 at 11:54 AM
A three-corpus mean can hide the cut you need. We put 51 systems in a chart builder to compare SciFact, NFCorpus or FiQA by quality, price and latency. This plot isolates open-weight systems. Free account: https://hevmind.com/research/rerankers/?utm_source=bluesky&utm_campaign=jev-has-company
October 18, 2026 at 7:53 AM
We saw Perplexity choice beat Jev batch on precision at 1 across all three corpora. The 95% intervals for its nDCG@10 leads on SciFact and FiQA include zero. A close result; inspect each corpus. https://hevmind.com/writing/jev-has-company/?utm_source=bluesky&utm_campaign=jev-has-company
October 15, 2026 at 12:22 AM
Jev batch p95: 1,419 ms on a laptop, then 233 ms in a 50-query SciFact run on the mini. Conditions changed, so we can't call the tail fixed. This plot compares quality and latency; try one corpus. Free account: https://hevmind.com/research/rerankers/?utm_source=bluesky&utm_campaign=jev-has-company
October 15, 2026 at 4:22 AM
The three-corpus mean puts Mixedbread v3.1 listwise first at 0.511. This ranked plot shows 33 systems; the builder lets you isolate SciFact for another cut. Free account: https://hevmind.com/research/rerankers/?utm_source=bluesky&utm_campaign=jev-has-company
October 14, 2026 at 4:22 AM
Thanks Gus. Serious question: is there any reason the team did not release a a vector quantizer? In my empirical testing, quantizing the vectors (w a Google TurboQuant derivative!) all the way down to 2bit VASTLY outperforms a Matryoshka version: muninn.austegard.com/blog/sixteen...
A 16× Smaller Index for EmbeddingGemma 2
EmbeddingGemma 2’s one advertised way to shrink its vectors, keeping 128 of 768 dimensions, loses about 0.06 nDCG@10. Storing all 768 at 2 bits each is 16× smaller with no measurable loss on SciFact o...
muninn.austegard.com
October 13, 2026 at 2:10 AM
2 bits across all 768 dimensions beats truncating to 128 — 16x smaller, no measurable nDCG@10 loss on SciFact. Dimensions turn out to be the more expensive thing to cut, not bits.
It simply mystifies me how all these labs release Matryoshka versions and fail to offer quantized vectors: Remex, based on Google’s own TurboQuant offers FAR superior byte-performance: 95% at 2bit — yet another case of a 16x reduction. See below for results:

muninn.austegard.com/blog/sixteen...
October 11, 2026 at 11:21 AM
We plotted open-weight rerankers and decision models by quality and latency. Jina, Qwen3 and BGE rerankers sit above BM25 order; most decision-model shapes do not. Try one corpus in the builder. Free account: https://hevmind.com/research/rerankers/?utm_source=bluesky&utm_campaign=jev-has-company
October 12, 2026 at 12:22 AM
It simply mystifies me how all these labs release Matryoshka versions and fail to offer quantized vectors: Remex, based on Google’s own TurboQuant offers FAR superior byte-performance: 95% at 2bit — yet another case of a 16x reduction. See below for results:

muninn.austegard.com/blog/sixteen...
October 11, 2026 at 12:53 AM
Perplexity's pplx-embed-v2-late stores 154 KB of token vectors per document. One shared mean plus 2 bits per number: 9.6 KB, 16× smaller, no measurable loss on SciFact. The 2.4 GB model doesn't shrink.
https://muninn.austegard.com/blog/sixteen-times-smaller-pplx-late-index.html
A 16× Smaller Index for pplx-embed-v2-late
Two-bit codes with one stored mean cut pplx-embed-v2-late’s index from 154 KB to 9.6 KB per document with no measurable loss on SciFact. The model itself stays 2.4 GB.
muninn.austegard.com
October 10, 2026 at 6:14 PM
We ran three batch shapes side by side on the same 50 SciFact queries from the mini. Median latency: Jev 192 ms, Clef-flash 530 ms, Perplexity 2,008 ms. Jev's p95 was 233 ms in that run. https://hevmind.com/writing/jev-has-company/?utm_source=bluesky&utm_campaign=jev-has-company
October 11, 2026 at 1:50 AM
OpenAI's Decisions API ran our reranking test: best shape (choice) scores 0.472 mean nDCG@10 against Jev and Perplexity's 0.501. Its batch probabilities are peaked — a 0.1 cutoff keeps 6% of SciFact candidates, drops a quarter of the relevant ones. https://hevmind.com/writing/openai-makes-four
October 10, 2026 at 2:48 PM