Track these benchmarks to evaluate Agentic RAG retrieval quality: NeedleInAHaystack, BeIR, FRAMES, RAGTruth, RULER, MMNeedle, FEVER.

A new study shows the first token of a hallucinated span offers a stronger detection signal than later tokens, using RAGTruth token‑level data. This pattern holds for model sizes. https://getnews.me/first-hallucinated-token-shows-stronger-signal-for-llm-error-detection/ #llmhallucination #ragtruth

Study shows the first token of a hallucinated span has a stronger logit deviation, yielding higher detection rates than later tokens, based on the RAGTruth extended dataset. Read more: https://getnews.me/first-hallucination-tokens-show-stronger-signals-than-later-tokens/ #hallucination #token

HalluGuard, a 4 billion‑parameter model, achieves 84% balanced accuracy on the RAGTruth benchmark, matching a 7 billion‑parameter system. It will be released under Apache 2.0. Read more: https://getnews.me/haluguard-model-reduces-hallucinations-in-retrieval-augmented-generation/ #halluguard #ai

Turk-LettuceDetect's Turkish ModernBERT scored 0.7266 F1 on token-level hallucination detection, handles up to 8,192 tokens, and comes with a 17,790‑instance RAGTruth dataset. https://getnews.me/turk-lettucedetect-ai-model-enhances-turkish-hallucination-detection/ #turkishnlp #ai

RAG Benchmarks Leaderboard: Retrieval Rankings 2026

https://awesomeagents.ai/leaderboards/rag-benchmarks-leaderboard/

#Rag #Embeddings #Benchmarks

RAG Benchmarks Leaderboard: Retrieval Rankings 2026

https://awesomeagents.ai/leaderboards/rag-benchmarks-leaderboard/

#Rag #Embeddings #Benchmarks

Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, Tong Zhang
RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
https://arxiv.org/abs/2401.00396

Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Cheng Niu, Randy Zhong, Juntong Song, Tong Zhang
RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models. (arXiv:2401.00396v1 [cs.CL])
http://arxiv.org/abs/2401.00396

introduce a perturbed multi-hop QA dataset with induced hallucinations. Via supervised fine-tuning on our dataset, we achieve better recall with a 7B model than GPT-4o on the RAGTruth hallucination detection benchmark and offer competitive performance [4/5 of https://arxiv.org/abs/2505.04844v1]

HDMBench. Experimental results demonstrate that HDM-2 out-performs existing approaches across RagTruth, TruthfulQA, and HDMBench datasets. This work addresses the specific challenges of enterprise deployment, including computational efficiency, domain [4/5 of https://arxiv.org/abs/2504.07069v1]

Evaluations on the RAGTruth corpus demonstrate an F1 score of 79.22% for example-level detection, which is a 14.8% improvement over Luna, the previous state-of-the-art encoder-based architecture. Additionally, the system can [5/6 of https://arxiv.org/abs/2502.17125v1]

Building on ModernBERT's extended context capabilities (up to 8k tokens) and trained on the RAGTruth benchmark dataset, our approach outperforms all previous encoder-based models and most prompt-based models, while being [3/6 of https://arxiv.org/abs/2502.17125v1]

今日のZennトレンド

RAGのウソを検知する新手法（LLM-as-a-Judgeを超えて）
この記事は、RAGのハルシネーションを高速に検出する手法「LettuceDetect」を紹介しています。
これは、軽量な言語モデルModernBERTをRAGTruthで学習させたもので、従来のLLMを用いた手法と同等の性能ながら高速かつ低コストで幻覚検知が可能です。
RAGシステムにおけるハルシネーション対策として有効な選択肢となりえます。