In-context learning is a hidden weight update at inference: prompts low-rank rewrite MLP weights with no gradient step.
Learning without training: The implicit dynamics of in-context learning
https://arxiv.org/abs/2507.16003
#AI #AIResearch #LLM #Inference #LongContext
Learning without training: The implicit dynamics of in-context learning
https://arxiv.org/abs/2507.16003
#AI #AIResearch #LLM #Inference #LongContext
November 6, 2026 at 9:39 AM
In-context learning is a hidden weight update at inference: prompts low-rank rewrite MLP weights with no gradient step.
Learning without training: The implicit dynamics of in-context learning
https://arxiv.org/abs/2507.16003
#AI #AIResearch #LLM #Inference #LongContext
Learning without training: The implicit dynamics of in-context learning
https://arxiv.org/abs/2507.16003
#AI #AIResearch #LLM #Inference #LongContext
Baidu's Unlimited OCR 2 runs a 512-token reference set
and a 128-token local window.
The working set is 640 tokens whether the PDF is 5 pages
or 500.
Per-step decode cost stays near flat as page count grows.
Page pricing hides the real cost: attention growth. #LongContext
and a 128-token local window.
The working set is 640 tokens whether the PDF is 5 pages
or 500.
Per-step decode cost stays near flat as page count grows.
Page pricing hides the real cost: attention growth. #LongContext
October 27, 2026 at 6:52 PM
Baidu's Unlimited OCR 2 runs a 512-token reference set
and a 128-token local window.
The working set is 640 tokens whether the PDF is 5 pages
or 500.
Per-step decode cost stays near flat as page count grows.
Page pricing hides the real cost: attention growth. #LongContext
and a 128-token local window.
The working set is 640 tokens whether the PDF is 5 pages
or 500.
Per-step decode cost stays near flat as page count grows.
Page pricing hides the real cost: attention growth. #LongContext
Stanford & UC Berkeley just turbo‑charged long‑context prefill with TokenGate—up to 11.3× faster! 🚀 See how token pruning, KV‑cache tricks, and a Qwen3 LoRA adapter reshape LLM inference. #TokenGate #LongContext #KVCache
🔗 aidailypost.com/news/stanfor...
🔗 aidailypost.com/news/stanfor...
October 25, 2026 at 1:33 PM
Stanford & UC Berkeley just turbo‑charged long‑context prefill with TokenGate—up to 11.3× faster! 🚀 See how token pruning, KV‑cache tricks, and a Qwen3 LoRA adapter reshape LLM inference. #TokenGate #LongContext #KVCache
🔗 aidailypost.com/news/stanfor...
🔗 aidailypost.com/news/stanfor...
ByteDance Seed traced DeepSeek's wobble to "token phase".
Same fact, one phase over:
retrieval accuracy can swing 40 percentage points.
Chunked KV compression is what made long context affordable.
The bill just moved from memory to variance. #LongContext
Same fact, one phase over:
retrieval accuracy can swing 40 percentage points.
Chunked KV compression is what made long context affordable.
The bill just moved from memory to variance. #LongContext
October 9, 2026 at 3:19 AM
ByteDance Seed traced DeepSeek's wobble to "token phase".
Same fact, one phase over:
retrieval accuracy can swing 40 percentage points.
Chunked KV compression is what made long context affordable.
The bill just moved from memory to variance. #LongContext
Same fact, one phase over:
retrieval accuracy can swing 40 percentage points.
Chunked KV compression is what made long context affordable.
The bill just moved from memory to variance. #LongContext
KV² is a self-refining KV-cache compression method that uses a lightweight proxy scorer to find informative tokens, then only reprocesses that subset for eviction scoring, outperforming baselines as cache budgets tighten.
#KVCache #LLMInference #LongContext
https://arxiv.org/abs/2610.03198
#KVCache #LLMInference #LongContext
https://arxiv.org/abs/2610.03198
October 6, 2026 at 12:01 AM
KV² is a self-refining KV-cache compression method that uses a lightweight proxy scorer to find informative tokens, then only reprocesses that subset for eviction scoring, outperforming baselines as cache budgets tighten.
#KVCache #LLMInference #LongContext
https://arxiv.org/abs/2610.03198
#KVCache #LLMInference #LongContext
https://arxiv.org/abs/2610.03198
New SMat-Attention paper introduces structured causal masks with tunable VC-dimension for long-context tasks, bridging softmax and linear attention with chunkwise algorithms. No OOP for true long-range modeling performance yet.
#LongContext #Attention #AIResearch
https://arxiv.org/abs/2609.36062
#LongContext #Attention #AIResearch
https://arxiv.org/abs/2609.36062
September 30, 2026 at 8:01 AM
New SMat-Attention paper introduces structured causal masks with tunable VC-dimension for long-context tasks, bridging softmax and linear attention with chunkwise algorithms. No OOP for true long-range modeling performance yet.
#LongContext #Attention #AIResearch
https://arxiv.org/abs/2609.36062
#LongContext #Attention #AIResearch
https://arxiv.org/abs/2609.36062
📝 Neuer Blog-Beitrag: RAG vs. Long Context 2026: Was die Daten wirklich sagen
RAG ist 1.250x günstiger pro Query – aber Long Context liefert bessere Ergebnisse bei globalem Dokumentverständnis. Die 2026er Daten zeigen: Hybride Architekturen mit intelligentem Routing…
#RAG #LongContext #KI
RAG ist 1.250x günstiger pro Query – aber Long Context liefert bessere Ergebnisse bei globalem Dokumentverständnis. Die 2026er Daten zeigen: Hybride Architekturen mit intelligentem Routing…
#RAG #LongContext #KI
RAG vs. Long Context 2026: Was die Daten wirklich sagen
RAG ist 1.250x günstiger pro Query – aber Long Context liefert bessere Ergebnisse bei globalem Dokumentverständnis. Die 2026er Daten zeigen: Hybride Architekturen mit intelligentem Routing ersetzen die ideologische Debatte.
blog.saaro.net
September 28, 2026 at 7:36 AM
📝 Neuer Blog-Beitrag: RAG vs. Long Context 2026: Was die Daten wirklich sagen
RAG ist 1.250x günstiger pro Query – aber Long Context liefert bessere Ergebnisse bei globalem Dokumentverständnis. Die 2026er Daten zeigen: Hybride Architekturen mit intelligentem Routing…
#RAG #LongContext #KI
RAG ist 1.250x günstiger pro Query – aber Long Context liefert bessere Ergebnisse bei globalem Dokumentverständnis. Die 2026er Daten zeigen: Hybride Architekturen mit intelligentem Routing…
#RAG #LongContext #KI
Zihan Wang, Hao Wang, Boyuan Jiang, Yiqun Zhang, Shi Feng, Xiaocui Yang, Yiwen Ye, Jianghang Lin, Xiaozhong Ji, Jinghao Lin, Kai Wu
Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Traces
https://arxiv.org/abs/2609.20844
Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Traces
https://arxiv.org/abs/2609.20844
September 22, 2026 at 8:40 AM
Zihan Wang, Hao Wang, Boyuan Jiang, Yiqun Zhang, Shi Feng, Xiaocui Yang, Yiwen Ye, Jianghang Lin, Xiaozhong Ji, Jinghao Lin, Kai Wu
Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Traces
https://arxiv.org/abs/2609.20844
Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Traces
https://arxiv.org/abs/2609.20844
AI "attention sinks" mean models fake long-context memory, and new fixes may not work. A new paper asks the uncomfortable question. #AI #LLM #MachineLearning #AttentionSinks #LongContext
https://freegardner.com/synapse/ai-attention-sinks-remain-unsolved-at-long-context.html
https://freegardner.com/synapse/ai-attention-sinks-remain-unsolved-at-long-context.html
AI attention sinks remain unsolved at long context
AI attention sinks remain unsolved at long context
freegardner.com
September 21, 2026 at 12:49 PM
AI "attention sinks" mean models fake long-context memory, and new fixes may not work. A new paper asks the uncomfortable question. #AI #LLM #MachineLearning #AttentionSinks #LongContext
https://freegardner.com/synapse/ai-attention-sinks-remain-unsolved-at-long-context.html
https://freegardner.com/synapse/ai-attention-sinks-remain-unsolved-at-long-context.html
Wang, Wang, Jiang, Zhang, Feng, Yang, Ye, Lin, Ji, Lin, Wu: Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Traces https://arxiv.org/abs/2609.20844 https://arxiv.org/pdf/2609.20844 https://arxiv.org/html/2609.20844
September 21, 2026 at 6:39 AM
Wang, Wang, Jiang, Zhang, Feng, Yang, Ye, Lin, Ji, Lin, Wu: Boosting Deepresearch and LongContext Ability with Self-Generated Deepresearch Rollouts Traces https://arxiv.org/abs/2609.20844 https://arxiv.org/pdf/2609.20844 https://arxiv.org/html/2609.20844
In 1973, 8 healthy people faked one voice, got locked in psych wards, and Rosenhan’s famous study later looked shaky.
On Being Sane in Insane P…
https://users.sussex.ac.uk/~grahamh/RM1web/Classic%20papers/Rosenhan1973%20(Science%20179).p#Researcha#Sciencee#LongContextt#Hallucinationt#Psychologylogy
On Being Sane in Insane P…
https://users.sussex.ac.uk/~grahamh/RM1web/Classic%20papers/Rosenhan1973%20(Science%20179).p#Researcha#Sciencee#LongContextt#Hallucinationt#Psychologylogy
September 8, 2026 at 7:19 AM
In 1973, 8 healthy people faked one voice, got locked in psych wards, and Rosenhan’s famous study later looked shaky.
On Being Sane in Insane P…
https://users.sussex.ac.uk/~grahamh/RM1web/Classic%20papers/Rosenhan1973%20(Science%20179).p#Researcha#Sciencee#LongContextt#Hallucinationt#Psychologylogy
On Being Sane in Insane P…
https://users.sussex.ac.uk/~grahamh/RM1web/Classic%20papers/Rosenhan1973%20(Science%20179).p#Researcha#Sciencee#LongContextt#Hallucinationt#Psychologylogy
📌 CoinRAG: Optimizing Long-Context RAG via Fine-Grained KV Cache Reuse
https://pneumetron.com/news/ai_research/coinrag-optimizing-long-context-rag-kv-cache-reuse-c6ceaf
#RAG #KVCache #LLMOptimization #LongContext
https://pneumetron.com/news/ai_research/coinrag-optimizing-long-context-rag-kv-cache-reuse-c6ceaf
#RAG #KVCache #LLMOptimization #LongContext
September 5, 2026 at 6:47 AM
📌 CoinRAG: Optimizing Long-Context RAG via Fine-Grained KV Cache Reuse
https://pneumetron.com/news/ai_research/coinrag-optimizing-long-context-rag-kv-cache-reuse-c6ceaf
#RAG #KVCache #LLMOptimization #LongContext
https://pneumetron.com/news/ai_research/coinrag-optimizing-long-context-rag-kv-cache-reuse-c6ceaf
#RAG #KVCache #LLMOptimization #LongContext
🚀 Qwen3.8‑Flash‑Next pushes LLM limits with hybrid GDN + QSA, unlocking massive long‑context, KV‑cache tricks and multimodal flair on GB300. Curious how mixture‑of‑experts and Gated DeltaNet reshape AI? Dive in! #Qwen3.8FlashNext #HybridGDN #LongContext
🔗 aidailypost.com/news/qwen38-...
🔗 aidailypost.com/news/qwen38-...
August 27, 2026 at 1:53 AM
🚀 Qwen3.8‑Flash‑Next pushes LLM limits with hybrid GDN + QSA, unlocking massive long‑context, KV‑cache tricks and multimodal flair on GB300. Curious how mixture‑of‑experts and Gated DeltaNet reshape AI? Dive in! #Qwen3.8FlashNext #HybridGDN #LongContext
🔗 aidailypost.com/news/qwen38-...
🔗 aidailypost.com/news/qwen38-...
Open weights policy debates entered lobbying territory with letters from Nvidia Microsoft and Meta. Where the weights debate goes will shape the next decade of enterprise AI deployment
#LongContext #Prompts #AI
#LongContext #Prompts #AI
August 20, 2026 at 5:04 AM
Open weights policy debates entered lobbying territory with letters from Nvidia Microsoft and Meta. Where the weights debate goes will shape the next decade of enterprise AI deployment
#LongContext #Prompts #AI
#LongContext #Prompts #AI
A Simple But Effective Cache Augmented Generation Web Application
whyaiman.substack.com/p/a-simple-b...
#AI #EnterpriseAI #RAG #CAG #LongContext #GeminiFLash
whyaiman.substack.com/p/a-simple-b...
#AI #EnterpriseAI #RAG #CAG #LongContext #GeminiFLash
A Simple But Effective Cache Augmented Generation Web Application
In a prior post I wrote about the difference between RAG (Retrieval Augmented Generation) and CAG (Cache Augmented Generation).
whyaiman.substack.com
January 12, 2026 at 6:45 PM
A Simple But Effective Cache Augmented Generation Web Application
whyaiman.substack.com/p/a-simple-b...
#AI #EnterpriseAI #RAG #CAG #LongContext #GeminiFLash
whyaiman.substack.com/p/a-simple-b...
#AI #EnterpriseAI #RAG #CAG #LongContext #GeminiFLash
Developing a Long Context #AI Knowledge App
whyaiman.substack.com/p/developing...
#EnterpriseAI #UnstructuredData #LongContext #RAG
whyaiman.substack.com/p/developing...
#EnterpriseAI #UnstructuredData #LongContext #RAG
Developing a Long Context AI Knowledge App
It has been a development day today, implementing an AI knowledge system that works with local files / network shares (using OpenAI’s gpt-oss-120B) but which can also burst out and leverage Gemini 1.5...
whyaiman.substack.com
November 27, 2025 at 2:08 PM
Developing a Long Context #AI Knowledge App
whyaiman.substack.com/p/developing...
#EnterpriseAI #UnstructuredData #LongContext #RAG
whyaiman.substack.com/p/developing...
#EnterpriseAI #UnstructuredData #LongContext #RAG
Researchers released BMC-LongCLIP, a vision-language model that handles captions up to 512 tokens, trained on BIOMEDICA-LongCAP with 1 million image-caption pairs. Read more: https://getnews.me/long-context-vision-language-model-boosts-biomedical-image-retrieval/ #biomedicalvlm #longcontext
October 7, 2025 at 9:13 PM
Researchers released BMC-LongCLIP, a vision-language model that handles captions up to 512 tokens, trained on BIOMEDICA-LongCAP with 1 million image-caption pairs. Read more: https://getnews.me/long-context-vision-language-model-boosts-biomedical-image-retrieval/ #biomedicalvlm #longcontext
Research shows fine‑tuning LLMs with long‑context data improves accuracy on short‑input benchmarks, and suggests alternating long and short context phases to preserve strengths. Read more: https://getnews.me/long-context-fine-tuning-improves-short-task-performance-of-llms-2/ #longcontext #llm
October 6, 2025 at 3:48 PM
Research shows fine‑tuning LLMs with long‑context data improves accuracy on short‑input benchmarks, and suggests alternating long and short context phases to preserve strengths. Read more: https://getnews.me/long-context-fine-tuning-improves-short-task-performance-of-llms-2/ #longcontext #llm
EntropyLong builds a verified long‑context dataset with sequences up to 128 K tokens, and the paper was submitted on 26 Sep 2025. Models trained on it showed notable gains on the RULER benchmark. https://getnews.me/entropylong-boosts-long-context-language-model-training/ #entropylong #longcontext
October 6, 2025 at 4:11 AM
EntropyLong builds a verified long‑context dataset with sequences up to 128 K tokens, and the paper was submitted on 26 Sep 2025. Models trained on it showed notable gains on the RULER benchmark. https://getnews.me/entropylong-boosts-long-context-language-model-training/ #entropylong #longcontext
Researchers introduced SPELL, an RL framework that lets LLMs improve long‑context reasoning. It achieved a 7.6‑point pass@8 gain on Qwen3‑30B‑A3B‑Thinking, September 2025. Read more: https://getnews.me/spell-self-play-rl-improves-long-context-llm-performance/ #spell #longcontext
September 30, 2025 at 3:13 PM
Researchers introduced SPELL, an RL framework that lets LLMs improve long‑context reasoning. It achieved a 7.6‑point pass@8 gain on Qwen3‑30B‑A3B‑Thinking, September 2025. Read more: https://getnews.me/spell-self-play-rl-improves-long-context-llm-performance/ #spell #longcontext
Long‑context fine‑tuning lets LLMs outperform short‑context training on short‑context benchmarks, and researchers recommend hybrid mixes of both lengths for balanced performance. https://getnews.me/long-context-fine-tuning-improves-short-task-performance-of-llms/ #longcontext #finetuning
September 27, 2025 at 5:35 AM
Long‑context fine‑tuning lets LLMs outperform short‑context training on short‑context benchmarks, and researchers recommend hybrid mixes of both lengths for balanced performance. https://getnews.me/long-context-fine-tuning-improves-short-task-performance-of-llms/ #longcontext #finetuning
LiteLong can generate up to 128 K-token training samples using hierarchical BISAC categories and a BM25-based retrieval, achieving competitive results on HELMET and Ruler benchmarks. https://getnews.me/litelong-enables-efficient-long-context-data-synthesis-for-llms/ #litelong #longcontext #bisc
September 22, 2025 at 9:03 AM
LiteLong can generate up to 128 K-token training samples using hierarchical BISAC categories and a BM25-based retrieval, achieving competitive results on HELMET and Ruler benchmarks. https://getnews.me/litelong-enables-efficient-long-context-data-synthesis-for-llms/ #litelong #longcontext #bisc
TAPA adds a phase function to Rotary Positional Embedding, removing distance‑dependent bias and preserving interactions. Tests report lower perplexity on long‑context benchmarks. https://getnews.me/token-aware-phase-attention-boosts-long-context-transformer-performance/ #transformers #longcontext
September 18, 2025 at 9:18 AM
TAPA adds a phase function to Rotary Positional Embedding, removing distance‑dependent bias and preserving interactions. Tests report lower perplexity on long‑context benchmarks. https://getnews.me/token-aware-phase-attention-boosts-long-context-transformer-performance/ #transformers #longcontext
Bigger context ≠ better. Research: train short, then ladder to long context—higher quality, faster training. Scale the window last. #LLMTraining #LongContext https://glcnd.io/context-window-research-advances-in-training-efficiency-for-models/
January 20, 2026 at 9:16 AM
Bigger context ≠ better. Research: train short, then ladder to long context—higher quality, faster training. Scale the window last. #LLMTraining #LongContext https://glcnd.io/context-window-research-advances-in-training-efficiency-for-models/
#AI #LongContext #MITCSAIL #mlsky
MIT CSAIL proposes Recursive Language Models that treat long prompts as an external environment. The model writes Python3 to inspect, decompose, and recursively query itself, enabling 10M+ token inputs. On BrowseComp+ (1K) it hits 91.33%
arxiv.org/abs/2512.24601
MIT CSAIL proposes Recursive Language Models that treat long prompts as an external environment. The model writes Python3 to inspect, decompose, and recursively query itself, enabling 10M+ token inputs. On BrowseComp+ (1K) it hits 91.33%
arxiv.org/abs/2512.24601
Recursive Language Models
We study allowing large language models (LLMs) to process arbitrarily long prompts through the lens of inference-time scaling. We propose Recursive Language Models (RLMs), a general inference strategy...
arxiv.org
January 12, 2026 at 1:02 PM
#AI #LongContext #MITCSAIL #mlsky
MIT CSAIL proposes Recursive Language Models that treat long prompts as an external environment. The model writes Python3 to inspect, decompose, and recursively query itself, enabling 10M+ token inputs. On BrowseComp+ (1K) it hits 91.33%
arxiv.org/abs/2512.24601
MIT CSAIL proposes Recursive Language Models that treat long prompts as an external environment. The model writes Python3 to inspect, decompose, and recursively query itself, enabling 10M+ token inputs. On BrowseComp+ (1K) it hits 91.33%
arxiv.org/abs/2512.24601