#ModelMerging
LightNews LightNews
nic221.bsky.social
How Sakana AI’s new evolutionary algorithm builds powerful AI models without expensive retraining https://venturebeat.com/ai/how-sakana-ais-new-evolutionary-algorithm-builds-powerful-ai-models-without-expensive-retraining/ #AI #ModelMerging
Text Shot: Model merging is a technique for integrating the knowledge of multiple specialized AI models into a single, more capable model. Instead of fine-tuning, which refines a single pre-trained model using new data, merging combines the parameters of several models simultaneously. This process can consolidate a wealth of knowledge into one asset without requiring expensive, gradient-based training or access to the original training data. For enterprise teams, this offers several practical advantages over traditional fine-tuning. In comments to VentureBeat, the paper’s authors said model merging is a gradient-free process that only requires forward passes, making it computationally cheaper than fine-tuning, which involves costly gradient updates. Merging also sidesteps the need for carefully balanced training data and mitigates the risk of “catastrophic forgetting,” where a model loses its original capabilities after learning a new task. The technique is especially powerful when…
August 30, 2025 at 5:57 AM
haiyuesong.bsky.social
🎉 Excited to share that our paper "OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training" has been accepted to EMNLP 2026 Main! See you in Budapest!

📄 Paper: arxiv.org/abs/2603.28858

#EMNLP2026 #ModelMerging #ContinualPretraining #BayesianOptimization
OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training
Continual pre-training is widely used to adapt LLMs to target languages and domains, yet the mixture ratio of training data remains a sensitive hyperparameter that is expensive to tune: they must be f...
arxiv.org
September 15, 2026 at 5:24 AM
malikeh97.bsky.social
Excited to be at @neuripsconf.bsky.social next week co-presenting our tutorial: "Model Merging: Theory, Practice, and Applications" 🔥

Proud to do this with my PhD advisor, Colin Raffel, our postdoc research fellow @mcicc.bsky.social, and an incredible panel of speakers 💙

#NeurIPS2025 #ModelMerging
November 27, 2025 at 2:43 PM
pneumetron.bsky.social
🔥 Consolidating Corporate LLM Traffic: A New Recipe for Self-Hosted Efficiency

https://pneumetron.com/news/ai_research/consolidating-corporate-llm-traffic-self-hosted-efficiency-c5dc80

#LLM #GRPO #SLERP #ModelMerging
September 5, 2026 at 6:42 AM
pneumetron.bsky.social
ISO: Unlocking Efficient RLVR Through Spectral Inheritance

https://pneumetron.com/news/ai_research/iso-rlvr-spectral-inheritance-0ff3c1

#RLVR #Optimization #MachineLearning #ModelMerging
July 23, 2026 at 3:06 AM
startuphub.bsky.social
Forget complex alignment! Direct weighted averaging for LLM merging shows promise, but beware the 'seesaw effect' from unbalanced ratios. Limits of simple fusion may bound advanced techniques. #AI #LLMs #ModelMerging

https://www.startuphub.ai/ai-news/ai-research/2026/simple-llm-merging-surprises
Simple LLM Merging Surprises
Direct weighted averaging of LLMs with dimensional adaptation proves surprisingly effective, but ratio control is paramount to avoid capability collapse.
www.startuphub.ai
July 22, 2026 at 6:05 PM
getnews-me.bsky.social
SyMerge introduces a single-layer adaptation technique for synergistic model merging. Read more: https://getnews.me/symerge-single-layer-adaptation-for-synergistic-model-merging/ #symerge #modelmerging #ai
SyMerge: single-layer adaptation for synergistic model merging
October 8, 2025 at 3:25 PM
getnews-me.bsky.social
Effective noise scale—combining learning rate, weight decay, batch size and augmentation—predicts model‑merging success, with a non‑monotonic optimum. Read more: https://getnews.me/optimizer-noise-shapes-model-merging-success-in-neural-networks/ #modelmerging #effectivenoisescale #optimizers
Optimizer Noise Shapes Model Merging Success in Neural Networks
October 8, 2025 at 3:49 AM
getnews-me.bsky.social
Chain of Merges (CoM) merges weights layer‑by‑layer, updates activation stats to curb covariate shift, and hits state‑of‑the‑art results on several benchmarks. Read more: https://getnews.me/chain-of-merges-new-layer-wise-method-improves-model-fusion/ #modelmerging #chainofmerges
Chain of Merges: New Layer‑wise Method Improves Model Fusion
October 3, 2025 at 9:34 AM
getnews-me.bsky.social
Study shows a one‑epoch fine‑tune yields a vector = –η∇L, linking arithmetic to gradients. Vision benchmarks confirm the first‑epoch gradient dominates, enabling merging. Read more: https://getnews.me/task-vectors-match-gradient-descent-theory-improves-model-merging/ #taskarithmetic #modelmerging
Task Vectors Match Gradient Descent: Theory Improves Model Merging
October 3, 2025 at 9:26 AM
getnews-me.bsky.social
Researchers unveiled a compact power-law scaling rule linking base model size and number of experts (k), showing gains drop about 1/k. Paper submitted September 2025. Read more: https://getnews.me/new-scaling-laws-reveal-predictable-gains-from-model-merging-in-llms/ #modelmerging #scalinglaws #llms
New Scaling Laws Reveal Predictable Gains from Model Merging in LLMs
September 30, 2025 at 7:14 PM
getnews-me.bsky.social
A recent study finds model merging provides little protection against transfer attacks, with a relative success rate over 95% across eight merging methods. Read more: https://getnews.me/model-merging-increases-vulnerability-to-adversarial-transfer-attacks/ #modelmerging #adversarial
Model Merging Increases Vulnerability to Adversarial Transfer Attacks
September 30, 2025 at 11:58 AM
getnews-me.bsky.social
Model merging blends two pretrained LLMs via weight arithmetic, creating models; the study found Pareto improvements where the merged model beat a parent on accuracy and token usage. Read more: https://getnews.me/model-merging-enables-tunable-reasoning-performance-in-llms/ #modelmerging #llms
Model Merging Enables Tunable Reasoning Performance in LLMs
September 29, 2025 at 11:13 AM
getnews-me.bsky.social
Linear weight merging of a retrieval model with a domain‑specific one improves search in medicine and Japanese text, matching LoRA fine‑tuning even with limited data. Read more: https://getnews.me/model-merging-boosts-domain-specific-ad-hoc-retrieval-effectiveness/ #modelmerging #adhocretrieval
Model Merging Boosts Domain‑Specific Ad‑hoc Retrieval Effectiveness
September 29, 2025 at 10:33 AM
getnews-me.bsky.social
Researchers merged Qwen2.5 models, then tested them on the MMLU benchmark and probing of morphology and syntax, finding stronger linguistic knowledge despite mid scores. Read more: https://getnews.me/evaluation-pipeline-connects-model-merging-behavior-and-internals/ #modelmerging #mmlu #probing
Evaluation Pipeline Connects Model Merging Behavior and Internals
September 26, 2025 at 2:00 PM
getnews-me.bsky.social
PriME merges language models with evolutionary algorithms, boosting task performance by up to 45% on the LaMP benchmark while cutting membership‑inference risk. Read more: https://getnews.me/privacy-preserving-evolutionary-merging-improves-language-models/ #privacy #modelmerging
Privacy-Preserving Evolutionary Merging Improves Language Models
September 22, 2025 at 10:08 PM
getnews-me.bsky.social
A model‑merging technique superposes task‑specific features with transformation matrices, avoiding fine‑tuning. Benchmarks in NLP and vision showed accuracy exceeding heuristic averaging. Read more: https://getnews.me/new-method-superposes-task-specific-features-for-model-merging/ #modelmerging #ml
New Method Superposes Task‑Specific Features for Model Merging
September 20, 2025 at 8:21 AM
kinews24.bsky.social
Kollektive KI-Intelligenz – Evolutionäre Algorithmen können Sprachmodelle weiterentwickeln

#KünstlicheIntelligenz #artificialintelligence #KI #AI #FoundationModels #Evolution #ModelMerging #Sprachmodelle #EvolutionäreAlgorithmen

kinews24.de/kollektive-k...
March 25, 2024 at 7:25 AM
azoai.bsky.social
🔥🤖📈Combining Large Models Unlocks New Levels Of Performance In AI Research www.azoai.com/news/2024101... #AIresearch #modelmerging #scalability #generalization #largemodels #expertmodels #zeroshot #instructiontuning #multitasktraining
Combining Large Models Unlocks New Levels Of Performance In AI Research
Researchers explored large-scale model merging, showing that combining large instruction-tuned models improves performance and generalization across various tasks, even surpassing multitask-trained mo...
www.azoai.com
October 14, 2024 at 5:29 AM