Iran's president insisted on Wednesday that the United States would achieve nothing with its new sanctions campaign after Washington laid out plans to collapse the country's economy. u.afp.com/SVfT

🇳🇵 Two hydropower workers were pulled alive on Friday from a tunnel buried by debris in Nepal, a rare ray of hope on the tenth day of rescue efforts after catastrophic Himalayan flooding.
➡️ u.afp.com/SVFT

Are you still using LoRA to fine-tune your LLM? 2024 has seen an explosion of new parameter-efficient fine tuning technique (PEFT), thanks to clever uses of the singular value decomposition (SVD). Let's dive into the alphabet soup: SVF, SVFT, MiLoRA, PiSSA, LoRA-XS 🤯...

Missed out on #Swift tickets? No worries—swing by our #SVFT poster at #NeurIPS2024 and catch *real* headliners! 🎤💃🕺
📌Where: East Exhibit Hall A-C #2207, Poster Session 4 East
⏲️When: Thu 12 Dec, 4:30 PM - 7:30 PM PST

#AI #MachineLearning #PEFT #NeurIPS24

svft

Should you need more trainable parameters, the SVFT paper (arxiv.org/abs/2405.19597) explores multiple ways of doing that, starting by adding more trainable weights on the diagonal...

My conclusion: to go beyond the LoRA standard with 10x fewer params, I like the simplicity of Transformers²'s SVF. And if you need more trainable weights, SVFT is an easy extension. Both use all singular values (full rank, no s.v. pruning) and are still cheap 😁. Happy tuning!

More importantly, the SVFT paper confirms that having more trainable values than just the diagonal is useful:

5/ SVFT: LoRA-style adapters parameterized by singular vectors of W; trains sparse coefficients and recovers up to 96% of full fine-tune performance with 0.006-0.25% params. X: https://x.com/atu_tej/status/2100629421065400484 Paper: https://arxiv.org/abs/2405.19597

EFTA00522128.jpg
#epsteinweb #efta00522128
https://epsteinweb.org
Available in the iOS app store now!
https://apps.apple.com/us/app/epstein-web/id6758880661

"As Vice President of the SVFT-R Chapter of AFT CT, AFL-CIO, she remained deeply engaged in organized labor and many causes she held dear." #LaborHistory @aftteach.bsky.social

#Warsaw #Warszawa Warszawa urośnie. W ciągu pięciu lat może powstać 19 nowych wieżowców https://wars.aw/sVFT

Friday, Dec. 13 at 11 a.m.: A very Krampus Christmas

Holiday fun with an attitude. Kickin It with Krampus is sponsored by South Vineland Firehouse Tattoo (SVFT), 2721 S. Main Rd., Vineland.
southjersey.media/?p=238250

#TechRadar Morocco vs Comoros free streams: How to watch AFCON 2025 Group A fixture for no cost at all https://techrad.ar/sVFT #HowtoWatchFootball #HowtoWatchSport #HowtoWatch

There have been many PEFT techniques based on the singular value decomposition: LoRA-XS, MiLoRA, PiSSA, SVFT... Some focus on large singular values, some only on the smaller one. But SVF is the simplest of them all: it fine-tunes the singular values directly.

"...solo estaba pensando lo inútil que es delirar,
Y creer estar bien cuando es invierno y tu
mantienes tus manos lejos y no me abrazas..."

nanananananana

No me lo puedo explicar
io no me lo so spiegare

www.youtube.com/watch?v=svFT...