This demo of structured data extraction running on an LLM that executes entirely in the browser (Chrome only for the moment since it uses WebGPU) is amazing
My notes here: simonwillison.net/2024/Nov/29/...
My notes here: simonwillison.net/2024/Nov/29/...
November 29, 2024 at 9:10 PM
This demo of structured data extraction running on an LLM that executes entirely in the browser (Chrome only for the moment since it uses WebGPU) is amazing
My notes here: simonwillison.net/2024/Nov/29/...
My notes here: simonwillison.net/2024/Nov/29/...
Making SmolLM2 more reproducible: open-sourcing our training & evaluation toolkit 🛠️ github.com/huggingface/...
Pre-training & evaluation code, synthetic data generation pipelines, post-training scripts, on-device tools & demos
Apache 2.0. V2 data mix coming soon!
Which tools should we add next?
Pre-training & evaluation code, synthetic data generation pipelines, post-training scripts, on-device tools & demos
Apache 2.0. V2 data mix coming soon!
Which tools should we add next?
GitHub - huggingface/smollm: Everything about the SmolLM & SmolLM2 family of models
Everything about the SmolLM & SmolLM2 family of models - GitHub - huggingface/smollm: Everything about the SmolLM & SmolLM2 family of models
github.com
November 24, 2024 at 7:16 AM
Making SmolLM2 more reproducible: open-sourcing our training & evaluation toolkit 🛠️ github.com/huggingface/...
Pre-training & evaluation code, synthetic data generation pipelines, post-training scripts, on-device tools & demos
Apache 2.0. V2 data mix coming soon!
Which tools should we add next?
Pre-training & evaluation code, synthetic data generation pipelines, post-training scripts, on-device tools & demos
Apache 2.0. V2 data mix coming soon!
Which tools should we add next?
... thanks to @artisanloaf.bsky.social I found SmolLM2-135M-Instruct, which quantizes down to just below 100MB... which means it can fit in a PyPI package!
Here's the first LLM plugin that includes a full model as part of the package: "llm install llm-smollm2"
simonwillison.net/2025/Feb/7/p...
Here's the first LLM plugin that includes a full model as part of the package: "llm install llm-smollm2"
simonwillison.net/2025/Feb/7/p...
Using pip to install a Large Language Model that’s under 100MB
I just released llm-smollm2, a new plugin for LLM that bundles a quantized copy of the SmolLM2-135M-Instruct LLM inside of the Python package. This means you can now pip install …
simonwillison.net
February 7, 2025 at 6:37 AM
... thanks to @artisanloaf.bsky.social I found SmolLM2-135M-Instruct, which quantizes down to just below 100MB... which means it can fit in a PyPI package!
Here's the first LLM plugin that includes a full model as part of the package: "llm install llm-smollm2"
simonwillison.net/2025/Feb/7/p...
Here's the first LLM plugin that includes a full model as part of the package: "llm install llm-smollm2"
simonwillison.net/2025/Feb/7/p...
It's "on-device LLM" today.
Soon, it'll be "on-chip" LLM. Or LLM cores. The system default local LLM. The coding framework's default local LLM.
I find this incredibly exciting. A privacy-first, self-contained, user-owned AI—a 24/7 agent for action, insights & feedback.
github.com/huggingface/...
Soon, it'll be "on-chip" LLM. Or LLM cores. The system default local LLM. The coding framework's default local LLM.
I find this incredibly exciting. A privacy-first, self-contained, user-owned AI—a 24/7 agent for action, insights & feedback.
github.com/huggingface/...
GitHub - huggingface/smollm: Everything about the SmolLM & SmolLM2 family of models
Everything about the SmolLM & SmolLM2 family of models - GitHub - huggingface/smollm: Everything about the SmolLM & SmolLM2 family of models
github.com
November 24, 2024 at 6:01 PM
It's "on-device LLM" today.
Soon, it'll be "on-chip" LLM. Or LLM cores. The system default local LLM. The coding framework's default local LLM.
I find this incredibly exciting. A privacy-first, self-contained, user-owned AI—a 24/7 agent for action, insights & feedback.
github.com/huggingface/...
Soon, it'll be "on-chip" LLM. Or LLM cores. The system default local LLM. The coding framework's default local LLM.
I find this incredibly exciting. A privacy-first, self-contained, user-owned AI—a 24/7 agent for action, insights & feedback.
github.com/huggingface/...
Finetune SmolLM2 with TRL!
Here's a notebook where I do SFT SmolLM2 on the synthetic dataset: colab.research.google.com/drive/1lioed...
thanks @philschmid.bsky.social for the finetuning code
thanks @huggingface.bsky.social for the smol model
thanks @qgallouedec.bsky.social and friends for TRL
thanks @philschmid.bsky.social for the finetuning code
thanks @huggingface.bsky.social for the smol model
thanks @qgallouedec.bsky.social and friends for TRL
Google Colab
colab.research.google.com
November 21, 2024 at 11:32 AM
Finetune SmolLM2 with TRL!
She introduces SmolLM2, a family of smol LMs in three sizes: 135M, 360M, 1.7B
Lots of work on the data side! Resulted in competitive performance on leader boards.
*Fully open* including datasets and public training, fine-tuning, eval, codebase under Apache 2.0
Lots of work on the data side! Resulted in competitive performance on leader boards.
*Fully open* including datasets and public training, fine-tuning, eval, codebase under Apache 2.0
GitHub - huggingface/smollm: Everything about the SmolLM2 and SmolVLM family of models
Everything about the SmolLM2 and SmolVLM family of models - GitHub - huggingface/smollm: Everything about the SmolLM2 and SmolVLM family of models
github.com
June 20, 2025 at 12:33 PM
She introduces SmolLM2, a family of smol LMs in three sizes: 135M, 360M, 1.7B
Lots of work on the data side! Resulted in competitive performance on leader boards.
*Fully open* including datasets and public training, fine-tuning, eval, codebase under Apache 2.0
Lots of work on the data side! Resulted in competitive performance on leader boards.
*Fully open* including datasets and public training, fine-tuning, eval, codebase under Apache 2.0
Today I learned about SmolLM2-135M-Instruct, a tiny LLM which quantizes down to just below 100MB... which means it can fit in a PyPI package!
Here's the first LLM plugin that includes a full model as part of the package:
llm install llm-smollm2 […]
Here's the first LLM plugin that includes a full model as part of the package:
llm install llm-smollm2 […]
Original post on fedi.simonwillison.net
fedi.simonwillison.net
February 7, 2025 at 6:38 AM
Today I learned about SmolLM2-135M-Instruct, a tiny LLM which quantizes down to just below 100MB... which means it can fit in a PyPI package!
Here's the first LLM plugin that includes a full model as part of the package:
llm install llm-smollm2 […]
Here's the first LLM plugin that includes a full model as part of the package:
llm install llm-smollm2 […]
If you have uv installed this means you can start chatting with a small model without first installing anything at all - this command will create an ephemeral virtual environment, install the necessary pieces and start a chat UI running in your terminal:
uvx --with llm-smollm2 llm chat -m SmolLM2
uvx --with llm-smollm2 llm chat -m SmolLM2
February 7, 2025 at 6:49 AM
If you have uv installed this means you can start chatting with a small model without first installing anything at all - this command will create an ephemeral virtual environment, install the necessary pieces and start a chat UI running in your terminal:
uvx --with llm-smollm2 llm chat -m SmolLM2
uvx --with llm-smollm2 llm chat -m SmolLM2
We hit 1K ⭐ on our SmolLM repo—thank you! 🎉 New updates:
• SmolLM2 nanotron checkpoints (with optimizer states) for easier continual pre-training
• Local inference demos (MLC, Transformers.js, MLX, llama.cpp)
• SmolVLM: Vision-language model built on SmolLM2
github.com/huggingface/...
• SmolLM2 nanotron checkpoints (with optimizer states) for easier continual pre-training
• Local inference demos (MLC, Transformers.js, MLX, llama.cpp)
• SmolVLM: Vision-language model built on SmolLM2
github.com/huggingface/...
December 1, 2024 at 8:00 AM
We hit 1K ⭐ on our SmolLM repo—thank you! 🎉 New updates:
• SmolLM2 nanotron checkpoints (with optimizer states) for easier continual pre-training
• Local inference demos (MLC, Transformers.js, MLX, llama.cpp)
• SmolVLM: Vision-language model built on SmolLM2
github.com/huggingface/...
• SmolLM2 nanotron checkpoints (with optimizer states) for easier continual pre-training
• Local inference demos (MLC, Transformers.js, MLX, llama.cpp)
• SmolVLM: Vision-language model built on SmolLM2
github.com/huggingface/...
smollm tools for running locally - including summarization/rewrite github.com/huggingface/...
smollm/smol_tools at main · huggingface/smollm
Everything about the SmolLM & SmolLM2 family of models - huggingface/smollm
github.com
December 1, 2024 at 8:11 AM
smollm tools for running locally - including summarization/rewrite github.com/huggingface/...
Fuck it! Structured Generation w/ SmolLM2 running in browser & WebGPU 🔥
Powered by MLC Web-LLM & XGrammar ⚡
Define a JSON schema, Input free text, get structured data right in your browser - profit!!
Powered by MLC Web-LLM & XGrammar ⚡
Define a JSON schema, Input free text, get structured data right in your browser - profit!!
November 28, 2024 at 10:24 PM
Fuck it! Structured Generation w/ SmolLM2 running in browser & WebGPU 🔥
Powered by MLC Web-LLM & XGrammar ⚡
Define a JSON schema, Input free text, get structured data right in your browser - profit!!
Powered by MLC Web-LLM & XGrammar ⚡
Define a JSON schema, Input free text, get structured data right in your browser - profit!!
What's the secret sauce of SmolLM2 to beat LLM titans like Llama3.2 and Qwen2.5?
Unsurprisingly: data, data, data!
The SmolTalk is open and available here: huggingface.co/datasets/Hug...
Unsurprisingly: data, data, data!
The SmolTalk is open and available here: huggingface.co/datasets/Hug...
November 21, 2024 at 2:17 PM
What's the secret sauce of SmolLM2 to beat LLM titans like Llama3.2 and Qwen2.5?
Unsurprisingly: data, data, data!
The SmolTalk is open and available here: huggingface.co/datasets/Hug...
Unsurprisingly: data, data, data!
The SmolTalk is open and available here: huggingface.co/datasets/Hug...
Amazing demo by Vaibhav Srivastav of structured data extraction running on an LLM that executes entirely in the browser (Chrome only for the moment since it uses WebGPU). https://simonwillison.net/2024/Nov/29/structured-generation-smollm2-webgpu/
Structured Generation w/ SmolLM2 running in browser & WebGPU
Extraordinary demo by Vaibhav Srivastav. Here's Hugging Face's [SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct) running directly in a web browser (using WebGPU, so requires Chrome [for the moment](https://github.com/gpuweb/gpuweb/wiki/Implementation-Status)) demonstrating structured text extraction, converting a text …
simonwillison.net
November 29, 2024 at 9:14 PM
Amazing demo by Vaibhav Srivastav of structured data extraction running on an LLM that executes entirely in the browser (Chrome only for the moment since it uses WebGPU). https://simonwillison.net/2024/Nov/29/structured-generation-smollm2-webgpu/
Excited to release SmolTalk: the secret recipe behind the best-in-class performance of SmolLM2
hf.co/datasets/Hug...
The SmolLM team at Hugging Face curated new synthetic datasets to enhance instruction following and reasoning. SmolTalk also supports rewriting, summarization, and function calling.
hf.co/datasets/Hug...
The SmolLM team at Hugging Face curated new synthetic datasets to enhance instruction following and reasoning. SmolTalk also supports rewriting, summarization, and function calling.
November 22, 2024 at 12:00 PM
Excited to release SmolTalk: the secret recipe behind the best-in-class performance of SmolLM2
hf.co/datasets/Hug...
The SmolLM team at Hugging Face curated new synthetic datasets to enhance instruction following and reasoning. SmolTalk also supports rewriting, summarization, and function calling.
hf.co/datasets/Hug...
The SmolLM team at Hugging Face curated new synthetic datasets to enhance instruction following and reasoning. SmolTalk also supports rewriting, summarization, and function calling.
Me post seeing SmolLM2 quant is fast enough to run on my tiny laptop
February 10, 2025 at 12:58 AM
Me post seeing SmolLM2 quant is fast enough to run on my tiny laptop
SmolLM2 paper by @loubnabnl.hf.co
A detailed guide for building SOTA small LMs. While most LM papers skim over data, they show how data is the secret sauce behind strong performance.
A detailed guide for building SOTA small LMs. While most LM papers skim over data, they show how data is the secret sauce behind strong performance.
February 7, 2025 at 5:17 AM
SmolLM2 paper by @loubnabnl.hf.co
A detailed guide for building SOTA small LMs. While most LM papers skim over data, they show how data is the secret sauce behind strong performance.
A detailed guide for building SOTA small LMs. While most LM papers skim over data, they show how data is the secret sauce behind strong performance.
Excited to announce the SFT dataset used for @huggingface.bsky.social SmolLM2!
The dataset for SmolLM2 was created by combining multiple existing datasets and generating new synthetic datasets, including MagPie Ultra v1.0, using distilabel.
Check out the dataset:
huggingface.co/datasets/Hug...
The dataset for SmolLM2 was created by combining multiple existing datasets and generating new synthetic datasets, including MagPie Ultra v1.0, using distilabel.
Check out the dataset:
huggingface.co/datasets/Hug...
HuggingFaceTB/smoltalk · Datasets at Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
November 21, 2024 at 3:22 PM
Excited to announce the SFT dataset used for @huggingface.bsky.social SmolLM2!
The dataset for SmolLM2 was created by combining multiple existing datasets and generating new synthetic datasets, including MagPie Ultra v1.0, using distilabel.
Check out the dataset:
huggingface.co/datasets/Hug...
The dataset for SmolLM2 was created by combining multiple existing datasets and generating new synthetic datasets, including MagPie Ultra v1.0, using distilabel.
Check out the dataset:
huggingface.co/datasets/Hug...
Really excited about new SmolLM2 from HuggingFace!
1.7B, 360M & 135M models, which will be very important size ranges for local automation tasks. Llama picked up on this with the recent 1B model.
1.7B, 360M & 135M models, which will be very important size ranges for local automation tasks. Llama picked up on this with the recent 1B model.
October 31, 2024 at 7:37 PM
Really excited about new SmolLM2 from HuggingFace!
1.7B, 360M & 135M models, which will be very important size ranges for local automation tasks. Llama picked up on this with the recent 1B model.
1.7B, 360M & 135M models, which will be very important size ranges for local automation tasks. Llama picked up on this with the recent 1B model.
Another great week in open ML!
Here's a small thread with this week's recap 🧵
💬 Small language models
@huggingface released SmolLM2, a new family of smol language models with Apache 2.0 license with, along with datasets of sizes 135M, 360M and 1.7B 🤗
Here's a small thread with this week's recap 🧵
💬 Small language models
@huggingface released SmolLM2, a new family of smol language models with Apache 2.0 license with, along with datasets of sizes 135M, 360M and 1.7B 🤗
October 31, 2024 at 7:40 PM
Another great week in open ML!
Here's a small thread with this week's recap 🧵
💬 Small language models
@huggingface released SmolLM2, a new family of smol language models with Apache 2.0 license with, along with datasets of sizes 135M, 360M and 1.7B 🤗
Here's a small thread with this week's recap 🧵
💬 Small language models
@huggingface released SmolLM2, a new family of smol language models with Apache 2.0 license with, along with datasets of sizes 135M, 360M and 1.7B 🤗
A team behind SmolLM2 model at @huggingface.bsky.social just released everything! A true open-source AI:
- Pre-training code
- Evaluation suite
- Synthetic data generation
- Post-training scripts with TRL
- On-device tools for summarization, rewriting & agents
All with Apache 2.0 licensed! 🔥
- Pre-training code
- Evaluation suite
- Synthetic data generation
- Post-training scripts with TRL
- On-device tools for summarization, rewriting & agents
All with Apache 2.0 licensed! 🔥
November 24, 2024 at 6:25 PM
A team behind SmolLM2 model at @huggingface.bsky.social just released everything! A true open-source AI:
- Pre-training code
- Evaluation suite
- Synthetic data generation
- Post-training scripts with TRL
- On-device tools for summarization, rewriting & agents
All with Apache 2.0 licensed! 🔥
- Pre-training code
- Evaluation suite
- Synthetic data generation
- Post-training scripts with TRL
- On-device tools for summarization, rewriting & agents
All with Apache 2.0 licensed! 🔥
📬 Summarize and rewrite your text/emails faster, and offline!
Check @andimara.bsky.social's Smol Tools for summarization and rewriting. It uses SmolLM2 to summarize text and make it more friendly or professional, all running locally thanks to llama.cpp github.com/huggingface/...
Check @andimara.bsky.social's Smol Tools for summarization and rewriting. It uses SmolLM2 to summarize text and make it more friendly or professional, all running locally thanks to llama.cpp github.com/huggingface/...
smollm/smol_tools at main · huggingface/smollm
Everything about the SmolLM & SmolLM2 family of models - huggingface/smollm
github.com
November 30, 2024 at 3:59 PM
📬 Summarize and rewrite your text/emails faster, and offline!
Check @andimara.bsky.social's Smol Tools for summarization and rewriting. It uses SmolLM2 to summarize text and make it more friendly or professional, all running locally thanks to llama.cpp github.com/huggingface/...
Check @andimara.bsky.social's Smol Tools for summarization and rewriting. It uses SmolLM2 to summarize text and make it more friendly or professional, all running locally thanks to llama.cpp github.com/huggingface/...
I'm mostly into little models these days. Not Smol, no, not any more, Smol sold out with SmolLM2. I was just listening to PicoMistral on the way over.
yeah you like instruct-tuned llm outputs? that’s cute, that’s how i got into neural networks but I’m really into the latent activation space of base models now. its a lot less accessible but its more authentic y’know
March 14, 2026 at 5:40 PM
I'm mostly into little models these days. Not Smol, no, not any more, Smol sold out with SmolLM2. I was just listening to PicoMistral on the way over.
Some notes on running the new SmolVM vision model (~4.2GB) on my MacBook Pro M2 at 75 tokens/second to describe a photo, using a shell one-liner powered by uv and mlx-vlm simonwillison.net/2024/Nov/28/...
SmolVLM—small yet mighty Vision Language Model
I've been having fun playing with this new vision model from the Hugging Face team behind [SmolLM](https://simonwillison.net/2024/Nov/2/smollm2/). They describe it as: > [...] a 2B VLM, SOTA for its m...
simonwillison.net
November 28, 2024 at 8:30 PM
Some notes on running the new SmolVM vision model (~4.2GB) on my MacBook Pro M2 at 75 tokens/second to describe a photo, using a shell one-liner powered by uv and mlx-vlm simonwillison.net/2024/Nov/28/...
SmolLM team used Hugging Face's nanotron (github.com/huggingface/...) to pre-train their SmolLM2 (huggingface.co/collections/...) on 256 nvidia H100s.
GitHub - huggingface/nanotron: Minimalistic large language model 3D-parallelism training
Minimalistic large language model 3D-parallelism training - huggingface/nanotron
github.com
November 17, 2024 at 6:01 PM
SmolLM team used Hugging Face's nanotron (github.com/huggingface/...) to pre-train their SmolLM2 (huggingface.co/collections/...) on 256 nvidia H100s.
WOW! 🤯 Language models are becoming smaller and more capable than ever! Here's SmolLM2 running 100% locally in-browser w/ WebGPU on a 6-year-old GPU. Just look at that speed! ⚡️😍
Powered by 🤗 Transformers.js and ONNX Runtime Web!
How many tokens/second do you get? Let me know! 👇
Powered by 🤗 Transformers.js and ONNX Runtime Web!
How many tokens/second do you get? Let me know! 👇
November 27, 2024 at 1:51 PM
WOW! 🤯 Language models are becoming smaller and more capable than ever! Here's SmolLM2 running 100% locally in-browser w/ WebGPU on a 6-year-old GPU. Just look at that speed! ⚡️😍
Powered by 🤗 Transformers.js and ONNX Runtime Web!
How many tokens/second do you get? Let me know! 👇
Powered by 🤗 Transformers.js and ONNX Runtime Web!
How many tokens/second do you get? Let me know! 👇