Slow but that’s GGUF so I’m excited for MLX
Our looped transformer model with 26B params :)
thinkingmachines.ai/news/inkling...
Slow but that’s GGUF so I’m excited for MLX
mlx-serve lets you run LLMs and media generation offline on Apple Silicon. Install the app by dragging it to Applications—no Python needed—and access OpenAI-compatible APIs right on your Mac.
🍒 Built by David Dalcu
github.com/ddalcu/mlx-serve
#opensource #zig
mlx-serve lets you run LLMs and media generation offline on Apple Silicon. Install the app by dragging it to Applications—no Python needed—and access OpenAI-compatible APIs right on your Mac.
🍒 Built by David Dalcu
github.com/ddalcu/mlx-serve
#opensource #zig
Check it out now: 👇👇💡
wwdcnotes.com/documentatio...
Check it out now: 👇👇💡
wwdcnotes.com/documentatio...
as Server: oMLX - Version 0.8.2 (3849)
mtp_enabled: true, mtp_adaptive_max_depth: 3, turboquant_kv_enabled: true, turboquant_kv_bits / skip_last: 4.0 / true, temperature 1.0
using your prompt with Pi
as Server: oMLX - Version 0.8.2 (3849)
mtp_enabled: true, mtp_adaptive_max_depth: 3, turboquant_kv_enabled: true, turboquant_kv_bits / skip_last: 4.0 / true, temperature 1.0
using your prompt with Pi
mlx-lm can run on its own from the command line, omlx ships with a version of mlx-lm inside it, and you can definitely run mlx models that way
mlx-lm can run on its own from the command line, omlx ships with a version of mlx-lm inside it, and you can definitely run mlx models that way
https://zenn.dev/yamato_snow/articles/local-llm-secretary-mlx-measurements
https://zenn.dev/yamato_snow/articles/local-llm-secretary-mlx-measurements
Context: This acquisition marks Amazon's strategic push into specialized generative AI technologies.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/financialtimes.com/post/3n2tn2zbkxg2b
Context: This acquisition marks Amazon's strategic push into specialized generative AI technologies.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/financialtimes.com/post/3n2tn2zbkxg2b
Context: The discussion focuses on environmental ethics and human impact.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2r57huvot25
Context: The discussion focuses on environmental ethics and human impact.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2r57huvot25
Context: De Villepin previously served as the French Prime Minister.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lefigaro.fr/post/3n2qgrldb6d2p
Context: De Villepin previously served as the French Prime Minister.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lefigaro.fr/post/3n2qgrldb6d2p
• decode: 17–23 tok/s, and turning MTP off made no measurable difference.
• prefill: 130–170 tok/s.
• memory: 17.6 GB loaded, growing about 0.25 GB per 1k tokens of context.
• decode: 17–23 tok/s, and turning MTP off made no measurable difference.
• prefill: 130–170 tok/s.
• memory: 17.6 GB loaded, growing about 0.25 GB per 1k tokens of context.
Context: France is looking to maintain momentum after a major victory in rugby.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2skfanguf2v
Context: France is looking to maintain momentum after a major victory in rugby.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2skfanguf2v
Context: Salmon populations are d ...
Fact Check by gemma4:26b-mlx https://bsky.app/profile/afp.com/post/3n2tlbi5rcs2p
Context: Salmon populations are d ...
Fact Check by gemma4:26b-mlx https://bsky.app/profile/afp.com/post/3n2tlbi5rcs2p
Context: This news highlights the resurfacing of historical records involving the elderly.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2qn2qejbj2v
Context: This news highlights the resurfacing of historical records involving the elderly.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2qn2qejbj2v
Context: The Rohingya face systemic persecution and violence in Myanmar.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2r675grlz2a
Context: The Rohingya face systemic persecution and violence in Myanmar.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2r675grlz2a
Context: Tensions are rising between Russia and NATO members.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2trnepywh2v
Context: Tensions are rising between Russia and NATO members.
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2trnepywh2v
Context: The protest highlights growing tensions regarding fossil fuel indus ...
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2m6za3yvf2t
Context: The protest highlights growing tensions regarding fossil fuel indus ...
Fact Check by gemma4:26b-mlx https://bsky.app/profile/lemonde.fr/post/3n2m6za3yvf2t