#TextEncoder
zenn.dev/sktt_panda/articl...
Base64エンコード/デコードをブラウザで実装する — 日本語で崩れるbtoa/atobをTextEncoderで直す
Base64エンコード/デコードをブラウザで実装する — 日本語で崩れるbtoa/atobをTextEncoderで直す
zenn.dev
September 13, 2026 at 6:07 AM
I really like this project 😍

Two ideas for maybe future additions

1. Show the gzip size next to file size
(TextEncoder + CompressionStream)

2. Allow setting browser targets
Often we/clients have a support minimum e.g. iOS 15. Based on this you could warn/note features that might not work
September 9, 2026 at 3:51 PM
New big yavascript update :3
github.com/suchipi/yava...

Highlights:
- ES2020 -> ES2023
- Improved top-level await support
- Added TextEncoder/TextDecoder
- Added import attribute support
- exec/Worker improvements on Windows
- Bytecode (de)serializer improvements.
- FreeBSD support added
Release v0.16.0 · suchipi/yavascript
This is a big update which has been a long time coming! New Stuff The underlying engine has been updated to support most of the ES2023 specification (it was previously ES2020). Notably, several ne...
github.com
June 7, 2026 at 4:56 AM
How to convert a single safetensors file to PEFT format
Oh. The size drop may be because the conversion above does not include the MLP LoRA tensors: * * * LLM-generated notes / rough analysis: I think the `850 MB -> 378 MB` drop is probably explainable from the converter itself, and the most likely cause is **not TextEncoder being skipped** , but rather **MLP LoRA tensors being skipped by default**. The relevant converter is this one: OpenxAILabs/Qwen-Image-2512-Lightning-8steps-V1.0-bf16-PEFT The script says: # Attention-only by default (recommended). You can optionally include MLP keys with --include-mlp. ALLOWED_QWEN_PREFIXES_ATTN = ( "attn.to_q", "attn.to_k", "attn.to_v", "attn.to_out", "attn.add_q_proj", "attn.add_k_proj", "attn.add_v_proj", "attn.to_add_out", ) # Optional MLP keys observed in Qwen-Image-Lightning (ComfyUI-style) ALLOWED_QWEN_PREFIXES_MLP = ( "img_mlp.net.0.proj", "img_mlp.net.2", "txt_mlp.net.0.proj", "txt_mlp.net.2", ) And the actual filter is: allowed_prefixes = ALLOWED_QWEN_PREFIXES_ATTN + ( ALLOWED_QWEN_PREFIXES_MLP if include_mlp else () ) So, unless `--include-mlp` is passed, the converter keeps only the attention/projection LoRA tensors and drops: img_mlp.net.0.proj img_mlp.net.2 txt_mlp.net.0.proj txt_mlp.net.2 This also matches the uploaded PEFT adapter’s `adapter_config.json` idea: the default target modules are attention/projection-ish modules, not MLP modules. Relevant links: * PEFT upload / script: OpenxAILabs/Qwen-Image-2512-Lightning-8steps-V1.0-bf16-PEFT * Original 850 MB file: lightx2v/Qwen-Image-2512-Lightning-8steps-V1.0-bf16.safetensors * vLLM-Omni LoRA docs: LoRA - vLLM-Omni * Qwen-Image transformer implementation: qwen_image_transformer.py ## Why the file size matches attention-only almost exactly From the vLLM-Omni Qwen-Image transformer implementation, the default model shape is roughly: num_layers = 60 num_attention_heads = 24 attention_head_dim = 128 inner_dim = 24 * 128 = 3072 The uploaded LoRA seems to be rank 64 / bf16. bf16 is 2 bytes per element. For one LoRA linear projection with shape `3072 -> 3072` and rank 64: lora_A: 64 x 3072 lora_B: 3072 x 64 elements = 64*3072 + 3072*64 = 393,216 bytes = 393,216 * 2 = 786,432 bytes = 0.75 MiB The default converter keeps 8 attention projections per block: attn.to_q attn.to_k attn.to_v attn.to_out attn.add_q_proj attn.add_k_proj attn.add_v_proj attn.to_add_out So the size estimate is: 0.75 MiB * 8 projections * 60 blocks = 360 MiB In decimal MB: 360 MiB = 377.5 MB That is almost exactly the reported converted size, `378 MB`. So I think the converted adapter size is not mysterious: it is basically the theoretical size of: 60 blocks * 8 attention LoRA projections * rank 64 * bf16 ## Why the original 850 MB also matches attention + MLP The original file is listed as `850 MB` here: Qwen-Image-2512-Lightning-8steps-V1.0-bf16.safetensors The missing difference is: 850 MB - 378 MB ~= 472 MB That also matches the expected MLP LoRA size. Qwen-Image blocks contain both image-stream and text-stream MLPs: img_mlp txt_mlp The converter explicitly recognizes these MLP keys: img_mlp.net.0.proj img_mlp.net.2 txt_mlp.net.0.proj txt_mlp.net.2 Assuming a usual MLP expansion of 4x, the MLP hidden size is approximately: inner_dim * 4 = 3072 * 4 = 12288 For one MLP LoRA linear `3072 -> 12288` or `12288 -> 3072`, rank 64: elements = 64*3072 + 12288*64 = 983,040 bytes = 983,040 * 2 = 1,966,080 bytes = 1.875 MiB There are 4 such MLP linears per block: img_mlp.net.0.proj img_mlp.net.2 txt_mlp.net.0.proj txt_mlp.net.2 So: 1.875 MiB * 4 * 60 = 450 MiB In decimal MB: 450 MiB = 471.9 MB That is basically the whole missing part. So the size arithmetic is: attention LoRA only ~= 377.5 MB MLP LoRA ~= 471.9 MB -------------------------------- total ~= 849.4 MB This is almost exactly the original `850 MB`. Therefore my rough conclusion is: original 850 MB ~= attention LoRA + MLP LoRA converted 378 MB ~= attention LoRA only ## So is there information loss? Probably yes, if the goal is to preserve the original LoRA exactly. But it is a specific kind of information loss: * attention/projection LoRA is preserved * MLP LoRA is probably dropped * `.alpha` keys are skipped, but those are tiny and not the source of the size drop * TextEncoder is not needed to explain the size drop I would not assume that this means the converted LoRA is useless. Attention-only LoRA can still have a strong effect, especially on rough prompt binding / layout / style direction. But for a Lightning/distillation LoRA, dropping the MLP part may reduce the low-step quality, details, texture, text rendering, and stability. My guess: simple prompts: maybe fairly close normal prompts: likely usable, but weaker than full LoRA complex text/layout: likely more visible degradation 4-step / 8-step edge cases: degradation likely more visible ## Why TextEncoder is probably not the main explanation TextEncoder skipping is possible in other LoRA conversion contexts, but here it is not necessary to explain the numbers. The converter targets keys like: transformer_blocks.N.<module>.lora_down.weight transformer_blocks.N.<module>.lora_up.weight It is not really written as a generic `text_encoder` / `lora_te` converter. Also, the sizes line up too cleanly with: attention-only = 378 MB attention + MLP = 850 MB So I would explain the size drop as MLP exclusion first, not TextEncoder exclusion. ## Can we keep MLP? Maybe. The script already has an option: python comfyui-to-vllm-omni-qwenimage.py \ --input Qwen-Image-2512-Lightning-8steps-V1.0-bf16.safetensors \ --output ./out_adapter_with_mlp \ --dtype bf16 \ --base-model Qwen/Qwen-Image-2512 \ --include-mlp If this works as intended, I would expect `adapter_model.safetensors` to become close to `850 MB`. However, the converter itself warns that MLP can be tricky: ap.add_argument( "--include-mlp", action="store_true", help="Also convert img_mlp/txt_mlp LoRA keys (may fail if vLLM expects different suffixes)", ) The likely issue is not writing the tensors. Writing the tensors is easy. The issue is whether vLLM-Omni accepts and correctly applies the MLP module suffixes. For example, the MLP targets include: img_mlp.net.0.proj img_mlp.net.2 txt_mlp.net.0.proj txt_mlp.net.2 Their suffixes are roughly: proj 2 `proj` is probably okay. The numeric suffix `2` may be the fragile part, because vLLM/vLLM-Omni LoRA validation can be strict about module suffixes. There is already a related vLLM issue for numeric-index module names such as `to_out.0`: vLLM issue #35734: LoRA loading fails for modules with numeric indices The current converter already works around the attention-side version of this by normalizing: attn.to_out.0 -> attn.to_out attn.to_add_out.0 -> attn.to_add_out But `net.2` is a different case. It may require the vLLM-Omni build to include `"2"` in expected LoRA modules, or it may need a more model-specific mapping. ## Suggested sanity check If anyone tries `--include-mlp`, I would check three things: ### 1. Size ls -lh ./out_adapter_with_mlp/adapter_model.safetensors Expected: ~850 MB If it is still around `378 MB`, MLP tensors were not included. ### 2. Key counts from safetensors.torch import load_file sd = load_file("./out_adapter_with_mlp/adapter_model.safetensors") for needle in [ "img_mlp.net.0.proj", "img_mlp.net.2", "txt_mlp.net.0.proj", "txt_mlp.net.2", ]: print(needle, sum(1 for k in sd if needle in k)) Expected rough count: each MLP target: 60 blocks * 2 tensors = 120 keys ### 3. vLLM-Omni load log The important question is whether vLLM-Omni reports that MLP modules were loaded and not silently ignored. The vLLM-Omni LoRA docs require a PEFT-style adapter folder: lora_adapter/ ├── adapter_config.json └── adapter_model.safetensors Docs: vLLM-Omni LoRA guide If loading fails on `net.2` / `"2"` / target module validation, then I think the clean solution would be either: 1. patch the converter / `adapter_config.json` target modules, or 2. patch vLLM-Omni’s diffusion LoRA mapper / supported modules for Qwen-Image MLP, or 3. avoid runtime adapter loading and fuse the LoRA into the base model. ## Practical recommendation For runtime PEFT LoRA: 1. Try the existing converter with `--include-mlp`. 2. Confirm the output is around `850 MB`. 3. Confirm `img_mlp` / `txt_mlp` keys exist. 4. Try loading in vLLM-Omni. 5. If it fails, the likely blocker is target module suffix validation around `net.2`. For maximum quality / minimum loader trouble: * fuse/merge the original LoRA into the Qwen-Image-2512 base weights using Diffusers or the reference loader * serve the fused model as a normal model in vLLM-Omni That avoids the whole PEFT key validation problem, although it is no longer a runtime LoRA adapter. ## TL;DR I think the 378 MB file is probably an attention-only converted adapter. The original 850 MB size is almost exactly: attention LoRA ~= 378 MB MLP LoRA ~= 472 MB total ~= 850 MB So the size drop is probably explained by the converter’s default behavior: attention-only by default MLP only if --include-mlp is passed `--include-mlp` may preserve the missing tensors, but whether vLLM-Omni can load/apply `img_mlp.net.2` and `txt_mlp.net.2` correctly is the part that needs testing.
discuss.huggingface.co
May 27, 2026 at 11:59 PM
Base64 Encode

Convert your text to Base64 with this easy to use encoder

#Base64Converter #TextEncoder #OnlineEncoder #EncodeYourText #Base64Decode
Base64 Encode
Convert your text to Base64 with this easy to use encoder
webby.tools
May 25, 2026 at 1:20 AM
Counting post length, it's graphemes, your UI client should use something like Intl.Segmenter to split the string and count visual clusters

But ATProto facet offsets are *bytes*, so encode the string to UTF-8 using (TextEncoder().encode(text)) then calc the start+end offsets from the Uint8Array.
May 5, 2026 at 11:35 PM
まさかの手動解読を……!?

UTF-8で入力された文字をTextEncoderでUtf8Arrayのバイト配列にして、今回はGBK(Simplified Chinese GBK)で無理やりデコードした結果で出してます。

細かい変換のところまでは把握できてないですが……
May 3, 2026 at 4:51 PM
🦖 TextEncoder: encoding property 🦖

https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder/encoding

The TextEncoder.encoding read-only property returns a string containing the name of the encoding algorithm used by the specific encoder.

#webdev
TextEncoder: encoding property
The TextEncoder.encoding read-only property returns a string containing the name of the encoding algorithm used by the specific encoder.
developer.mozilla.org
April 18, 2026 at 8:56 AM
🦖 Random MDN: TextEncoder: encoding property 🦖

https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder/encoding

The TextEncoder.encoding read-only property returns a string containing the name of the encoding algorithm used by the specific encoder.

#webdev
TextEncoder: encoding property
The TextEncoder.encoding read-only property returns a string containing the name of the encoding algorithm used by the specific encoder.
developer.mozilla.org
March 5, 2026 at 2:29 AM
Something like this. It seems to work, and I hope I didn't miss anything (code in the alt text) 🙃
January 27, 2026 at 4:04 PM
Here's what it supports so far:
- console, fs, process, path, Buffer, URL, EventEmitter, TextEncoder/TextDecoder
- Both callback and Promise APIs work setTimeout, setInterval, async/await
- Proper exception handling with stack traces
January 2, 2026 at 3:08 PM
and facets are done, now to make it look good
idk how bad the code is, but it works
November 29, 2025 at 9:13 PM
ECMAScript excitement 😉

Congrats to @jasnell.me on advancing Typed Array Find Within to Stage 1 at TC39 today 🎉

It aims to provide a native indexOf operations for Typed Arrays.

docs.google.com/presentation...
November 18, 2025 at 8:05 AM
ECMAScript excitement 😉

Congrats to @jasnell.me on advancing Typed Array Concatenation to Stage 1 at TC39 today 🎉

It aims to improve the performance of this concat operation compared to userland libraries.
November 18, 2025 at 7:52 AM
Is TextEncoder (developer.mozilla.org/en-US/docs/W...) the API you mention? I think that solves this issue, no? Seems like it's baseline widely supported; have you seen issues with it?
TextEncoder - Web APIs | MDN
The TextEncoder interface enables you to encode a JavaScript string using UTF-8.
developer.mozilla.org
November 6, 2025 at 10:33 AM
I disliled the default web syntax to call .pipeThrough(new TextEncoder()) or other class based options I thought about:

"To have unified convention, a class-based system, all classes must then use a .readable and .writable property...
October 14, 2025 at 7:29 AM
У старого проекту на роботі є приховані таланти - ламається не відразу, а через декілька комітів) Видно що з душею зроблене.. Помилка - TextEncoder is not defined у тестах
September 9, 2025 at 6:38 AM
Widely available: Text encoding and decoding

Description: The TextEncoder API transforms a stream of code points into a byte stream with UTF-8 encoding, and TextDecoder does the reverse.

Text encoding and decoding on Webplatform Feature Explorer
The TextEncoder API transforms a stream of code points into a byte stream with UTF-8 encoding, and TextDecoder does the reverse.
web-platform-dx.github.io
August 26, 2025 at 12:40 PM
Why is TextEncoder Web API but Uint8Array JS? Here's a fun one: clearly *modules* are just JavaScript, right? No. *Technically*, half the spec is in ECMA, the other half is HTML5. There are good *implementation* reasons for why it arguably *has* to be this way, but its a mess from the outside.
June 29, 2025 at 7:03 PM
And if you want to work with text then you need to use the asymmetric TextEncoder/TextDecoder 🫠
June 18, 2025 at 6:11 AM
This month in Servo: Android nightlies, right-to-left, WebGPU, and more!
Servo has had several new features land in our nightly builds over the last month: * as of 2024-09-10, we now support **< ul type>** and **< ul compact>** (@simonwuelker, #33303) * as of 2024-09-10, we now support **console.timeLog()** (@simonwuelker, #33377) * as of 2024-09-10, we now support the **encodeInto() method on TextEncoder** (@webbeef, @mrobinson, #33360) * as of 2024-09-10, we now support **< link rel=prefetch>** (@simonwuelker, #33345) * as of 2024-09-12, we now support **right-to-left languages** , except for floats (@mrobinson, @atbrakhi, #33375) * as of 2024-09-14, we now support **‘table-layout: fixed’** (@Loirooriol, @mrobinson, #33384, #33442) * as of 2024-09-17, we now support the **‘reset’ event on XRReferenceSpace** properties (@msub2, #33460) * as of 2024-09-19, we now support the **‘object-fit’** and **‘object-position’** properties (@mrobinson, @Loirooriol, #33479) * as of 2024-09-19, **Crypto.getRandomValues()** can now take **BigInt64Array** or **BigUint64Array** (@msub2, #33485) * as of 2024-09-25, we now support **‘innerText’** and **‘outerText’ on HTMLElement** (@Melchizedek6809, @shanehandley, #33312) Servo’s flexbox support continues to mature, with support for **‘align-self: normal’** (@Loirooriol, #33314), plus corrections to **cross-axis percent units** in descendants (@Loirooriol, @mrobinson, #33242), **automatic minimum sizes** (@Loirooriol, @mrobinson, #33248, #33256), **replaced flex items** (@Loirooriol, @mrobinson, #33263), **baseline alignment** (@mrobinson, @Loirooriol, #33347), and **absolute descendants** (@mrobinson, @Loirooriol, #33346). Our table layout has improved, with support for **width** and **height presentational attributes** (@Loirooriol, @mrobinson, #33405, #33425), as well as better handling of **‘border-collapse’** (@Loirooriol, #33452) and **extra <col> and <colgroup> columns** (@Loirooriol, #33451). We’ve also started working on the intrinsic sizing keywords **‘min-content’** , **‘max-content’** , **‘fit-content’** , and **‘stretch’** (@Loirooriol, @mrobinson, #33492). Before we can support them, though, we needed to land patches to calculate intrinsic sizes, including for **percent units** (@Loirooriol, @mrobinson, #33204), **aspect ratios** of replaced elements (@Loirooriol, #33240), **column flex containers** (@Loirooriol, #33299), and **‘white-space’** (@Loirooriol, #33343). We’ve also worked on our **WebGPU support** , with support for **pipeline-overridable constants** (@sagudev, #33291), and major rework to **GPUBuffer** (@sagudev, #33154) and our **canvas presentation** (@sagudev, #33387). As a result, **GPUCanvasContext** now properly supports (re)configuration and resize on **GPUCanvasContext** (@sagudev, #33521), presentation is now faster, and both are now more conformant with the spec. ## Performance and reliability __ Servo now **sends font data over shared memory** (@mrobinson, @mukilan, #33530), saving a huge amount of time over sending font data over IPC channels. We now debounce resize events for **faster window resizing** (@simonwuelker, #33297), limit **document title updates** (@simonwuelker, #33287), and use DirectWrite kerning info for **faster text shaping on Windows** (@crbrz, #33123). Servo has a new kind of **experimental profiling support** that can send profiling data to Perfetto (on all platforms) and HiTrace (on OpenHarmony) via `tracing` (@atbrakhi, @delan, #33188, #33301, #33324), and we’ve instrumented Servo with this in several places (@atbrakhi, @delan, #33189, #33417, #33436). This is in addition to Servo’s existing HTML-trace-based profiling support. We’ve also added a new `profiling` Cargo profile that builds Servo with the recommended settings for profiling (@delan, #33432). For more details on building Servo for profiling, benchmarking, and other perf-related use cases, check out our updated Building Servo chapter (@delan, book#22). ## Build times __ The first patch towards **splitting up our massive`script` crate** has landed (@sagudev, #33169), over ten years since that issue was first opened. `script` is the heart of the Servo rendering engine — it contains the HTML event loop plus all of our DOM APIs and their bindings to SpiderMonkey, and the script thread drives the page lifecycle from parsing to style to layout. `script` is also a monolith, with over 170 000 lines of hand-written Rust plus another 520 000 lines of generated Rust, and it has long dominated Servo’s build times to the point of being unwieldy, so it’s very exciting to see that we may be able to change this. Contributors to Servo can now enjoy faster **self-hosted CI runners** for our **Linux builds** (@delan, @mrobinson, #33321, #33389), cutting a typical **Linux-only build** from over half an hour to **under 8 minutes** , and a typical **T-full try job** from over an hour to **under 42 minutes**. We’ve now started exploring self-hosted macOS runners (@delan, ci-runners#3), and in the meantime we’ve landed several fixes for self-hosted build failures (@delan, @sagudev, #33283, #33308, #33315, #33373, #33471, #33596). ## Beyond the engine __ You can now **downloadthe Servo browser for Android** on servo.org (@mukilan, #33435)! servoshell now **supports gamepads by default** (@msub2, #33466), **builds for OpenHarmony** (@mukilan, #33295), and has **better navigation on Android** (@msub2, #33294). **Tabbed browsing** on desktop platforms has become a lot more polished, with visible **close and new tab buttons** (@Melchizedek6809, #33244), **key bindings for switching tabs** (@Melchizedek6809, #33319), as well as better handling of **empty tab titles** (@Melchizedek6809, @mrobinson, #33354, #33391) and the **location bar** (@webbeef, #33316). We’ve also fixed **several HiDPI bugs** in servoshell (@mukilan, #33529), as well as **keyboard input** and scrolling on Windows (@crbrz, @jdm, #33225, #33252). ## Donations __ Thanks again for your generous support! We are now receiving **4147 USD/month** (+34.7% over August) in recurring donations. This includes donations from **12 people** on LFX, but we will stop accepting donations there soon — **please move your recurring donations toGitHub or Open Collective**. Servo is also on thanks.dev, and already **eleven GitHub users** that depend on Servo are sponsoring us there. If you use Servo libraries like url, html5ever, selectors, or cssparser, signing up for thanks.dev could be a good way for you (or your employer) to give back to the community. **4147** USD/month **10000** With this money, we’ve been able to pay for our web hosting and self-hosted CI runners for Windows and Linux builds, and when the time comes, we’ll be able to afford macOS runners, perf bots, and maybe even an Outreachy intern or two! As always, use of these funds will be decided transparently in the Technical Steering Committee. For more details, head to our Sponsorship page.
servo.org
June 17, 2025 at 11:39 PM