Idiopathic sudden sensorineural hearing loss (ISSNHL) is defined as a loss of at least 30 dB across three consecutive frequencies within 72 hours. It affects 5 to 27 per 100,000 people, and recovery rates range from 32% to 65% [13]. Because its causes and recovery course are uncertain, prognosis is difficult and clinical management is often planned with limited individualized guidance. Machine learning has been widely explored for clinical diagnosis and outcome prediction [1], and ISSNHL-specific studies have applied ML models to predict hearing outcomes and recovery categories [2,9,11,17,19].

**Authors:** Siddharth Pritam Existing link-based combinatorial preprocessing methods speed up the computation of persistent homology by removing a vertex or edge only when its link remains a cone. We replace this condition with a quantitative homological certificate. The reduced homology of the filtered link of a generator (a vertex or edge) defines a local obstruction module whose future part describes the effect of deleting that generator. Its barcode certifies either exact deletion or an explicit bound on the bottleneck error, and a conflict colouring extends this guarantee to families of generators. Our implementation, HomTrim, removes an additional 13% to 41% of the input edges beyond domination-only preprocessing and reduces backend persistence time by factors ranging from 1.55 to 4.61 on weighted flag filtrations. The same module also yields regularity diagrams that measure how far a generator can move before becoming visible to homology, together with a multiscale stability result.

Snorkel AI's data architecture provides a platform to meet the increasing demand for AI training data, enabling companies to efficiently generate and manage training data. It integrates data collection, preprocessing, labeling, and quality evaluation processes, allowing for rapid construction of custom training data. By following the setup procedures outlined in the official documentation, companies can introduce Snorkel AI's data architecture to improve AI model training efficiency.

This tutorial explores the pre-processing and processing of images obtained from ZWO Seestar. Pre-processing will be carried out using a dedicated script. Preprocessing 1. Do...

YouTube video by Deep Space Astro

_Originally published at cchinchilla.dev. Part 1 of From code to weights, a 12-part series on ML fundamentals for engineers._ A language model has never seen a letter. It sees integers. Somewhere between your string and the first matrix multiply there's a function that turns text into a list of ids, and that function was frozen the day the model was trained. It isn't preprocessing. It's part of the artifact. Swap it and the weights are still there, but the model you had is gone. I came to this from the compiler side, where the parser is something you own and can change on a Tuesday. Here you own it exactly once, before training, and never again. This is the first post of a series for engineers moving into ML engineering, from software or from data science. Each part names one thing that does not behave the way software behaves. This one is the parser: it ships inside the model. ## A lexer with a different goal If you've written a compiler, you've written a tokenizer. The lexer takes characters and emits a stream of symbols: identifiers, keywords, literals. Its goal is to recognise structure so the parser can build a tree. A model's tokenizer has the same shape and a different goal. There's no structure to recognise. The goal is to compress: turn a string into as few ids as possible from a fixed vocabulary, so the model sees fewer positions and the vocabulary stays small enough to put a row of the embedding matrix behind every id. Both extremes fail. Characters give a tiny vocabulary and sequences that are far too long; every position costs attention, and attention costs the square of the length. Whole words give short sequences and an unbounded vocabulary, plus a hole for every word that isn't in it. The data scientist knows this hole as out-of-vocabulary; the software engineer knows it as an unhandled case. Subwords are the compromise. Sennrich, Haddow and Birch (2016) took byte-pair encoding, a compression algorithm from 1994, and pointed it at this problem instead: learn the sub-word units from the corpus rather than declaring them up front. ## BPE is one loop I read tiktoken's `_educational.py`. Its encoder is a readable stand-in for the Rust core, which I checked by running both over every sample below and comparing ids. Its trainer is a stand-in for nothing: the Rust core ships no trainer, so `bpe_train` (`_educational.py:119-185`) is the only version of the loop in the package. It is one loop. The body of `bpe_train`: while len(ranks) < vocab_size: stats = collections.Counter() for piece in words: for pair in zip(piece[:-1], piece[1:]): stats[pair] += 1 most_common_pair = max(stats, key=lambda x: stats[x]) token_bytes = most_common_pair[0] + most_common_pair[1] token = len(ranks) # Add the new token! ranks[token_bytes] = token # ... then replace that pair through the corpus, and loop Start with 256 tokens, one per byte. Count every adjacent pair across the corpus. Take the most frequent pair, give it the next id, replace it through the corpus left to right. Repeat until the vocabulary is the size you asked for. The vocabulary is the list of merges, in order. That order is the whole model of the language. Encoding replays it: split the input into bytes, find the adjacent pair with the lowest merge rank, merge it, repeat until nothing merges. `bpe_encode` (`_educational.py:83-116`) is thirty-four lines. It never iterates the merge list; training takes the most frequent pair and encoding takes the lowest-ranked one, which comes to the same thing. > **The original has an interactive widget that replays the merge loop on a toy corpus, one merge per click.** Open it on cchinchilla.dev to use it. The widget runs that loop on a toy corpus, one merge per click. Twelve of its eighteen steps are ties, so the tie-break rule (first pair seen wins) is doing real work. Two details matter for the engineer. First, it works on bytes, not characters, so any input encodes and there's no unknown token. A character the merges never saw costs up to one token per UTF-8 byte instead of raising. Under GPT-2, `ß` is one token because its two bytes merged during training, `漢` is three, one per byte, and a Gothic letter is four. Newer vocabularies bought some of that back; `o200k_base` has `漢` whole. The Gothic letter still costs four everywhere. Second, before BPE runs there is a regex that splits the text into chunks, and merges never cross chunk boundaries. GPT-2's pattern keeps a leading space attached to the word that follows it, so `" token"` and `"token"` are different tokens. That's why the model treats the start of a line differently from the middle of one, and why a stray space in a prompt template changes the output. ## The vocabulary is the training corpus GPT-2's vocabulary has 50,257 entries. That number isn't round because it's a sum: 256 byte tokens, 50,000 learned merges, and one special token, `<|endoftext|>`, id 50256. Radford et al. (2019) describe the byte-level design those numbers come out of. The two newer OpenAI encodings I checked, `cl100k_base` and `o200k_base`, are 100,277 and 200,019. The merges encode the training corpus. GPT-2 was trained mostly on English web text, so English words are cheap: `" tokenizer"` is two tokens. The same concept in Spanish, `" tokenizador"`, is three. `" desvanecimiento"` is six. A German word with an umlaut, `" Wörter"`, is three; one of those three is the two bytes of `ö` welded to the `r` after it. The tokenizer has no notion of a character; it has bytes and frequencies. Numbers are the case that surprises engineers. GPT-2 tokenizes `1234567` as `123`, `45`, `67`, because that's what the merge ranks happen to produce. `cl100k_base` changed the regex to cap digit runs at three, which bounds the damage without fixing it: the cap groups left to right, so `1000` becomes `100` and `0`. That's a tokenizer decision with a direct effect on what the model can compute, and it was made in a regex. Be fair about why. GPT-2's merges were learned from frequency alone, and in web text `1000` is far more common than `1`, `0`, `0`, `0`; a merge table that respects frequency will swallow it whole. Nobody chose to break arithmetic. The cost only became visible once people started asking these models to add. ## The same paragraph costs more in Spanish I took one paragraph of 184 bytes, wrote it in English and in Spanish, and built a JSON payload of roughly the same size, 182 bytes. Then I encoded all three with the three tokenizers. Spanish costs about 1.7× English under GPT-2 and still 1.2× under the newest encoding. JSON costs about 2.3× under all three: braces, quotes, and keys do not compress. The 26 quotes in that payload never appear as a token on their own; each one is welded to the brace, colon, or space beside it, in `"`, `":`, `",`, `{"`. Twenty-six quotes, twenty-six tokens carrying one. > **The original has an interactive widget where you switch the sample text and the encoding.** Open it on cchinchilla.dev to use it. Switch the sample and the encoding. Newer vocabularies mostly close the Spanish gap; the JSON one does not move. Three things follow, and all three are billing or capacity problems, not modelling problems. A context window quoted in tokens holds less Spanish than English, and much less JSON. A per-token price is a per-language price. And a prompt template that wraps every request in structured metadata is paying the JSON rate on every call. ## Nothing in the type system says so In software, the parser is code and the model is data. You can change the parser, rebuild, and read the same data. Here the parser is coupled to the weights. The embedding matrix has one row per id, and the meaning of row 11,241 is `" token"` only because that's what GPT-2's merges said when the model was trained. Under `cl100k_base` the same row is `Seg`. Encode the same text with a different tokenizer and you get ids that index the same matrix and mean something else. Whether that is loud or silent is luck. The same English paragraph under `o200k_base` reaches id 99,665, which a 50,257-row matrix refuses. Go the other way, GPT-2 ids into a model with a larger vocabulary, and every id lands. Nothing throws. Even the refusal is softer than it looks: production models pad the embedding to a multiple of 64 or 128 for tensor-core alignment, so the matrix is wider than the vocabulary, and on a GPU an out-of-range index surfaces as an asynchronous assert at some unrelated line rather than where it happened. This is what makes the tokenizer part of the artifact. In GPT-2 that same matrix is also the output head, so a tokenizer swap corrupts the lookup on the way in and the sampling distribution on the way out, from one array of weights. The model file and the tokenizer file are two halves of one thing, and the failure mode when they drift is silent: the model produces fluent text about the wrong input. ## What the ML engineer watches Four things to check before any of this reaches production. **Tokenizer and model versioned together.** Store the tokenizer's hash next to the weights and check it at load time. A mismatch that doesn't fail on load is the worst kind. **Cost per language and per domain.** Measure it on your real traffic before you estimate a bill or promise a context length. The 8k window is 8k English tokens; for the JSON payload above it is closer to 3.5k of the same content. **Special tokens.** `tiktoken.encode` raises by default if the text contains something that looks like `<|endoftext|>`, precisely because a user could type it. That default is a security decision; check what your serving layer does with it, and where your chat template inserts its own. **Length in tokens, not characters.** Every limit the model has is in tokens. Truncation, context budgeting, and rate limits all need the tokenizer in the request path, not a character count. None of this is modelling work. It's the part of the model that behaves most like ordinary software, which is exactly why it ships without review: a function that turns strings into integers looks like preprocessing right up until the moment it isn't. _Part 1 of twelve. This part starts before the model exists: the tokenizer ships inside the artifact rather than in front of it, and nothing in the type system says so. A token id is an index; the next part reads what sits at that index, the embedding matrix, which is a lookup table and not much else._ _Written with AI assistance. See AI policy._