Image: Concept render (AI-generated).
https://duobay.ai/insights/int8-quantization-wearable-model-accuracy-check?utm_source=bluesky&utm_medium=social&utm_campaign=card
Image: Concept render (AI-generated).
https://duobay.ai/insights/int8-quantization-wearable-model-accuracy-check?utm_source=bluesky&utm_medium=social&utm_campaign=card
Source: arXiv cs.LG
Source: arXiv cs.LG
128GBのユニファイドメモリを半分割り当てて AI の画像や映像が1分以内で生成できるのは魅力的。(自分のよわよわ環境だとMiniMax H3が30分とか出てた)
ただ100万超えかぁ・・・。(ASUSの小型PCは64GBだがこれでも81万円) […]
128GBのユニファイドメモリを半分割り当てて AI の画像や映像が1分以内で生成できるのは魅力的。(自分のよわよわ環境だとMiniMax H3が30分とか出てた)
ただ100万超えかぁ・・・。(ASUSの小型PCは64GBだがこれでも81万円) […]
for (const [i, f] of enc) {
const wt = scales[i] * f;
const offset = i * 256;
for (let k = 0; k < 256; k++) {
out[k] += wt * table[offset + k];
}
}
for (const [i, f] of enc) {
const wt = scales[i] * f;
const offset = i * 256;
for (let k = 0; k < 256; k++) {
out[k] += wt * table[offset + k];
}
}
For example, vs exact fp32 search:
int8: 97.24% recall@100
int8 + fp16/fp32 rescoring: 99.94% recall@100
For example, vs exact fp32 search:
int8: 97.24% recall@100
int8 + fp16/fp32 rescoring: 99.94% recall@100
https://github.com/Video-Rebirth/hyperflow-h3-flash
https://github.com/Video-Rebirth/hyperflow-h3-flash
techfi24.com #LLM
techfi24.com #LLM