AI inference infrastructure must evolve as model complexity,and accelerator constraints reshape deployment across data centers.

Darkbloom, a decentralized AI inference network that aggregates and sells spare compute capacity from Macs, told its members to expect revenues to fall as token prices continue to drop. According to...

TrendForce warns that expanding AI inference workloads are creating mounting shortages of high-capacitance MLCCs essential for DDR5 modules.

AI inference is no longer only about faster GPUs. Nvidia’s BlueField-4 STX points to a new server layer built to store, move, and reuse model context at rack scale.

分散型AI推論プラットフォーム「llm-d 1.0」がCNCFガバナンスのもとリリースされ、エージェント型ワークロードのコストを大幅削減。

How to choose an AI inference engine for production: compatibility, latency metrics, benchmark pitfalls, and where vLLM, SGLang, LMDeploy and llama.cpp fit.

#Intel’s D-series #GPU uses #480GB LPDDR5X for #AIinference, supporting models up to 300 billion parameters. Rated at 1.3 petaflops FP8 and 1.5TB/s bandwidth, it uses 350W, PCIe form factor and Xeon integration. https://www.techradar.com/pro/behold-intel-plans-a-480gb-gpu-a-video-card-with-more-memory-than-the-average-laptop-ssd-has-storage-but-of-course-its-for-ai-and-curiously-it-will-use-lppdr5x-not-even-gddr7?eicker.news #tech #news #ainews

Marvell’s new Structera X CXL expanders point to a practical shift in AI racks: more shared DRAM around CPUs and accelerators, not just more GPUs, for memory-hungry inference.

Intel’s Crescent Island is now GPU D-Series, an inference-focused accelerator built for high-memory AI workloads in familiar air-cooled PCIe servers.

Microsoft Scales Up Its Own AI Silicon Microsoft has announced a major expansion of its in-house Azure Maia AI inference systems. By 2027, the company expects to reach a striking multi-gigawatt deployment scale. The move makes Maia the main inference engine for future Microsoft products, including Copilot. It also signals a deliberate plan to reduce heavy reliance on outside GPUs, especially those from NVIDIA.

d-Matrix's Squadron platform scales to 16 racks and 1,024 Corsair XPUs, marking a move into integrated AI infrastructure.

The San Francisco startup will build an A-1 photonic inference system targeting 10,000 tokens per second on models past 20 trillion parameters, shipping in 2027.

AI racks are running into a memory problem, not just a compute problem. Astera’s Leo X aims to put DRAM closer to accelerators for larger KV caches.

ロボットAI推論のオフロードによる性能向上と開発効率化の技術報告。