Alibaba Cloud launches Qwen3.8-Flash-Next with new architecture and better cost efficiency

Alibaba Cloud has released the open weights of Qwen3.8-Flash-Next, a multimodal mixture-of-experts model that also serves as an early preview of the architecture behind its upcoming Qwen4 series. The company says the model represents a systematic upgrade across four core areas of its design, attention, residual connections, embedding, and optimization, aimed at boosting capability while pushing down computational cost, model capacity requirements, and training instability.

The technical changes are layered. On the attention side, the model pairs Gated DeltaNet (GDN), which compresses the history of prior context, with Qwen Sparse Attention (QSA), a compressed lightweight indexer that picks out the most important context at micro-block granularity, substantially cutting the cost of attention on long sequences. The residual stream is widened into four branches under a Gated Residual (GR) scheme, with a dynamic gate controlling reads and writes to strengthen cross-layer information flow and training stability. For embedding, a new N-gram Embedding component scales model capacity with minimal extra computation by looking up a table using local context, and because the embedding table can be offloaded to host memory and prefetched asynchronously, it overlaps with model computation. Finally, the training stack adopts the Muon optimizer, refined around orthogonalization accuracy, the division of labor between Muon and AdamW, and the splitting of fused parameters, with the scaling law refitted to fit the new architecture.

In terms of scale and efficiency, Qwen3.8-Flash-Next features a 125-billion-parameter main model supplemented by an additional 51 billion parameters in N-gram embeddings, with just 6 billion parameters activated per token. Against the previous Qwen3.7-Plus, the company says the new model delivers superior capabilities in coding and office tasks while cutting training cost to roughly one-ninth of the predecessor's. The model natively supports 262,144 tokens of context, and that window can be extended to one million tokens with YaRN.

Weights for Qwen3.8-Flash-Next are available now on Hugging Face and ModelScope, making the architecture immediately accessible to researchers and developers as Alibaba Cloud lays groundwork for the Qwen4 generation.