Alibaba's Qwen team, open-sourced on Wednesday (26) the weights of Qwen3.8-Flash-Next, a multimodal Mixture of Experts (MoE) model that previews part of the architecture planned for the future Qwen4 family. The company says it has reduced training cost to about one-ninth of Qwen3.7-Plus, keeping only 6 billion active parameters per token.
The model combines 125 billion parameters in the main core with an additional 51 billion in N-gram embeddings. Despite the total size, only 6 billion are activated per token, a strategy used to reduce the amount of computation required during inference and training.
The commercial version will be offered on QwenCloud as Qwen3.8-Flash, with context of up to 1 million tokens and integrated tools. The announced price is US$ 0.16 per million input tokens and US$ 0.47 per million output tokens. The API was still marked as “coming soon” in the official publication.
New architecture targets long contexts
Qwen3.8-Flash-Next introduces changes in four main areas: attention, residual connections, embeddings, and training optimization.
One of the new features is the Qwen Sparse Attention (QSA), combined with Gated DeltaNet. Three out of every four layers use Gated DeltaNet to condense history into a fixed-size state, while the remaining layer resorts to global attention to retrieve specific information across the context.
QSA reduces this cost by grouping the sequence into small blocks and first identifying which regions are most relevant. According to tests published by Qwen, in a context of 1 million tokens, the mechanism achieved acceleration of up to 7.6 times in the prefill stage and 4.9 times during decode.
In an experimental scenario with a 90% hit rate on Prefix Cache, Qwen3.8-Flash-Next recorded 8.6 times the prefill throughput of Qwen3.7-Plus when working with 1 million tokens.
The model natively supports 262,144 context tokens and can be expanded to 1 million with YaRN, a technique used to extend the operational window of language models.
Alibaba increases capacity without activating all parameters
Another change is in the use of N-gram Embedding. Instead of querying representations only from an individual token, the architecture also considers combinations formed by the current token and its local context.
Qwen added 51 billion parameters through this mechanism. Because this information is accessed by query and can be anticipated, the data can remain in server memory and be loaded asynchronously, reducing the need to keep it permanently in GPU memory.
The architecture also introduces Gated Residual, which splits the traditional residual stream into four parallel paths. Dynamic gates control how much of each stream should be read or updated according to the processed content, with the goal of preserving information between layers and increasing training stability.
The residual state can also be stored in FP8, a lower-precision format that reduces the volume of data moved through memory during execution.
Qwen3.8-Flash beats Claude Opus 4.6 in some tests
In benchmarks published by Qwen itself, the new model surpassed Claude Opus 4.6 Max in several evaluations, though not all.
On SWE-bench Pro, focused on real software engineering tasks, Qwen3.8-Flash-Next scored 62.5 points, compared to 53.4 for Claude Opus 4.6. On SWE-bench Multilingual, the results were 81.0 and 77.5, respectively.
On CoWorkBench's extended productivity tasks, Alibaba's model achieved 73.9 points, versus 68.2 for Claude. On JobBench, focused on professional activities, the difference was 55.7 against 36.6.
Qwen also came out ahead on GPQA Diamond, with 91.7 against 91.3, and on LiveCodeBench v6, with 91.9 against 88.8. On Humanity's Last Exam, Claude Opus 4.6 recorded 40.0 points, above Qwen3.8-Flash-Next's 35.9.
In the multimodal tests released by the company, the model also showed an advantage over Claude in tasks such as Android device usage, long video comprehension, real-world image perception, and visual math problem solving.
The numbers are results presented by Qwen itself, under configurations and evaluation tools specified by the company, and do not mean the model is superior in all categories.
Project serves as a preview of Qwen4
The release of Qwen3.8-Flash-Next also serves as an early presentation of the architecture that Alibaba intends to develop in the next generation of the Qwen family.
The strategy repeats the path taken with Qwen3-Next, whose hybrid architecture was later used in the series Qwen3.5, Qwen3.6, Qwen3.7, and Qwen3.8. The team says it decided to release the changes again before the next generation to allow the community to evaluate the architecture before building the full Qwen4 family.
Qwen3.8-Flash-Next weights are being made available through Hugging Face and ModelScope. The official repository was also opened with documentation for running the model.
The version hosted on QwenCloud will be compatible with interfaces in the OpenAI and Anthropic standard. The documentation presented by the team also includes integration with programming tools and agents such as Claude Code, Codex, Qwen Code, Qoder CLI, and OpenClaw.
The release expands on the information initially presented in the base text, which highlighted cost reduction, the 6 billion activated parameters, and the model's arrival on Qwen services.
