Dream 7B

Introducing Dream 7B, the most powerful open diffusion large language model to date. Matches/exceeds similar-sized AR models (LLaMA3, Qwen2.5). Excels at planning & offers flexible inference.

**Dream 7B** is a cutting-edge open diffusion large language model developed through a collaboration between The University of Hong Kong and Huawei Noah’s Ark Lab. It outperforms existing diffusion models and rivals or surpasses autoregressive models like LLaMA3 and Qwen2.5 in general, math, and coding tasks. Key strengths include superior planning abilities and flexible inference, enabled by its bidirectional contextual modeling. Unlike autoregressive models, Dream 7B refines sequences in parallel, allowing dynamic adjustments in generation speed and quality. Trained on 580B tokens and initialized from Qwen2.5, it leverages innovations like context-adaptive noise rescheduling for efficient learning. Dream 7B excels in tasks requiring complex reasoning, making it a promising alternative for applications like autonomous agents and long-horizon planning. The model weights are openly released to support further research.