Researchers have introduced DreamX-Creator 1.0, a system designed for joint audio-video generation that produces synchronized 2K resolution content. The system, detailed in a paper published on arXiv, uses a compact 7-billion-parameter generator. This generator processes modality-specific audio and video streams, which are then coupled through a mechanism called Gated Cross-Modal Attention. The project repository for DreamX-Creator 1.0 was initialized on September 1, 2026, with a technical report released concurrently.
DreamX-Creator 1.0 addresses a limitation in many existing video generators, which often either omit audio or synthesize it in a separate process. This separation can restrict the reciprocal modeling of visual dynamics and acoustic events. The new system's approach involves jointly denoising these specialized streams. In the initial phase of the network, audio and video are processed independently. They are then coupled in the latter half through Gated Cross-Modal Attention, which uses token- and head-wise output gates to modulate each active cross-modal attention-head output. This design enables bidirectional audio-video interaction.
The system's architecture includes a unified Audio-Video Data System. This component constructs and filters temporally coherent clips, generates structured multimodal annotations, and organizes clips into capability-oriented data pools. Training for DreamX-Creator 1.0 occurs through a Progressive Joint Training process, which consists of two audio-video pre-training stages followed by a High-Quality Finetuning stage. Further refinement is achieved through Audio-Video Reinforcement Learning, which post-trains the generator using Modality-Aware Multimodal Feedback to route video, audio, and cross-modal feedback to their respective streams.
For high-resolution output, DreamX-Creator 1.0 incorporates an Autoregressive 1-Step 2K Refinement pipeline. This pipeline adapts a bidirectional multi-step teacher into an autoregressive multi-step refiner and distills it into a student model. This process requires one denoising evaluation per temporal chunk to achieve 2K resolution output. The refinement process aims to upgrade generated video to high-quality 2K while preserving content, motion, and audio-aligned timing.
The researchers behind DreamX-Creator 1.0 intend for the release of their compact 7B generator and 2K Refiner to democratize native audio-video generation. They aim to provide an accessible foundation for further research in unified audio-video generative modeling. The system achieves synchronized audio-video generation with performance competitive with existing open-source systems. The project is licensed under the Apache License 2.0.
The development of DreamX-Creator 1.0 follows other advancements in multimodal generative models, such as DreamX-World 1.0, a general-purpose interactive world model released in June 2026. DreamX-World 1.0 focused on controllable long-horizon generation for interactive worlds, supporting camera navigation and promptable events across various visual domains. These efforts collectively demonstrate a trend toward creating more integrated and controllable generative AI systems that can handle multiple modalities simultaneously.