Tencent releases Hy-Nomas, an open-weight interactive world model with text and camera control
Tencent has released Hy-Nomas, an open-weight interactive video world model designed to generate and progressively modify visual environments through natural-language instructions and camera trajectories.
Unlike conventional text-to-video systems that generate a fixed sequence from an initial prompt, Hy-Nomas operates across successive video segments, allowing users to introduce objects, modify scenes, change visual styles and control viewpoint movement while retaining information from previously generated content.
The release includes model weights, inference and training code, an interactive demonstration, low-memory deployment configurations and a dedicated evaluation suite. Its architecture combines a segment-autoregressive multimodal diffusion transformer with Qwen-based text conditioning, camera-aware cross-attention, compressed video representations and a persistent key-value cache.
Tencent documents a 54-block diffusion transformer with a 2,048-dimensional hidden representation, a default distilled inference configuration using eight denoising steps and video output at 720 × 1280 resolution and 24 frames per second.
The standard inference configuration uses eight-way GPU sequence parallelism, while a separate quantized implementation supports a single NVIDIA GeForce RTX 4090 through INT8 storage, FP8 decoding, bounded GPU history and CPU offloading.
The company also published Hy-Nomas-Bench with 280 cases in four categories. These releases do not establish independently verified superiority, sustained real-time generation or reliable long-horizon physical consistency.
··········
HY-NOMAS GENERATES INTERACTIVE WORLDS THROUGH SUCCESSIVE VIDEO SEGMENTS.
Hy-Nomas divides world generation into consecutive segments rather than treating an entire output as one predetermined video. Users describe an initial environment and can then introduce objects, change existing elements, modify visual styles or specify new camera movement.
Each new segment incorporates the latest instructions and information from previous generations. The continuity mechanism relies on persistent attention representations rather than reconstructing the whole world from a written summary.
The architecture is an interactive video-world generator rather than a deterministic game engine. It does not guarantee stable object states, collision handling or physically consistent simulation across arbitrarily long sessions.
........
Capability | Implementation | Significance |
Video generation | Segment-autoregressive diffusion | Incremental worlds |
Text interaction | Per-segment instructions | Dynamic scene edits |
Camera control | Explicit trajectories | Viewpoint conditioning |
History | Persistent KV cache | Context reuse |
Video representation | DCAE-V latents | Compressed processing |
Evaluation | Hy-Nomas-Bench | Interaction testing |
........
··········
THE ARCHITECTURE COMBINES AUTOREGRESSIVE GENERATION WITH LATENT DIFFUSION.
Autoregression controls progression across segments while diffusion generates each new segment in compressed video latent space. The model transforms latent noise into video conditioned on text, camera information and cached attention states.
The dense multimodal diffusion transformer contains 54 dual-stream blocks, with visual and text streams connected through joint attention. Camera cross-attention supplies additional viewpoint conditioning.
Tencent specifies 2,048 hidden dimensions, 16 attention heads of 128 dimensions each, and feed-forward expansion to 8,192. The backbone predicts latent-space flow velocity and uses an Euler ODE solver in the published inference configuration.
These dimensions alone do not establish complete pipeline parameter count or performance, which also depends on the text encoder, decoder, hardware and sampling configuration.
........
Property | Configuration |
Backbone | Dense MMDiT |
Dual-stream blocks | 54 |
Single-stream blocks | 0 |
Hidden dimension | 2,048 |
Attention heads | 16 |
Dimensions per head | 128 |
Feed-forward intermediate | 8,192 |
Latent channels | 32 |
Positional encoding | 3D RoPE |
Sampling solver | Euler ODE |
........
··········
QWEN-BASED TEXT CONDITIONING ALLOWS INSTRUCTIONS TO CHANGE BETWEEN SEGMENTS.
Hy-Nomas uses the language-model component of Qwen2.5-VL-7B-Instruct as its text encoder. It produces 3,584-dimensional representations processed by a two-layer token refiner and projected into the diffusion transformer's 2,048-dimensional hidden space.
The Qwen component represents semantic instructions, while the diffusion transformer generates visual content. The interactive web demo also uses Nomas-PE, a separate prompt-enhancement service that converts user instructions into structured prompts.
Offline inference can consume prepared prompt files directly, so Nomas-PE should not be described as an inseparable component of the generative backbone. The full web demo has infrastructure requirements beyond video inference alone.
··········
CAMERA TRAJECTORIES ARE INTEGRATED THROUGH DEDICATED CROSS-ATTENTION.
The model supports normalized 4 × 4 camera extrinsic matrices rather than relying exclusively on text descriptions of movement. Camera data is grouped over four video frames and converted into 64-dimensional inputs aligned with temporal latent compression.
Learned pose embeddings and projection layers transform these inputs for cross-attention within each dual-stream block. Visual tokens query camera representations after the video feed-forward operation.
Trajectory files contain camera-to-world transforms and intrinsic matrices in arrays named c2ws and intrinsics, corresponding to 4 × 4 and 3 × 3 matrices. They must align with the intended video duration and frame rate.
Explicit camera conditioning improves the available control interface but does not guarantee geometrical consistency during long or complex camera movement.
··········
THE PERSISTENT KV CACHE PRESERVES CONTEXT ACROSS GENERATION STEPS.
Hy-Nomas retains attention keys and values from previously generated segments so that subsequent steps can reuse historical representations rather than recomputing the entire sequence.
Attention is causal across segments, allowing the current segment to depend on prior content without using future information. Tencent describes the mechanism as a persistent or lifelong KV cache, although actual history retention depends on the inference configuration.
The standard configuration can preserve full historical cache, which increases memory and computational requirements as sessions grow. The low-memory configuration retains a bounded GPU selection and a CPU archive, with a documented default budget of 18 historical latent frames.
That budget concerns historical context, not the current segment length. Cached representations support continuity but do not guarantee perfect long-term object identity or spatial memory.
··········
DCAE-V COMPRESSION AND PIXELUMM DECODING RECONSTRUCT THE GENERATED VIDEO.
The diffusion transformer operates on DCAE-V compressed video latents rather than full-resolution RGB frames. The f32t4c32 representation uses 32 channels, spatial compression by 32 along each dimension and temporal compression by four.
A fine-tuned PixelUMM decoder reconstructs RGB video from generated latents. Tencent specifies 36 transformer layers, a 4,096-dimensional hidden representation, 32 query-attention heads and eight key-value heads.
The decoder uses 4 × 16 × 16 RGB tubelets and overlapping temporal windows, with ten latent frames per window and five frames of overlap by default.
The PixelUMM decoding process is separate from the diffusion transformer's eight denoising steps; eight-step inference does not mean that the entire pipeline performs only eight operations.
........
Component | Configuration | Function |
DCAE-V | 32 channels | Video compression |
Spatial compression | 32× per axis | Smaller representation |
Temporal compression | 4× | Fewer time tokens |
PixelUMM | 36 layers | RGB reconstruction |
Decoder hidden size | 4,096 | Internal representation |
Decoder heads | 32 query / 8 KV | Attention |
Tubelets | 4 × 16 × 16 | Reconstruction units |
Decoder window | 10 latents / 5 overlap | Continuity |
........
··········
DISTILLED INFERENCE USES EIGHT DENOISING STEPS IN THE STANDARD CONFIGURATION.
Tencent provides a distilled diffusion checkpoint for reduced sampling iterations alongside a non-distilled autoregressive DiT used for training and fine-tuning. The principal standard inference path uses eight denoising steps.
The release includes a base DiT, distilled DiT, INT8-quantized distilled checkpoint, cinematic and robotics LoRA adapters, DCAE-V, PixelUMM and prompt-enhancement components.
The robotics adapter was trained using RealSource-World robotic-arm data, while the cinematic adapter targets visual presentation. These are adaptations of compatible base checkpoints, not independent foundation models.
The low-memory INT8 loader does not support loading a separate LoRA directly. A separate Diffusers-format implementation documents a four-step example, so the eight-step setting is the standard published pipeline rather than an immutable property of every implementation.
........
Asset | Use |
Base autoregressive DiT | Training and fine-tuning |
Distilled DiT | Standard inference |
INT8 distilled DiT | Low-memory inference |
Cinematic LoRA | Visual adaptation |
Robotics LoRA | Robotics scenes |
DCAE-V | Latent representation |
PixelUMM / FP8 PixelUMM | Video decoding |
Nomas-PE | Prompt enhancement |
........
··········
STANDARD INFERENCE USES EIGHT-WAY GPU PARALLELISM.
Tencent's standard configuration uses eight-way sequence parallelism with sp_size=8 and distributed PixelUMM decoding, requiring eight GPUs within an inference group.
The default output is 720 × 1280 pixels at 24 frames per second, with eight DiT denoising steps and Euler sampling. The frame rate describes video playback, not the wall-clock rate at which the model generates frames.
Persistent historical context can increase memory use during long sessions. Tencent has not published a complete hardware-normalized performance analysis across session lengths, cache sizes and GPU types.
··········
A QUANTIZED CONFIGURATION SUPPORTS A SINGLE RTX 4090.
Tencent provides a low-memory inference configuration for one NVIDIA GeForce RTX 4090, using INT8 storage for distilled DiT weights and attention KV cache, while retaining BF16 arithmetic for DiT computations.
The decoder uses an FP8 checkpoint, FlexAttention is selected for DiT attention, and CUTLASS FP8 supports decoding. Sequence parallelism is disabled through sp_size=1.
The system offloads the text encoder, DiT and decoder to CPU memory between stages and maintains a bounded GPU history alongside a CPU archive. This reduces simultaneous GPU memory pressure but increases data-transfer costs and system RAM dependence.
Support for one RTX 4090 does not establish real-time generation or performance parity with eight GPUs. Total memory needs vary with segment length, resolution, cache and deployment choices.
........
Property | Standard | Single GPU |
Execution | Eight GPUs | RTX 4090 |
DiT weights | Distilled floating point | INT8 storage |
Arithmetic | BF16 | BF16 with INT8 storage |
Decoder | PixelUMM | FP8 PixelUMM |
History | Persistent full cache | GPU window and CPU archive |
Offloading | Not primary strategy | CPU offloading |
Parallelism | sp_size=8 | sp_size=1 |
........
··········
THE REPOSITORY PROVIDES REPRODUCIBLE INFERENCE AND TRAINING WORKFLOWS.
The official Tencent-Hunyuan/Hy-Nomas GitHub repository contains inference scripts, environment configurations, interactive demo components, data annotation utilities and LoRA fine-tuning workflows.
Tencent recommends a prepared Docker environment; manual setup includes Python 3.12 or later, CUDA 13.0, PyTorch 2.10.0 and optimized attention implementations such as FlashAttention and MagiAttention.
The documented setup begins by cloning the repository and running uv sync --extra all, then downloading model assets from tencent/Hy-Nomas using the Hugging Face CLI.
The standard inference launcher reads configs/experiment/inference.yaml and requires appropriate DiT and PixelUMM checkpoints, a prompt JSON and an output path. The RTX 4090 workflow uses a separate YAML configuration and compatible INT8 and FP8 weights.
Reproducibility depends on checkpoint layout, CUDA compatibility, attention kernels and hardware; access to code and weights does not remove deployment engineering requirements.
··········
HY-NOMAS-BENCH CONTAINS 280 INTERACTIVE EVALUATION CASES.
Tencent released Hy-Nomas-Bench with 280 test cases covering multi-turn free-form text interaction, camera navigation, interleaved text and camera control, and spatial memory.
The dataset includes structured prompts, interaction sequences, keyboard timelines and camera trajectories. It enables researchers to test whether generated worlds follow changing instructions while retaining relevant context.
A benchmark containing 280 cases is not an accuracy score. The published materials do not establish an independent leaderboard showing superiority over competitors under identical evaluation conditions.
Meaningful comparisons would require clearly defined metrics, comparable inference settings and consistent human or automated assessment.
........
Category | Evaluated capability |
Multi-turn text interaction | Changing instructions |
Camera navigation | Viewpoint trajectory following |
Interleaved text and camera | Combined control |
Spatial memory | Consistency across interactions |
........
··········
APACHE 2.0 COVERS THE RELEASED CODE AND MODEL WEIGHTS.
Tencent licenses the released code, parameters and weights under Apache License 2.0 while preserving separate licensing conditions for third-party components.
The license generally permits use, modification and redistribution, including commercial use, subject to its terms. Organizations must still review licenses of the Qwen-based encoder and other external dependencies.
The release is best described as an open-weight model accompanied by open-source implementation, rather than a fully self-contained package governed by a single license.
The Hugging Face repository listing is approximately 196 GB, and the distilled DiT checkpoint is approximately 34.2 GB. These are storage figures, not GPU VRAM requirements; downloading only the required assets can reduce local storage.
··········
INTERACTIVE WORLD GENERATION STILL FACES CONSISTENCY AND LATENCY LIMITATIONS.
The persistent KV cache does not provide a guaranteed stable three-dimensional scene graph. Objects may change appearance, disappear or become inconsistent after complex camera movement or long sequences of edits.
Hy-Nomas can generate plausible motion and includes robotics-oriented adaptation, but the release does not demonstrate deterministic rigid-body dynamics, reliable collision handling or validated physical simulation.
Eight-step distillation reduces diffusion sampling iterations but substantial transformer and decoder computation remains. CPU offloading in the single-GPU configuration trades lower VRAM use for transfer overhead.
Long-session memory growth, bounded historical context, model checkpoint compatibility and dependency versions create further operational constraints. Public evidence does not establish sustained real-time generation or universal consistency.
··········
HY-NOMAS EXPANDS THE SCOPE OF OPEN INTERACTIVE VIDEO RESEARCH.
Hy-Nomas combines changing text instructions, explicit camera trajectories, segment-autoregressive generation, cached historical attention and multiple inference configurations within a publicly inspectable pipeline.
Weights, training tools, LoRA adapters and benchmark cases make the release useful for research and adaptation, extending beyond a closed demonstration.
The system advances interactive video-world generation without establishing the persistent geometric state or deterministic physics of a conventional simulator. Its practical value will depend on independently measured improvements in controllability, long-horizon consistency, efficiency and reproducibility.
··········
FOLLOW US FOR MORE.
·····
DATA STUDIOS
·····
datastudios.org
