Alibaba Qwen Releases Qwen-Drive 1.0: 4B Open-Source Autonomous Driving VLM, 3D Perception, Reinforcement Learning, and Motion Planning

Qwen-Drive-1.0 is Alibaba Qwen's first open-weight driving foundation model built directly on the Qwen3.5 vision-language stack, combining 3D perception, driving-scene question answering, and motion planning inside one shared architecture rather than treating perception and trajectory generation as completely separate systems.

The release is technically more nuanced than the 4B label suggests. Qwen3.5-4B remains the shared multimodal backbone, but trajectory generation is handled by a separate Planning Expert of roughly 1 billion parameters, while an external BEV perception head probes the same learned representations for 3D object detection, semantic occupancy, and map segmentation. The original language-model decoder is kept unchanged so the system can continue performing both general and driving-specific visual question answering.

That design makes Qwen-Drive important less as a claim that a 4B model can replace a production self-driving stack and more as a test of whether one multimodal representation can support scene understanding, explicit 3D structure, reasoning, and future ego-trajectory generation at the same time. The weights and inference code are open under Apache 2.0, but the system remains a research model rather than a road-certified autonomous-driving product.

··········

QWEN-DRIVE 1.0 ARCHITECTURE: ONE VLM, MULTIPLE DRIVING HEADS.

The model keeps Qwen3.5-4B as the common representation layer and attaches specialized modules only where driving requires outputs that a normal language decoder cannot provide.

........

Component	Technical role	What it outputs	Important constraint
Qwen3.5-4B VLM	Shared multimodal backbone for all tasks.	Vision-language features, general VQA and driving VQA representations.	The 4B figure describes the shared VLM, not the complete parameter count when driving heads are attached.
BEV Perception Head	Reads shared features to expose explicit 3D scene structure.	3D object detection, semantic occupancy prediction and BEV map segmentation.	It is an external perception module, not a replacement for the vehicle's complete sensor and localization stack.
Planning Expert	Separate ~1.0B-parameter network that cross-attends to VLM key/value caches.	Future ego trajectories.	Adds model capacity beyond the 4B backbone and must be loaded for planning.
LLM Decoder	Original Qwen3.5 decoder is retained unchanged.	General and driving-specific natural-language answers.	Language output is separate from the numeric trajectory output produced by the planner.
SFT / RL planners	Two interchangeable planner checkpoints built on the same Planning Expert design.	Direct or reasoning-conditioned trajectory proposals.	The RL planner is intended for reasoning-planning mode because its reward optimization was performed on reasoning-conditioned rollouts.

........

The architectural choice is significant because the Planning Expert does not require another image encoder. It reads the internal caches already produced by the Qwen3.5 attention layers through cross-attention, allowing scene information to flow into trajectory generation without first converting the entire scene into a separate hand-designed intermediate representation.

The BEV head serves a different purpose. It makes part of the 3D structure inspectable by exposing object locations, occupancy and map geometry, which is useful for evaluating whether the shared VLM representation contains the spatial information that a driving planner needs.

··········

HOW THE TRAINING AND REINFORCEMENT-LEARNING PLANNER WORK.

Qwen-Drive uses staged training rather than forcing the backbone to learn every driving behavior in one pass. Driving-specific supervision is mixed with general-purpose vision-language data so the model can acquire road-scene competence while retaining the broader visual understanding and instruction-following abilities of Qwen3.5.

The planning branch is available in two forms. The SFT planner is imitation-trained and supports both direct trajectory generation and reasoning-based planning. The RL planner begins from the same planning architecture but is further reward-optimized on reasoning-conditioned rollouts, so its intended inference path includes an explicit reasoning step before trajectory sampling.

The official demo illustrates this with six candidate trajectories sampled for a scene. The planner returns tensors shaped as six trajectories by 50 future steps by three values — x position, y position and heading — corresponding to a five-second planning horizon at 10 Hz. That makes the output closer to a motion-planning proposal than to low-level steering, throttle or braking commands.

Reasoning is therefore being used as an input to the planning process rather than only as an explanation shown after the fact. The research question is whether a VLM that can verbally model causal road context — traffic lights, parked vehicles, intersections, right-of-way and potential conflicts — can use those same representations to improve the geometry of its future trajectory.

The team evaluates the system across open-loop, pseudo-closed-loop and closed-loop settings and reports competitive planning results, but those evaluations remain research benchmarks. They do not establish the failure rates, redundancy, latency guarantees or regulatory safety case required for unattended public-road deployment.

··········

MODEL SIZE, HARDWARE REQUIREMENTS, AND WHAT IS ACTUALLY OPEN.

The release is unusually concrete about the weights and inference footprint, which makes it possible to distinguish the shared model from the task-specific modules instead of treating Qwen-Drive as one opaque checkpoint.

........

Released asset	Approx. published size	Purpose	Deployment implication
Qwen-Drive-1.0-4B root	9.1 GB	Shared Qwen3.5-based VLM used by all modes.	Can run VQA without loading a planning head.
planner-sft	2.1 GB	Imitation-trained Planning Expert.	Supports direct and reasoning planning.
planner-rl	2.1 GB	Reward-optimized Planning Expert.	Alternative to planner-sft; intended for reasoning-conditioned planning rather than loaded as a second planner simultaneously.
perception	0.5 GB	BEV perception head.	Adds explicit 3D detection, occupancy and map outputs.
Recommended GPU	24 GB+ VRAM	Officially recommended baseline for local inference.	The model is open-weight, but practical experimentation still assumes a modern CUDA-capable GPU and a relatively heavy software stack.
Software / license	Python 3.10+, PyTorch 2.8+, Transformers 5.14.x; Apache 2.0	Inference, evaluation and modification of the released code and weights.	Users remain responsible for licenses and conditions attached to any external datasets they separately obtain.

........

The root VLM and the task heads are distributed in one model directory, which makes the modularity operational rather than merely conceptual. A developer can load the VLM alone for question answering, attach the SFT or RL planner when trajectories are required, or use the perception-specific class for BEV outputs.

The inference stack is not lightweight in the mobile-device sense. The repository recommends a GPU with at least 24 GB of memory and depends on CUDA-oriented components such as FlashAttention. The open release is therefore best understood as accessible to research labs, robotics teams and developers with workstation-class hardware rather than as an on-device automotive model ready to drop into an embedded ECU.

··········

WHY QWEN-DRIVE MATTERS — AND WHAT IT STILL DOES NOT SOLVE.

Qwen-Drive represents a broader shift in autonomous-driving research from narrowly specialized perception networks toward multimodal foundation models that can combine visual understanding, language reasoning, explicit spatial representations and planning. The attraction is architectural reuse: one backbone can potentially learn richer scene semantics than a pipeline in which every task is optimized independently and information is discarded at each interface.

That does not remove the engineering layers that make real vehicles safe. A production autonomous-driving system still needs synchronized sensors, calibration, localization, deterministic control, real-time scheduling, fault detection, degraded-mode behavior, redundancy, cybersecurity, validation across rare edge cases, and a safety case that can be audited independently. Qwen-Drive currently generates trajectories inside a research framework; it does not provide that surrounding stack.

The strongest technical contribution is therefore the separation between a reusable multimodal representation and specialized, inspectable driving heads. If the approach scales, future systems could share more of their perception and reasoning infrastructure while swapping or independently validating the modules responsible for 3D structure and motion planning.

The release also makes the parameter-count discussion more precise. Calling Qwen-Drive simply a 4B autonomous-driving model understates the planning configuration because the shared 4B VLM is joined by a separate ~1B Planning Expert. At the same time, the modular design means that additional capacity is only loaded when the task needs it, preserving a cleaner boundary between general multimodal intelligence and driving-specific trajectory generation.

For developers, the immediate value is reproducibility: open weights, inference code, demo scenes, evaluation scripts and separate planning/perception modules make the architecture inspectable in a way that closed automotive foundation models often are not. For the autonomous-driving industry, the larger question is whether this class of VLM-based planner can maintain its apparent flexibility once the benchmark environment is replaced by the latency, reliability and long-tail safety requirements of real traffic.

··········

FOLLOW US FOR MORE.

DATA STUDIOS

datastudios.org