K2 Horizon: 375B Parameters, 512K Context, Fully Open Training Data, Code, and AI Models

The Institute of Foundation Models in Abu Dhabi has released K2 Horizon, a six-model family that spans compact on-device models through a 375-billion-parameter flagship. The technical headline is unusually broad disclosure: IFM says the Horizon program opens model weights, architecture details, training code, data or reproducible data-construction recipes, configurations, logs, evaluation material, and intermediate checkpoints across the training lifecycle. Reuters independently reported the release as an unusually comprehensive open-source effort focused on reproducibility rather than weight access alone.

The flagship, K2-Horizon-375B-A23B, is a sparse Mixture-of-Experts model with 375 billion stored parameters, roughly 23 billion active per token, and a native 512K-token context window. IFM positions the family for reasoning, mathematics, coding, tool use, and long-horizon agentic work. Those performance comparisons remain vendor-reported until broader independent testing establishes how consistently the models perform outside IFM's evaluation setup.

K2 HORIZON SPANS SIX MODEL SIZES FROM EDGE DEVICES TO LARGE-SCALE AGENTIC SYSTEMS.

The family mixes dense and sparse architectures so deployment can be matched to memory, latency, and workload requirements rather than forcing one model size across every environment.

IFM lists six main Horizon releases: 0.9B, 3.7B, 7B, 32B, 36B-A4B, and 375B-A23B. The smallest models target constrained and on-device environments, while the two sparse variants use Mixture-of-Experts routing to keep the number of active parameters per token materially below the total parameter count.

........

Model	Architecture / active scale	Primary deployment profile
K2 Horizon 0.9B	Dense, sub-1B class	Highly constrained edge devices and compact local inference
K2 Horizon 3.7B	Dense	Phones and lightweight on-device workloads
K2 Horizon 7B	Dense	Local assistants, coding, and general-purpose inference
K2 Horizon 32B	Dense	Higher-capability local or server inference
K2 Horizon MoVA 36B-A4B	Sparse model, about 4B active	Efficient server inference with larger stored capacity
K2 Horizon 375B-A23B	Sparse MoE, about 23B active	Large-scale reasoning, agents, tool use, and long-context workloads

........

For the flagship, the parameter count does not describe per-token compute by itself. Sparse routing activates only a fraction of the experts for each token, which is why the 375B-A23B label is operationally important: the model stores hundreds of billions of parameters while activating roughly 23 billion per token. Memory footprint, expert-parallel communication, KV-cache requirements, and serving topology still make the flagship a data-center-scale model rather than a conventional local LLM.

··········

THE 512K CONTEXT WINDOW AND AGENTIC DESIGN TARGET LONG-HORIZON WORKLOADS.

The flagship extends native context to 524,288 tokens and includes explicit reasoning and tool-call interfaces, but practical deployment depends on serving configuration and hardware capacity.

The official model card states that K2-Horizon-375B-A23B has native 524,288-token context from mid-training onward. That enables very large document collections, long codebases, extended tool traces, and multi-step agent histories to remain inside one model context, although maximum advertised context and efficient production context are different engineering questions.

IFM's recommended serving setup for the flagship uses reasoning-aware chat templates and tool-call parsing. Its published SGLang recipe is validated on an eight-H200 node with tensor and expert parallelism, BF16 precision, and FlashAttention-3. The example itself shows why the model's openness should not be confused with low deployment cost: the weights may be openly available while production-grade inference still requires substantial accelerator memory, interconnect bandwidth, and operational tuning.

The company also advertises day-zero support across vLLM, SGLang, and Ollama, together with deployment paths on NVIDIA, AMD, and Cerebras hardware. These compatibility statements are useful for portability, but throughput, latency, quantization behavior, and full-context memory demand should be validated against the exact runtime and hardware used in production.

··········

K2 HORIZON PUSHES OPEN AI FROM WEIGHT ACCESS TOWARD TRAINING REPRODUCIBILITY.

IFM is publishing a broader development trail than a conventional open-weight release, although some of the announced training artifacts are still being staged rather than appearing simultaneously with every final model checkpoint.

The central distinction is the scope of disclosure. IFM says Horizon opens intermediate checkpoints, training code, configurations, fine-grained logs, evaluation results, model architecture, mixture composition, and training data where licenses permit. When redistribution rights prevent direct publication of a dataset, the institute says it provides source descriptions, filtering procedures, construction methods, and mixture composition so researchers can inspect how the training corpus was assembled.

There is an important timing nuance. IFM's launch materials describe the fleet as fully open and present the training lifecycle as part of the release, while the current Hugging Face card for the 375B-A23B flagship says the final checkpoint is available and that intermediate checkpoints, data, and training code will be released. The practical level of reproducibility therefore depends on which artifacts are actually available at the moment a researcher tries to reproduce a training stage.

........

Artifact	IFM commitment	Practical implication
Final model weights	Released under Apache 2.0	Models can be downloaded, adapted, and self-hosted subject to the license
Training code and configurations	Announced as open; staged availability may vary by model	Researchers can inspect or reproduce training procedures as the corresponding artifacts land
Training data	Direct datasets where licensing allows; construction recipes where redistribution is restricted	Data provenance is more inspectable than in weight-only releases, but exact corpus reproduction may still depend on source licenses
Intermediate checkpoints	Announced across the training lifecycle	Makes it possible to study how capabilities and behavior evolve between stages
Logs and evaluations	Fine-grained training logs and evaluation results announced	Supports deeper audit of optimization behavior and benchmark methodology

........

The licensing structure also needs to be read precisely. IFM states that the models and code are under Apache 2.0, while datasets retain their applicable licenses, including licenses such as ODC-BY. A project can therefore be highly open without every underlying data asset sharing one universal license.

··········

K2 HORIZON IS MOST CONSEQUENTIAL IF THE OPEN TRAINING STACK BECOMES REPRODUCIBLE IN PRACTICE.

The long-term significance depends less on the 375B headline than on whether researchers can actually reconstruct, inspect, and modify the development process across data, training stages, checkpoints, and agentic post-training.

For developers who only need inference, K2 Horizon already offers a wide range of model scales and familiar serving frameworks. For research teams, the more unusual value is the promised visibility into how the models were built. Intermediate checkpoints can expose capability transitions, data recipes can reveal training-distribution choices, and open training infrastructure can make controlled modifications possible without rebuilding an entire stack from undocumented assumptions.

The flagship's 512K context and sparse 375B-A23B architecture also create a useful separation between openness and accessibility. A model can be legally and technically open while still being expensive to run at full scale. Smaller Horizon variants may therefore become the more practical test of whether IFM's training approach transfers into local, academic, and commercial deployments that do not have multi-H200 infrastructure.

Benchmark leadership should remain a secondary conclusion until independent evaluations reproduce IFM's results under comparable prompting, reasoning effort, context length, and tool environments. The stronger confirmed development is the release strategy itself: IFM is attempting to expose substantially more of the model-building pipeline than the industry norm, and the value of that strategy will become measurable as the remaining promised artifacts become available and third parties try to reproduce or extend the models.

FOLLOW US FOR MORE.

·····

DATA STUDIOS

·····

[datastudios.org]