0
MODEL SIGNAL · META

Llama 4 Scout

Meta’s open-weight natively multimodal Mixture-of-Experts model with an industry-leading 10M token context window. 17B active parameters with 16 experts (109B total) and designed to fit on a single NVIDIA H100 GPU.

CATEGORYMultimodal
CONTEXT10M
RELEASEDApril 5, 2025
Key Features
  • 10M token context window
  • Fits on a single NVIDIA H100 GPU
  • Open-weight release under the Llama 4 Community License
  • Mixture-of-Experts architecture: 17B active parameters, 16 experts, 109B total parameters
  • Natively multimodal (text + image) with early fusion
  • Instruction-tuned for assistant-style, multilingual text and code output

Provider announcement →

Read the Model Signal report →

MODEL SIGNAL

Llama 4 Scout: Meta's 10M Context Multimodal MoE

An open-weight, 109B total parameter Mixture-of-Experts model engineered to run natively multimodal workloads on a single NVIDIA H100 GPU.

Bottom line

Meta’s Llama 4 Scout fundamentally alters the hardware economics of massive-context AI. By pairing a 10M token context window with an efficient 17B-active Mixture-of-Experts (MoE) architecture, Meta delivers early-fusion multimodal capabilities that fit entirely on a single H100 GPU under the Llama 4 Community License.

Model profile

According to Meta's primary release channels, Llama 4 Scout is a natively multimodal (text and image) model utilizing an early-fusion architecture. It operates on an MoE foundation with 109B total parameters, utilizing 16 experts to activate only 17B parameters during inference. The model is instruction-tuned specifically for assistant-style, multilingual text and code generation, and boasts an industry-leading 10M token context window.

Signal

The core signal is the democratization of extreme context processing. Historically, 10M token windows have been the exclusive domain of proprietary, frontier-class API providers. By architecting an MoE that routes efficiently enough to constrain active parameters to 17B, Meta is packaging massive context and native multimodality into a footprint tailored for standard enterprise hardware.

The operator read here is significant: this model shifts the boundary of what can be built entirely in-house. Enterprises restricted by data residency or privacy constraints can now deploy extreme-context ingestion—capable of swallowing entire codebases, comprehensive legal dockets, or vast image sets—without relying on external endpoints.

Noise

The headline 109B total parameter count can create a false impression of the model's operational weight; the critical metric for operators is the 17B active parameter footprint that governs runtime compute and memory requirements. Additionally, while the 10M token window is a confirmed spec, operators should treat the practical latency and throughput of maximizing that context on a single H100 as an unresolved variable. Pushing 10 million tokens through any single node will likely incur substantial time-to-first-token (TTFT) costs that are not detailed in the launch posture.

Assessment & Operator implications

The emerging pattern is a deliberate move toward high-efficiency, application-specific routing (via the 16 experts) to bypass hardware bottlenecks. If the provider facts hold regarding its single-H100 design, Llama 4 Scout changes the baseline infrastructure requirements for private multimodal workloads. Teams no longer need multi-node clusters to handle massive, multi-document, cross-modal reasoning tasks.

Where it fits

Llama 4 Scout is explicitly positioned for environments that require both hardware efficiency and massive data ingestion. It fits as a direct replacement for complex Retrieval-Augmented Generation (RAG) pipelines, allowing operators to simply load entire knowledge bases directly into the context window. It is also an ideal fit for on-premise multimodal processing, such as analyzing interleaved text and imagery in lengthy financial reports or medical records, where data gravity prevents sending the information to an external cloud API.

Model Signal · Signal + Noise · Isaiah Steinfeld