Future of AI Is Built on 3D, Not Language Alone

Spatial Intelligence and World Models
📌 The focus has shifted from image recognition and simple video understanding to deeply perceptual, spatial intelligence, connecting to robotics, embodied AI, and ambient AI.
🗺️ World models are crucial for achieving general artificial intelligence because much human knowledge is not captured in text, requiring models to gain experience firsthand or through video.
✨ World Labs’ product, Marble, generates complex 3D spaces from the model's internal world representations.

AI Architectures and Learning Paradigms
🤖 The evolution of AI necessitates moving beyond current LLMs, which learn from a finite subset of human knowledge encoded in text, toward models that learn directly from the world via multimodal input.
⚖️ Both implicit and explicit representations are likely needed for a universal world model; explicit output (like 3D representation) is necessary for practical usefulness in industries like VFX and design.
🔄 The discussion touched upon continuous learning, contrasting it with the fixed parameters of current models, though current work leans toward batch/offline learning while remaining open to online modalities.

The Search for a Universal Task Function
💡 The success of Gen AI stems from the objective function of next token prediction in language models.
❓ For world modeling, finding an equally powerful Universal Task Function (UTF) is a profound challenge, with candidates like 3D reconstruction or next frame prediction (RTFM) being debated for their sufficiency and efficiency.
🤔 RTFM (Real-Time Frame Model), which predicts the next frame with 3D consistency, requires significant compute but allows the model to learn the structure of the world, though it treats the world as 2D inherently.

Understanding, Physics, and Future Architectures
🧠 AI "understanding" is currently semantic and functional (e.g., knowing to change a pink couch to blue), but it lacks the anthropomorphic, embodied consciousness that defines human understanding.
⚛️ Current Gen AI physics simulation relies on statistical patterns derived from observed dynamics (e.g., water movement) rather than abstract deduction of Newtonian laws, which requires deep causal abstraction.
🚀 The speaker believes architectural breakthroughs beyond the Transformer are coming, and the future goal is a model connecting perception, spatial reasoning, planning, and imagination ("seeing to doing").

Key Points & Insights
➡️ Spatial intelligence is the next frontier for AI because significant human intelligence and critical tasks (like firefighting or complex reasoning) go beyond language.
➡️ Future AI systems require multimodality in input (video, text, 3D layouts, sound, tactile data) to achieve a comprehensive, multi-sensory learning paradigm, similar to biological systems.
➡️ To reach a truly general intelligence capable of scientific discovery (like formulating special relativity), AI needs the capacity for profound causal abstraction above pure statistical pattern generation, which may require post-Transformer architectures.

📸 Video summarized with SummaryTube.com on Jan 17, 2026, 04:21 UTC