A scene simulated by LaGSplat. Dragging inside the video applies a force to the object, and every frame is generated in real time under that force.
LaGSplat: Physics-Governed Interactive Simulation from Monocular Video
- World Models
- Gaussian Splatting
- PyTorch
LaGSplat infers the physics of an object from a few seconds of monocular video. A visual encoder projects each frame into a low-dimensional latent space in which a Lagrangian Neural Network learns the equations of motion, and a Gaussian Splatting decoder renders the scene in real time. LaGSplat can extrapolate in time, replay the video from different initial conditions or even apply an interactive force to the scene, even though no force data was ever seen during training. Patent filed, as sole inventor. Project carried out as part of my work at CEA-List.