LaGSplat: Latent Lagrangian Gaussian Splatting

Preprint·2026·arXiv:2608.16324·PDF ↓·Code·Post·CEA‑List

Architecture LaGSplat : encodeur CNN, réseau lagrangien latent, décodeur Gaussian Splatting, et rappel de la force utilisateur dans l'espace latent par la jacobienne du décodeur — LaGSplat identifie la physique d'un objet à partir de quelques secondes de vidéo : un encodeur visuel projette chaque image dans un espace latent de faible dimension, un réseau de neurones lagrangien y apprend les équations du mouvement, puis un décodeur Gaussian Splatting reconstruit la scène en temps réel. Un effort extérieur peut être appliqué à tout instant, en tout point de la scène, alors que les données d'entraînement ne contiennent aucune force mesurée.

experiment 1 · synthetic

Gaussians parameterised by the latent state

Damping ζ 0.43

Density · gaussians / half-swing 10

Amplitude 100°

input

Filmed video

planar simple damped pendulum

current frame

4DGS · parameterised by t

Space-time volume

axes (x, y, t): the plane always moves to the right

drag to rotate

- Gaussians tiling
the space-time volume

LaGSplat · parameterised by q

Space-state volume

axes (x, y, q), q = angle θ: a helical manifold, the plane swings

drag to rotate

- Gaussians tiling
the space-state volume

experiment 2 · real test case

One-degree-of-freedom test case: an oscillating rocking chair

State q -

Damping γ -

background

LaGSplat · learned gaussians

Space-state volume

axes (x, y, q): 2048 gaussians, the plane is the current state

drag to rotate

reconstruction from q

Reconstructed frame

model's RGB output, sliced at q = current

grab & drag the object

Grab the object: dragging applies a force, mapped through the decoder into the latent space and added to the LNN equation.

Damping γ -

training video over

original video · one fixed camera

Vue de référence (caméra réelle) — reference view (outlined) + synthesised novel views

Vue nouvelle synthétisée, caméra virtuelle — reference view (outlined) + synthesised novel views

LaGSplat · 3D reconstruction

Interactive 3D scene

left-drag on the object to push it · right-drag to rotate · scroll to zoom

↗ Full screen

experiment 3 · real test case

Two-degree-of-freedom test case: a bag hanging from its handle

Speed ×1.0

Dissipation -

LaGSplat · latent state

Latent space trajectory

axes (q₁, q₂): the 2-D state of the dynamical system; faint curve = trajectory in the training videos

drag the dot

⟶ Latent force Jᵀf, applied in the LNN equation. Drag the dot to sweep the plane and see how the dynamical system evolves.

live GS decoder

GS decoder output

15 000 gaussians conditioned on (q₁, q₂) and re-rasterised every frame

grab & drag the object

⟶ Interactive user applied force f, pulled back to the latent space through Jᵀf. Grab the object to apply it.

experiment 4 · real test case

Four-degree-of-freedom test case: a pneumatically actuated soft arm

A two-segment soft continuum robot, trained on the public dataset of Krauss et al. (2026). The latent space here has four dimensions, so the Gaussians are parameterised by (x, y, q₁, q₂, q₃, q₄) and the state no longer fits in a plane. The arm is not only released from an initial condition, it is also driven by the four pneumatic chambers of the real robot. Pressure enters the latent Euler-Lagrange equation on the right-hand side, so the sliders below carry the same input the physical actuator receives.

ddt ∂ℒ∂q̇ + ∂𝒟∂q̇ − ∂ℒ∂q = b(q)^⊤P + J(q)^⊤f

ℒ = 𝒯 − 𝒱 and 𝒟 are the learned kinetic, potential and dissipation terms, P the four chamber pressures set by the sliders, and f a force applied by dragging in the image, transported to the latent space by the decoder Jacobian J(q).

Speed ×1.0

Chamber pressure · kPa

p₁ · segment 1 -

p₂ · segment 1 -

p₃ · segment 2 -

p₄ · segment 2 -

LaGSplat · latent state

Four latent coordinates over time

the 4-D state cannot be drawn as a plane, so each coordinate is traced separately, ordered by visibility: the share of on-screen motion each one carries. The first two carry 98 % of it between them, which is why the render stays right even when the last two drift. Shaded band: the range covered by the training videos.

live GS decoder

GS decoder output

15 000 gaussians conditioned on (q₁, q₂, q₃, q₄) and re-rasterised every frame

grab & drag the object

⟶ Interactive user applied force f, pulled back to the latent space through Jᵀf — on top of the pressure the chambers are already applying.

Cite this work

This page illustrates the LaGSplat preprint. If you use this work, please cite it as:

@misc{pottier2026lagsplat,
  title         = {LaGSplat: Inferring Physics-Governed Interactive Simulation from
                   Monocular Video Using Latent Lagrangian Gaussian Splatting},
  author        = {Pottier, Louen},
  year          = {2026},
  eprint        = {2608.16324},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2608.16324}
}