Generalized Dynamics Generation
towards Scannable Physical World Model

PhysWM: Learning generalizable dynamics from 3D scans for physical simulation and prediction

Yichen Li1 Zhiyi Li1 Brandon Feng1 Antonio Torralba1
1MIT CSAIL
Paper arXiv Code
PhysWM Teaser

Abstract

We present PhysWM, a framework for learning generalizable physical dynamics from 3D scans of real-world objects. Our approach enables the prediction and simulation of how scanned objects will behave under various physical interactions, bridging the gap between static 3D captures and dynamic physical simulation. By learning a latent dynamics model that generalizes across object categories and material properties, PhysWM can predict physically plausible deformations, motions, and interactions without requiring object-specific training. We demonstrate that our model can be applied to novel scanned objects at test time, enabling applications in robotics, virtual reality, and physical reasoning.

Method Overview

PhysWM learns to predict physical dynamics by combining neural scene representations with learned physics priors. Given a 3D scan, our model infers latent physical properties and simulates forward dynamics under applied forces or interactions.

Method Overview

3D Scene Encoder

Encodes 3D Gaussian splat representations into latent geometry and appearance features.

Physics Prior Network

Infers material properties and physical parameters from visual observations.

Dynamics Generator

Generates future states by simulating physics-aware deformations and motions.

Generation Pipeline

1

Scan Input

3D Gaussian splat from real-world capture or reconstruction

2

Physics Inference

Estimate material properties and physical parameters

3

Dynamics Rollout

Simulate forward in time with learned dynamics

4

Render Output

Generate novel views of the dynamic scene

Key Features

Zero-Shot Generalization

Apply to novel objects without retraining, enabling instant physical simulation of new scans.

Physically Plausible

Dynamics respect physical constraints like conservation of momentum and material properties.

Real-Time Rendering

Gaussian splat representation enables fast, high-quality novel view synthesis.

Diverse Materials

Handle rigid, deformable, and articulated objects with varying material properties.

BibTeX 

@article{li2025physwm,
  title={Generalized Dynamics Generation towards Scannable Physical World Model},
  author={Li, Yichen and Li, Zhiyi and Feng, Brandon and Zhang, Dinghuai and Torralba, Antonio},
  journal={arXiv preprint arXiv:2510.15041},
  year={2025}
}