Abstract
The primary contribution of this work is the first end-to-end framework for optimizing internal physics controls directly from image-space loss.
•
Beyond Initial State Optimization: We move from optimizing what a simulation starts with to optimizing how it behaves throughout its entire duration.
•
Appearance-Driven Physics Control: We demonstrate a powerful new paradigm where a high-level artistic goal (a target image) can be used to derive low-level physical controls (dFc) automatically
Technical Pipeline
Summary:
CompGraph → x_low, F_low
→ (1) Soft surface detection (PCA sheetness, percentile+EMA+hysteresis → sigmoid p)
→ (2) Build a density map
→ (3) Top-K Gumbel-Softmax Upsampling
→ (4) Taubin smoothing
→ (5) Normal smoothing
→ (6) Covariance consturction
Our framework establishes a seamless, fully differentiable loop between a C++ physics core and a Python-based rendering and optimization frontend.
1.
Forward Physics Pass (C++):
•
The DiffMPM solver runs a forward simulation based on the current control sequence (dFc).
•
This produces the state of every particle for each frame, including its position (x) and its deformation gradient (F).
2.
State Transfer (C++ → Python):
•
Using pybind11, the final particle states (x and F) are efficiently transferred to Python as NumPy arrays.
3.
Surface synthesis (From Sparse Volemetric Physics Sim to Dense Surface Physics Sim)
•
soft surface gating (PCA sheetness + percentile + EMA + hysteresis) → tangent‑plane reparam sampling → (ours) differentiable density‑equalization relax → ED smoothing of → kernel interpolation → covariance (Top: Initial mesh, Bottom Target mesh)
◦
Current, voxel occupacy is close to 5 ~ 6%
4.
Render with 3DGS and make the pipeline differentiable
•
Currently, finished implementing basic renders
