Search

Publications

Search

Proposal (Summary)

•
Challenge: Hard to achieve both physical accuracy and visual fidelity in digital twins.
•
Existing methods:
•
Our idea:
•
Work with;

PhaseBand-GS: Supplement Equations [pdf]

Equations.pdf
160.7 KB
Real-Time Digital Twins with Material-Aware Gaussian Splatting

Abstract

We propose PhysMorph‑GS, an end‑to‑end framework that optimizes internal physics controls directly from image‑space losses.
•
Beyond initial-state tuning. Instead of only optimizing how a simulation starts, we optimize how it evolves over time by learning a sequence of control deformation gradients F~p\tilde{F}_pF~p.
•
Appearance‑driven physics control. A high‑level visual target (silhouette + depth of a desired shape) is converted into low‑level physical controls, aligning the MPM deformation field with the rendered images while preserving mass and material behavior.
•
Bidirectional physics–rendering bridge. Differentiable MPM and 3D Gaussian Splatting (3DGS) are coupled through a deformation‑aware upsampling bridge so that pixel‑space gradients flow back to both particle positions and deformation gradients.

Technical Pipeline

High‑level summary

We build a fully differentiable loop:MPM (C++) → sparse physics state (x_low, F_low) → deformation‑aware upsampling & F‑smoothing → Gaussian covariance construction (μ, Σ) → 3DGS rendering + image losses → gradients back to controls F
More concretely:
1.
Forward physics pass (C++ / DiffMPM)
2.
State transfer (C++ → Python)
PhysMorph-GS: Differentiable Shape Morphing via Joint Optimization of Physics and Rendering Objectives

Isaac Sim–generated datasets meet an MLS‑MPM/CPIC cutting engine for physically grounded learning.

•
Current result:
•
Mesh → SDF Pipeline
•
Colliders (Knife/Board)
•
Elasticity/Plasticity
•
MPM Kernels
•
Rendering/Overlay
•
CLI / Output
SDF Demos:
Demo video (MLS-MPM/CPIC for the Robot Cutting Sim) :
CulinaryCut-VLAP: A Vision–Language–Action–Physics Framework for Food Cutting via a Force-Aware Material Point Method

Describe Lava’s localization with wax

•
Use Numeric Animation for Real Data in Gen AI
Lava Localization Simulation Based on the Wax

TVCG Teaser Video:

Rendering results:
•
Sphere to Bunny & Duck to Cow:
•
D to Dragon:
•
TVCG:
•
Abstract :
A Differentiable Material Point Method Framework for Shape Morphing
•
Abstract :
Rendering results:
•
Sand particle example:
•
Snow with Car:
•
Primitive Rectangle (vs MPM):
•
Snowflakes (Neo-Hookean):
•
Mater’s Thesis: (Dissertation Defense: 11/17)
MPM-Based Angular Animation of Particles using Polar Decomposition Theory