Search

Publications

Search
1. Background & Motivation (Why?)
Commanding a robot to "cut the apple in half" is deceptively difficult.
•
Physical Complexity: Food deforms, fractures, and changes shape under pressure. Standard datasets for rigid bodies cannot capture these dynamics.
•
Data Scarcity: Collecting real-world data (e.g., slicing thousands of fruits) is expensive and dangerous. Previous simulations lacked physical accuracy regarding forces and friction.
•
Quantitative Grounding: Few existing models can understand and execute precise numerical instructions, such as "cut at the 30% mark from the right."
2. Key Solution: CulinaryCut & VLAP (How?)
The researchers bridged ManiSkill (Robot Simulator) and MPM (Physics Simulator) to create a safe, intelligent environment for learning food processing.

Core Components

1.
Hybrid Simulation
2.
CulinaryCut Benchmark
3.
Safety & Style Modules
3. Experimental Results
CulinaryCut-VLAP: A Vision-Language-Action-Physics Framework for Food Cutting via a Force-Aware Material Point Method
1. Background & Motivation (Why?) Traditional shape morphing techniques have suffered from a fundamental problem known as the "Rendering Gap."
•
Physics Simulation (MPM): Efficient because it uses sparse particles, but it lacks sufficient surface detail for high-quality visuals.
•
Rendering: Typically requires dense surface information to display high-fidelity images.
•
The Problem: The process connecting these two domains was non-differentiable. This meant that rendering errors couldn't be used to correct or optimize the underlying physical motion.
2. Key Idea: PhysMorph-GS (How?) The researchers proposed a new pipeline that establishes a bidirectional coupling between Differentiable MPM (Physics) and 3D Gaussian Splatting (Rendering). Core Mechanisms 1) Deformation-Aware Upsampling
•
Instead of relying solely on the sparse "Anchor Particles" from the physics engine, the system generates virtual "Child Particles" specifically for rendering.
•
It adaptively places more child particles in areas with high deformation (stretching or compressing) to preserve geometric details.
2) Physics-Based Gaussian Construction
•
The shape (covariance) of the Gaussians isn't just learned arbitrarily; it is computed deterministically based on the actual physical deformation gradient ($F$). This ensures the visual output remains physically plausible.
3) Bidirectional Optimization
•
Rendering Loss: Visual errors, such as Silhouette (Lα\mathcal{L}_{\alpha}) and Depth (Ld\mathcal{L}_{d}), are backpropagated to update particle positions and deformation controls in the physics engine.
PhysMorph-GS: Differentiable Shape Morphing via Joint Optimization of Physics and Rendering Objectives

TVCG Teaser Video:

Rendering results:
•
Sphere to Bunny & Duck to Cow:
•
D to Dragon:
•
TVCG:
•
Abstract :
A Differentiable Material Point Method Framework for Shape Morphing
•
Abstract :
Rendering results:
•
Sand particle example:
•
Snow with Car:
•
Primitive Rectangle (vs MPM):
•
Snowflakes (Neo-Hookean):
•
Mater’s Thesis: (Dissertation Defense: 11/17)
MPM-Based Angular Animation of Particles using Polar Decomposition Theory

Describe Lava’s localization with wax

•
Use Numeric Animation for Real Data in Gen AI
Lava Localization Simulation Based on the Wax

Proposal (Summary)

•
Challenge: Hard to achieve both physical accuracy and visual fidelity in digital twins.
•
Existing methods:
•
Our idea:
•
Work with;

PhaseBand-GS: Supplement Equations [pdf]

Equations.pdf
160.7 KB
Real-Time Digital Twins with Material-Aware Gaussian Splatting