Researchers have developed a new system named Artisan that moves beyond generating static 3D models to creating articulated assets with realistic physical behaviors. This multi-agent pipeline integrates physical simulation feedback, allowing for the generation of 3D assets that exhibit collision-free motion, stability under gravity, and accurate mass and material properties. The system builds upon prior work, extending the Articraft method by introducing a feedback loop where generated assets are tested in a simulator, and any detected issues are used to refine the asset.
Artisan employs a sequence of agents. A Design Agent interprets text prompts and reference images to create a blueprint of parts, joints, and functional constraints. A Coding Agent then translates this blueprint into an asset, and a Review Agent uses physical simulation to test the asset's behavior. This iterative process aims to ensure that generated assets are not only visually representative but also functionally sound for applications in simulation, product design, and robotics.
To evaluate the capabilities of Artisan and compare it with other methods, the researchers have also constructed a unified benchmark. This benchmark assesses assets across four key aspects: spatial fidelity (how closely the asset matches reference inputs), physical plausibility (stability, mass, material properties, and collision-free motion), downstream task performance (usability in tasks like robotic manipulation), and overall quality as judged by humans. The benchmark includes 100 reference assets from ArtVIP and LightWheel, 50 stability prompts, and 50 ABO objects with reference masses.
In evaluations using this benchmark, Artisan demonstrated competitive spatial fidelity while achieving the lowest collision rates and highest stability rates among the tested methods. The researchers state that their system and benchmark provide insights for advancing 3D generation towards creating physically plausible and functionally usable assets. The project has also released the official implementation and benchmark for Artisan on GitHub, along with a dataset for the benchmark on Hugging Face.