Generating stable placements with generative artificial intelligence

When a robot sets an object down among others, the pose it chooses decides whether the arrangement holds or collapses. Finding such a pose is called placement planning, and it is traditionally done by trial and error: sample a candidate pose, check whether it is stable, and start over until one passes. Because a very small proportion of all poses are stable, this is slow. This project turns the approach around and has a model generate the pose directly. Diffusion models and flow matching naturally produce several valid answers from a description of a scene, which matches the problem well, since most scenes admit many acceptable placements. Generation is guided by physics: at every step it is steered by a differentiable measure of stability, so that the proposed placements actually hold.

Applications range from a domestic robot tidying everyday objects to building mixed pallets, laying out tools and parts on a workbench, and automatically generating realistic training scenes for other learning systems.

The work runs on GPU workstations and is validated on a Franka Research 3 arm fitted with a depth camera, using a set of 3D-printed and machined objects whose geometry and mass are known exactly.

How to apply

To apply, send a complete application for the NSERC Canada Graduate Scholarship (CGS) Doctoral Program (written in English) to Prof. Philippe Nadeau. The CGS is the primary funding mechanism for PhD students in Canada, and competition for it is fierce. Preparing a full application up front may look like extra work, but it pays off: you'll arrive with a strong first draft of your CGS application well ahead of the deadline, improving your odds of being funded by the time you join the group.

Note: If you are considering applying for a Master's with thesis, the same steps apply but we will be using your application package to apply for the appropriate funding sources.