Correlated sampling for efficient light transport simulation

Photorealistic image synthesis is essential to many applications in feature film production, video game development, architectural design, and medical imaging. Images are generated via a rendering process that simulates how light propagates and scatters in a virtual 3D scene using mathematical models and numerical solvers. This process is also known as light transport simulation.

The project seeks to develop new sampling techniques that exploit correlation in path-space to improve the performance of state-of-the-art rendering algorithms. Indeed, correlated sampling schemes take advantage of the integral natural local smoothness in light transport simulation. Moreover, these techniques often share and reuse sampling decisions on the image plane, improving the performance of rendering algorithms further. These sampling techniques are the backbone of many rendering techniques, for example, in gradient-domain rendering (Hua et al. 2019), Markov-chain Monte Carlo (Rioux-Lavoie et al. 2020), or, more recently, in differentiable rendering (Zhang et al. 2021). 

However, constructing correlated paths is difficult for an arbitrary scene. This project will study how to optimize these sampling techniques by leveraging the rich information in path-space automatically. This optimization is now made possible with the recent advance in differentiable rendering, which allows for optimizing rendering algorithms directly to minimize the variance (Weier et al. 2021; Jakob et al. 2022). The developed automatic design will extend these operators to more challenging cases, such as heterogeneous participating media (e.g., clouds), which are very important for VFX industries or medical imaging. Finally, improving the robustness of these operators will allow new development in forward and differentiable rendering. 

References:
- Hua, Binh-Son, Adrien Gruson, Victor Petitjean, Matthias Zwicker, Derek Nowrouzezahrai, Elmar Eisemann, et Toshiya Hachisuka. 2019. « A Survey on Gradient-Domain Rendering ». Computer Graphics Forum 38 (2): 455‑72. https://doi.org/10.1111/cgf.13652.
- Jakob, Wenzel, Sébastien Speierer, Nicolas Roussel, et Delio Vicini. 2022. « DR.JIT: A Just-in-Time Compiler for Differentiable Rendering ». ACM Transactions on Graphics 41 (4): 1‑19. https://doi.org/10.1145/3528223.3530099.
- Rioux-Lavoie, Damien, Joey Litalien, Adrien Gruson, Toshiya Hachisuka, et Derek Nowrouzezahrai. 2020. « Delayed Rejection Metropolis Light Transport ». ACM Transactions on Graphics 39 (3): 1‑14. https://doi.org/10.1145/3388538.
- Weier, Philippe, Marc Droske, Johannes Hanika, Andrea Weidlich, et Jiří Vorba. 2021. « Optimised Path Space Regularisation ». Computer Graphics Forum 40 (4): 139‑51. https://doi.org/10.1111/cgf.14347.
- Zhang, Cheng, Zhao Dong, Michael Doggett, et Shuang Zhao. 2021. « Antithetic Sampling for Monte Carlo Differentiable Rendering ». ACM Transactions on Graphics 40 (4): 1‑12. https://doi.org/10.1145/3450626.3459783.