Dissertation: Synergy of physics and learning-based models in computational imaging and display

Date:

Title: Synergy of physics and learning-based models in computational imaging and display
Abstract: Computational imaging (CI) is a class of imaging systems that optimize both the opto-electronic hardware and computing software to achieve task-specific improvements. Machine/deep learning models have proven effective in drawing statistical priors from adequate datasets. Yet when designing computational models for CI problems, physics-based models derived from the image formation process (IFP) can be well incorporated into learning-based architectures. In this talk, I will propose a group of synergistic models (synergy between physics-based and learning-based models) and apply such models in several CI tasks. The core idea is to derive differentiable models to approximate the IFP, enabling automatic differentiation and integration into learning-based models. Tasks covered in this talk include privacy-preserving action recognition (using coded aperture cameras), high frame-rate video frame synthesis (using event cameras) and the simulation of 3D holographic display (based on volume holograms).
[YouTube]