2D/3D Sensor Fusion

Principal Investigator: Prof. Björn Ottersten
Funding source(s): IEE S.A., AFR/FNR
Researcher(s): Frederic Garcia, Bruno Mirbach IEE S.A.

Recently, 3D-cameras based on the time-of-flight principle have been developed to allow capturing range images of a scene. The benefit of such a 3D-camera is that it enables the robust segmentation and localization of objects in 3D-space. Objects can be classified based on their contour, thus independently on the light condition and the texture of the object, which is crucial for safety critical applications. A general drawback of time-of-flight 3D-cameras is, however, their low resolution, being approximately a factor 100 below the resolution of standard 2D-imager and limited in frame rate compared to video rate. These drawbacks limit the field of applications for 3D-cameras. An approach to overcome the limitations of the low resolution is to combine a 3D-camera with a high resolution imager and to perform a fusion of the data on software level. The core of the project will be the development and elaboration of methods for image fusion targeting on increasing the spatial resolution of the 3D image and at the same improve the depth precision of the range information. The elaboration of these methods will be done based on real data acquired by a first prototype camera to be building up using an existing 3D camera. The project both comprises the elaboration and development of entirely new mathematical concepts, as well as the implementation of the methods for real-time applications. Mathematical modelling of the camera behaviour is addressed and the development of tools for calibration.