3D light fields based foreground segmentation in static scenes

To address the problem of extracting foreground objects from complex scenes, a 3D light field based method for foreground segmentation of static scenes was presented. Firstly, the method relies on densely sampled 3D light field formed by stacking a sequence of images captured at different viewpoints. These viewpoints were equally spaced along a linear path so that object trace was smooth in epipolar plan image (EPI). Secondly, line segment detection (LSD) method was performed to extract edges and compute the corresponding depth in EPI, based on which a depth map of the scene was obtained through a fast local depth interpolation algorithm. Piecewise cubic Hermite interpolating polynomial (PCHIP) algorithm proved to have desirable results in this phase. Finally, the recovered dense depth information was exploited to facilitate foreground segmentation. A threshold approach was used to separate different objects in scenes. Preliminary experimental results show that our method is able to estimate the correct relative spatial relation of multiple objects and our proposed foreground segmentation method reduces over-segmentation effects existed in traditional methods based on region clustering and mathematical morphology.