Shadow segmentation

Markovian Approach on Foreground-Background-Shadow Segmentation of Video Scenes


We give a new model for foreground-background- shadow separation. Our method extracts the accurate silhouettes of foreground objects even if they have partly background like colors and shadows are observable on the image. It does not need any a priori information about the shapes of the objects, it assumes only they are not point-wise. The method exploits temporal statistics to characterize the background and shadow, and spatial statistics for the foreground. We improved a statistical model for shadows, which is robust regarding the forthcoming artifacts in real-world surveillance scenes, a contains an automatic parameter-update procedure, therefore, the method is able to accommodate to the illumination changes. A Markov Random Field model is used to enhance the accuracy of the separation. We validated our method on outdoor and indoor video sequences captured by the surveillance system of the university campus, and we also tested it on well-known benchmark videos.


Results were published at IEEE. Trans. Image Processing (2008) and at ACCV 2006.


shadow segmentation

Segmentation results 1st column: video image, 2nd: result of a pixel-based preliminary classifier, 3rd: pre. classifier result enhanced by morphology, 4th: proposed MRF result.


segmentation results

Different parts of the day on the sequences of an outdoor surveillance camera, segmentation results. Above left: in the morning ('am'), right: at noon, below left: in the afternoon ('pm'), right: wet weather


[1] Cs. Benedek and T. Szirányi: "Bayesian Foreground and Shadow Detection in Uncertain Frame Rate Surveillance Videos", IEEE Transactions on Image Processing, vol. 17, no. 4, pp. 608-621, 2008,

[2] Cs. Benedek and T. Szirányi: ”Markovian Framework for Foreground-Background-Shadow Segmentation of Real World Video Scenes”, Asian Conference on Computer Vision (ACCV), Lecture Notes in Computer Science, Springer, vol. LNCS 3851, pp. 898-907, Hyderabad, India, January 13-16, 2006

Links:  Csaba BenedekSzirányi Tamás


Contact: MTA-SZTAKI, Kende u. 13-17, 1111 Budapest, Hungary        
Tel: (+36) 1 279 6106    Information:  Head of dept: