A Mixed Markovian Approach on Change Detection in Optical Aerial Images

This page introduces a change detection method for optical aerial images accimplished by Multi-Layer Conditional Mixed Markov

The article describing the main results appeared in IEEE Transactions on Geoscience and Remote Sensing in 2009 [1]. Results were partially presented in ICPR 2008 [2] and SPPRA 2007 [3] conferences. The model was also introduced in an invited seminar at the Dept. of Statistics, Florida State University [4]. The test sets used for the evaluation (with ground truth) are available from our benchmark site (please ask for password by e-mail). For more information please contact Csaba Benedek.

Abstract

In this work we have proposed a probabilistic model for detecting relevant changes in registered aerial image pairs taken with significant time differences. The introduced approach, called the Conditional Mixed Markov model (CXM), is a combination of a mixed Markov model and a conditionally independent random field of signals. The model integrates global intensity statistics with local correlation and contrast features. A global energy optimization process ensures simultaneously optimal local feature selection and smooth, observation consistent segmentation. Validation is given on real aerial image sets provided by the Hungarian Institute of Geodesy, Cartography and Remote Sensing and Google Earth.

Motivation

Change detection is an important precursor in several computer vision applications. However, it is difficult to define changes in situations, where the images, which we compare, were taken with significant time difference. Due to the illumination changes and altering shadow effects the appearance of corresponding territories may be much different.

Since context dependent class models can be hardly encoded in conventional MRFs, different schemas have been proposed recently to overcome this limitation. Mixed Markov model extend MRFs by admitting data-dependent links between the processing nodes, which enables configurable structures in feature integration. Since the later property proved to be crucial in our multiple feature based approach we follow the Mixed Markovian schema. As a key contribution of this work, we exploit the dynamic connections in a multilayer framework. Here the different layer-regions are considered or ignored upon local statistical estimation of the reliability of their corresponding features.

Feature extraction and model structure

In this work we have proposed a robust multi-layer Conditional MiXed Markov model (CXM) to tackle the change detection problem in remote sensing optical images. Changes are identified through complementary features: global intensity statistics and local correlation. A contrast based selection process is responsible for choosing locally the more reliable feature in the different image regions, while a smooth change map is ensured using local connectivity constraints.

We construct a mixed Markovian probabilistic model on a graph G whose structure is shown in Fig.1. We arrange the nodes of G into four layers: Sg (intensity), Sc (correlation), So (contrast) and S* (segmentation), where each layer has the same size as the image lattice (S). Sg, Sc and So are called the feature layers, and S* is the combined segmentation layer. We assign to each pixel s a unique node in each layer: e.g. sg is the node corresponding to pixel s on the layer Sg. However, instead of segmenting the layers independently, we obtain the result here by stochastic optimization of a single energy function which encapsulates all constraints of the model: spatial smoothness, optimal local feature selection and observation-consistent classification.

Fig. 1 Model structure

Experimental results

For evaluation we used three sets of optical aerial image pairs provided by the Hungarian Institute of Geodesy Cartography & Remote Sensing (FÖMI) and Google Earth. Data set SZADA contains images taken by FÖMI in 2000 and in 2005, respectively. This test set consists of seven - also manually evaluated - photo pairs, covering in aggregate 9.5km2 area at 1.5m/pixel resolution (the size of each image in the test set is 952 x 640 pixels). One image pair has been used here for training and the remaining six ones for validation. The second test set called TISZADOB includes five photo pairs from 2000 resp. 2007 (6.8km2) with similar size and quality parameters to SZADA. Finally, in the test ARCHIVE, we compared an aerial image taken by FÖMI in 1984 to a corresponding Google Earth photo from around 2007. The latter case is highly challenging, since the photo from 1984 has a poor quality, and several major differences appear due to the 23 years time difference between the two shots. In addition we have performed a few experiments with high resolution (0.5m/pixel) images as well. In those cases, already very fine differences caused change alarms, such as planting single trees, unlike in photos with 1.5m resolution. Ground truth masks have been generated manually for each image pair of the training and test sets. The following changes have been considered: new built-up regions, building operations, planting forests or individual trees (the latter only 10 at high resolution), fresh plough-lands and groundworks before building over.

Fig. 2 Change results with red obtained by different methods

References

[1] Cs. Benedek and T. Szirányi: ”Change Detection in Optical Aerial Images by a Multi-Layer Conditional Mixed Markov Model”, IEEE Transactions on Geoscience and Remote Sensing, vol. 47, no. 10, pp. 3416-3430, 2009

[2] Cs. Benedek and T. Szirányi: ”A Mixed Markov Model for Change Detection in Aerial Photos with Large Time Differences”, International Conference on Pattern Recognition (ICPR), Tampa, Florida, USA, December 8-11, 2008

[3] Cs. Benedek and T. Szirányi: ”Markovian Framework for Structural Change Detection with Application on Detecting Built-in Changes in Airborne Images", IASTED International Conference on Signal Processing, Pattern Recognition and Applications (SPPRA), ACTA, pp. 68-73, Innsbruck, Austria, February 14-16, 2007

[4] Cs. Benedek: "Change Detection in Optical Aerial Images by a Multi-Layer Conditional Mixed Markov Model", invited seminar at the Florida State University, Department of Statistics, Tallahassee, Florida, USA, 16 December 2008