User:Danzeru

Contextual image classification is a topic of pattern recognition within computer vision, it is an approach of classification based on contextual information extracted from images. The approach focuses on the relationship of every pixel with those neighbouring it and uses this information to better the classification results of image analysis.

Theory
This form of image classification encompasses image segmentation techniques that work towards being effective despite noisy data and/or if the spectral properties of an image cannot reliably determine the class of each pixel. Noise or substantial variations in in-class pixel properties often result in a phenomenon where small (often one pixel) regions are misclassified.

Several approaches can be applied in order to avert the aforementioned common misclassification in pixel labelling. These approaches all use contextual information to some extent to improve on these prevailing problems. Some of these techniques are explored in "Image Processing, Analysis and Machine Vision"

Post-processing applications
This filtering aims to eliminate small or single-pixel regions. This is done by extracting the most probable label from the local neighbourhood, this label is then assigned to the region in question. This approach is very successful whenever these small misclassified regions are caused by noise. However, there are many instances when these small regions can result from true regions (correctly classified regions) with different properties in the original multi-spectral image, and in this case such filtering would worsen labelling results. These post-processing filters are often used in remote sensing applications as sensors are notorious for requiring noise filtration when seeking accurate results and this method provides a good fix.
 * Post-processing filter to a labelled image:

This method consists of taking pixel labels resulting from pixel classification in a given neighbourhood; with these labels it then forms a new feature vector for each pixel, and a second-stage classifier based on the new feature vectors assigns final pixel labels. Note that the contextual information must be included in the labelling process of the second-stage classifier learning.
 * Post-processing classification improvement:

From a pixel neighbourhood, mean values, variances, texture description, etc. can be extracted. This statistical information can then be supplied to (or indeed replace) original spectral data to further improve classification results. This approach is often utilised in textured image recognition.
 * Acquiring pixel feature descriptions from a pixel neighbourhood:

Combining spectral and spatial information
This approach modifies the classification process to take into account both spectral and spatial information. This is accomplished by assigning a label to an image pixel depending not only on multi-spectral grey level properties of each particular pixel but also the context of the pixel's neighbourhood.

Generally speaking, contextual classification of an image is based on the Bayes minimum error classifier. For each pixel x0, a vector consisting of (sometimes multi-spectral) values f(xi) of pixels in a specified neighbourhood N(x0) is used as a feature representation of the pixel x0.

Each pixel is represented by the vector:

ξ = (f(x0), f(x1), ..., f(xk))

xi ε N(x0)

i = 0, ..., k

Let labels of pixels in neighbourhood N(x0) be represented by the vector:

η = (θ0, θ1, ..., θk)

θi ε {ω1, ω2, ..., ωR}

θs denotes the assigned label.

Now, let the neighbouring labels excluding x0 be represented by the vector:

ή = (θ1, θ2, ..., θk)

Theoretically, there may not be a limitation to the neighbourhood size, however the majority of contextual information is believed to be present in a small neighbourhood of the pixel x0. Therefore, a 3 x 3 neighbourhood in 4-connectivity or in 8-connectivity is usually considered appropriate. Another justification for this approach is resources: computational demands increase exponentially with growth of neighbourhood size.

For a minimum error classification method, a pixel x0 is assigned to a class θr whenever the likelihood of x0 being of the class θr is the highest of all classification probabilities:

θ0 = ωr     if      P(ωr|ξ) = maxundefined P(ωs|ξ)

The posterior probability P(θs|xi) is then computed using Bayes' theorem.

Algorithm
Conclusively, here is the algorithm describing the aforementioned approach:

Algorithm: Contextual image classification
 * 1) For each pixel, determine its feature vector ξ
 * 2) Using data from the training set, determine parameters of probability distributions P(ξ|ωs) and P(ωs).
 * 3) Calculate maximum posterior probabilities P(ωr|ξ) and subsequently label all pixels in image.

Limitations
A primary consideration for contextual classification is computational resources. Computational demands rise exponentially with the processing of increasing neighbourhood size for each pixel. Recursion is a good method for overcoming this hindrance.

Recursive contextual classification
The primary reasoning behind this method is the propagating of contextual information throughout the image while maintaining computation within small neighbourhoods. Note that spectral information is also used in classification.

Context from a different neighbourhood can be communicated via the classifying θ0 of pixel x0. The vector of labels in neighbourhoods work to further ameliorate the contextual representation.

As already outlined, if the spectral data in a pixel's neighbourhood is not reliable, this leads to an ambiguous classification (the pixel may be given a number of different labels). The labelling information in nearby neighbourhoods may increase the likelihood and confidence of one of these classes. Therefore, if the surrounding pixels are labelled as members of class θi, the probability that the pixel in question should also be a member of class θi increases.

However, more often than not, complex images provide more challenging noisy data and for this decision rule is used:

θ0 = ωr     if      P(ωr|ξ, ή) = maxundefined P(ωs|ξ, ή)

After several applications of Bayes' theorem, the rule becomes:

θ0 = ωr     if      p(ξ, ήr) * P(ωr|ξ, ή) = maxundefined p(ξ, ήs) * P(ωs|ξ, ή)

Assuming all necessary probability distribution parameters were successfully determined, the recursive contextual classification algorithm is as such:

Algorithm: Recursive contextual image classification
 * 1) Using the non-contextual classification scheme, determine an initial image pixel labelling.
 * 2) Update labels in each image pixel x0, applying the current label vectors η, ή, and local spectral vector ε to the decision rule.
 * 3) If the labels of all pixels in the image are stable then end the algorithm, else repeat step 2.