Improving the potential of pixel-based supervised classification in the absence of quality ground truth data

The accuracy of classified results is often measured in comparison with reference or “ground truth” information. However, in inaccessible or remote natural areas, sufficient ground truth data may not be cost-effectively acquirable. In such cases investigative measures towards the optimisation of the classification process may be required. The goal of this paper was to describe the impact of various parameters when applying a supervised Maximum Likelihood Classifier (MLC) to SPOT 5 image analysis in a remote savanna biome. Pair separation indicators and probability thresholds were used to analyse the effect of training area size and heterogeneity as well as band combinations and the use of vegetation indices. It was found that adding probability thresholds to the classification may provide a measure of suitability regarding training area characteristics and band combinations. The analysis illustrated that finding a balance between training area size and heterogeneity may be fundamental to achieving an optimum classified result. Furthermore, results indicated that the addition of vegetation index values introduced as additional image bands could potentially improve classified products and that threshold outcomes could be used to illustrate confidence levels when mapping classified results.


Introduction
The use of remotely sensed data for vegetation studies has been described extensively in scientific literature (Kar and Kelkar, 2013, Turner et al., 2003, Woodcock and Strahler, 1987).One of the main advantages of remotely sensed information is most likely the fact that data covering extensive areas on the earth can be acquired quickly and repeatedly, while otherwise inaccessible areas may also be viewed and analysed (Liu et al., 2007).However, in inaccessible or remote areas, sufficient ground truth data may be absent or costly to acquire.In such cases investigative measures to optimise the classification process may be required.The goal of this paper is to investigate and describe the impact of various parameters when applying a supervised Maximum Likelihood Classifier (MLC) to SPOT 5 image analysis in a remote savanna biome.
Numerous land cover classifications using either pixel-or object based techniques or a combination thereof have been described in literature.Regularly the two techniques have been compared (Dingle Robertson and King, 2011, Johansen et al., 2010, Martinfar et al., 2007, Ouyang et al., 2011, Whiteside and Ahmad, 2005).Results of these comparisons are varying and ultimately dependant on a plethora of research specific variables including, but not limited to, the image characteristics, the features of the study area, the type of land cover to be extracted, the algorithms used and the level of generalization that could be tolerated by the research objective(s).Often, object-based approaches are described as being more robust and accurate when applied to high-resolution images while pixel-based may be best for certain land-cover categories (Aguirre-Gutiérrez et al., 2012).
Examples of standard supervised pixel-based classifiers are parallelepiped, minimum distance, maximum likelihood, and mahalanobis distance (Adams and Gillespie, 2006, Lillesand et al., 2004, Xiang et al., 2005).The Maximum Likelihood Classifier (MLC) incorporates the variance and covariance of the class signatures to consistently allocate each image pixel to the class with which it has the highest calculated probability of class membership (Adams andGillespie, 2006, Aguirre-Gutiérrez et al., 2012).Any training sample should be representative of the corresponding class that is to be identified.The selection of training samples depends largely upon the analyst and an understanding of the data, the study area and the classes to be extracted.
In addition, vegetation indices are often applied when investigating temporal dynamics and phenology of vegetation (Beck et al., 2007).The process of photosynthesis in green plants is key to the development of most vegetation indices.Chlorophyll in green plants absorbs strongly at red (R) wavelengths and the cellular leaf structure is highly reflective in the near infrared (NIR) (Adams and Gillespie, 2006).Because the R and NIR radiations can be measured by multispectral sensors they form the basis of most vegetation indices.Probably the most renowned vegetation index is the Normalized Difference Vegetation Index (NDVI) which varies in a fixed interval between -1 and 1.
NDVI is derived as follows: High NDVI values are usually associated with thick healthy green vegetation and low NDVI values are then associated with fractional vegetation cover or a non-green/non-vegetation cover like exposed bare soil or water.This index has also been applied in the estimation of the cumulative effect of rainfall on vegetation -this includes studies on carrying capacity of rangelands and potential crop yields (Flynn, 2006, Kawamura et al., 2005, Wardlow and Egbert, 2008, Wiegand et al., 2008).In a comprehensive account of the sources of variability in canopy reflectance, Ollinger (2011) comments on the fact that NDVI has been applied in more than 2500 studies to identify a wide range of plant traits using a relatively small number of spectral features.
In areas with little vegetation cover, the influence of soil noise is naturally more significant and NDVI in particular are strongly affected by these soil properties (Baret and Guyot, 1991).Variations in texture, colour, composition and moisture content of soils, will influence its reflectance spectra.This resulted in various attempts to compute indices that may reduce the effect of soil noise.Using a constant soil-adjustment factor (L) to account for soil background variations, a Soil Adjusted Vegetation Index (SAVI) was introduced by Huete (1988).Huete found that a constant factor of L = 0.5 was able to reduce soil noise across various vegetation densities.
Several modifications of the SAVI index have been documented (Table 1) but none of them seems to be as widely used as the NDVI index (Bagheri et al., 2012, Hashim et al., 2014, Kalbi et al., 2013, Liu et al., 2007).Applying these indices may inevitably reduce the effect of soil noise at the cost of the diminishing the dynamic range of the index and therefore it may be less sensitive to vegetation changes than NDVI (Huete, 1988, Ray, 1994).In the study area for this analysis, animal and anthropogenic pressures resulted in the occurrence of extensive areas with little or no vegetation cover across diverse soil conditions.It was decided to incorporate a modified soil index, the Second Modified Soil Adjusted Vegetation Index (MSAVI 2 in Table 1), in the investigation because it did not require an assumption on what would be a suitable constant or soil line (Qi et al., 1994, Ray, 1994).
It was anticipated that derived indices like NDVI and MSAVI 2 may be helpful in distinguishing between vegetation structural types which may be spectrally similar but vary with regards to seasonal changes.In this study, the effect of adding these indices as additional bands to the classification process was evaluated.

Name
Equation Characteristics

Soil Adjusted
Vegetation Index [2] The adjustment factor L was found by trial and error.The soil line is assumed as 1 and intercept as 0. Range: -1 to +1 Where: L = adjustment factor between 0 for very high vegetation cover and 1 for very low vegetation cover.The value typically used is 0.5.

Transformed Soil Adjusted
Vegetation Index The correction factor (L) is based on the product of NDVI and a weighted difference vegetation index (WDVI) (Qi et al., 1994).
[5] The need to pre-calculate WDVI and NDVI and find the soil line is removed.
Range -1 to +1 Quantitative evaluation of classified products generally involves a comparison of the classified products against reference or "ground truth" information to assess and validate the success of a specific result (Lillesand et al., 2004).For this type of assessment sufficient and good quality reference data are required.If the study area is remote or inaccessible the collection of appropriate ground truth data may be problematic.In such cases, the refinement of the classification parameters and techniques may assist in i) improving the classification and ii) providing uncertainty information which could be used to disseminate the confidence levels inherent to the classification product.
The objective of this investigation is to explore techniques which can be applied towards the optimization of training areas, vegetation indices and band combinations in a pixel-based supervised maximum likelihood classification in the absence of suitable reference data.Pair separation statistics and probability thresholds were used to refine classification training areas and select band combinations with indices to build confidence in the classification results.Additionally, the impact of adding sub-classes and the masking of agricultural areas were quantitatively examined.

Study area
The study area is located in the Greater Limpopo Trans-frontier Conservation Area (GLTFCA) along the northern border of South Africa (Figure 1).This diverse savannah biome is currently an area of interest for cross-border animal movement studies aimed at identifying potential contact zones favourable to disease transfer between wildlife and livestock.The selected area of approximately 87.243km 2 coincides with movement data from three buffalo and seven cattle herds tracked in 2011.A large portion of this region comprises of the Makuleke Contractual Park which is managed by South African National Parks (SANParks) and includes the Makuleke Wetlands which has been proclaimed an official Ramsar site in 2007.In the 2008 management plan for the Kruger National Park (KNP), this area is classified as "primitive" with wilderness qualities (Freitag-Ronaldson and Venter, 2008).

Figure 1. Location and characteristics of the study area
Adjacent to the KNP in the west are additional smaller contract parks and nature reserves.The rest of the study area in South Africa and Zimbabwe mostly consists of communal land with small scale crop production and traditional livestock farming (FAO, 2004).Due to the remoteness of the study area and the fact that it extends across international borders, continuous and effective in-situ field sampling to measure and assess vegetation characteristics is not viable.Similar issues are relevant with regards to the acquisition of ground truth data.
The fact that savanna vegetation is essentially a double layered system with a canopy formed by varying densities of trees and shrub and a lower layer of sub-strata consisting of grass and herbaceous vegetation -also in varying densities, results in a very complex problem when classifying vegetation with remote sensors that essentially measures what is perceived from above (Higgins et al., 2011, Hill et al., 2011, Tuanmu et al., 2010).In this instance, the aim was to extract the vegetation structural groups and land cover types listed as target classes in Table 2. Together with animal telemetry data recorded in 2011, these classes are to be used in forage and animal movement research applications in the GLTFCA.

Data collection and pre-processing
Through the South African National Space Agency (SANSA), Government institutions and tertiary educational facilities in South Africa have reasonable access to free SPOT images.The relevance of any comparison between images acquired on different dates over the same area depends on similar illumination conditions.The SPOT sensors strive to achieve this by its sun-synchronous orbit that ensures that the satellite passes over any given point on the earth's surface at the same local time.SPOT 5 sensors acquire data in two panchromatic bands which is used to generate a 2.5 m panchromatic product (0.48 to 0.71 µm), three 10 m multispectral bands (0.50 to 0.89 µm) and one 20 m short-wave infrared band (1.58 to 1.75 µm).
Notwithstanding the fact that SPOT multispectral imagery has a finer spatial resolution compared to Landsat and MODIS, significantly less vegetation studies using SPOT derived products could be found in scientific literature.Tu et al. (2012) compared various modelling approaches towards predicting potential habitat and species distribution.SPOT 5 imagery were also used for vegetation patch detection in China (Liu et al., 2011) and vegetation change detection within three coarse vegetation classes in the Brazilian Amazon (Lu et al., 2008).However, these studies were all applied in forested areas and cannot be readily adapted to the savanna biome.
Two 2011 SPOT 5 images, one coinciding with the end-of-wet-season dated 30 April 2011 and the other dated 12 August 2011, typical of the end-of-dry season, were acquired.Radiometric correction in both images was achieved through conversion of DN to radiance and then to ground surface reflectance (Lillesand et al., 2004).
Image processing software (ENVI 4.8) was used to co-register the SPOT 5 multi-spectral images to a 2.5 m pansharpened geo-corrected SPOT 5 image which was acquired from SANSA.A subset encompassing the study area was produced from each of the two co-registered SPOT 5 images.These two image subsets were used for all the classification procedures and will subsequently be referred to as the April image and the August image.

3.2
Image classification processes

Size and characteristics of training areas
The quality of the training sites often impacts on the value of the supervised classification (Kar andKelkar, 2013, Lillesand et al., 2004).Various factors may be influential to the effectiveness of training areas or Regions of Interest (ROIs) -especially in this very fragmented and natural study area.For instance, in a 10 m SPOT 5 pixel, one big tree may produce similar reflectance values than 5 smaller shrubs.Similarly, it may be difficult to distinguish between the reflectance from dry bushes, dry grassland and/or dry litter.The potential impact of ROIs, band selection and the two indices on the separability of training areas and the effectiveness of a supervised maximum likelihood classifier (MLC) in this situation were investigated.
The MLC assigns a pixel to the "most likely" class based on the assumption that the cluster distribution is normal (Gaussian) (Pal and Mather, 2003).When using this classifier it is important to ensure that enough pixels are selected in each training area to describe a normal distribution (Adams and Gillespie, 2006).For instance, in an n dimensional spectral space, the covariance matrix will be of size n x n which implies that a minimum of n(n+1) training samples are needed.Each pixel however contains n sample values (one for each image waveband), meaning that the minimum number of independent training pixels is only (n + 1), which would imply 5 pixels per class for the 4 band SPOT 5 pixels (Lillesand et al., 2004, Richards andJia, 2006).It is however often suggested that a minimum of 10n to 100n is desirable because a higher number of training sites may improve classification results by providing a better statistical presentation of each spectral class to be extracted (Kar andKelkar, 2013, Lillesand et al., 2004).
Apart from the size and homogeneity of ROIs, using the same ROIs on temporally different imagery may also affect analysis results.In theory, a vegetation structural class for a natural vegetation area should be stable from April to August in one year, but in reality seasonal changes may impact on the spectral properties of an image and the classification outcome.To test this, the pair separation between a set of training ROIs created from the April image was computed for April image bands but also for the August image bands and vice versa.
If the spectral distance between any two ROIs is not significant for any combination of bands, then the ROIs may not be distinct enough to produce a valuable classification (Gambarova et al., 2010).Both the Jeffries-Matusita (J-M) and the Transformed Divergence distance (separability) measures are available when using the ENVI 4.8 image analysis software.In this investigation, the Jeffries-Matusita distance measure, which has a saturating behaviour with increasing class separation, is reported as a quantitative measure to evaluate the training ROIs.
The J-M distance between a pair of prospective distributions (spectral class values) uses a function of the distance between class means and produces derived values between 0 and 2 (Borges et al., 2007, Marçal et al., 2005, Richards and Jia, 2006, Venkataraman et al., 2006).The separability between two classes is generally considered good when the J-M distance is above 1.9 but class separation is considered poor when the J-M distance is below 1.0 (Thomas et al., n.d., Thomas et al., 2000).

Using thresholds
In the course of the research, various ROI data sets were applied using the MLC and the results were compared.However, a good separability report between ROIs in a training set may not necessarily be an indication that the resultant classification is reliable.To investigate the effect of the size and separability associated with ROIs further, thresholds were applied when the sets of training regions were used in classification procedures.If a threshold value is applied to the MLC, all pixels for which the highest probability (to any class) is smaller than the specified threshold will remain unclassified.Schowengert (1997) suggested that suitable threshold values or intervals may be mostly case dependant.For this study various thresholds were thus selected on an experimental basis.The MLC products associated with various ROI types and the selected thresholds were derived, compared and analysed.
During initial trial runs it became apparent that the relatively small areas where shadow occurs due to topography (±1% of the total image area) may also affect the statistical means and classification results.In the images used for the analysis in this study area, shadow occurred predominantly in the deep v-shaped Levuvhu river valley and in the southwestern quadrant of the study area where a number of ridges occur.To compensate for this across various spectral bands, a shadow class was added to subsequent training files.Additional to the effect of shadow in the SPOT 5 images, the possible impact of subsistence agricultural activities occurring in portions of the study area outside the KNP borders was investigated.These areas may influence the classification results due to varying growth stages in crops, grazing practices, etc. Agricultural areas were masked using a vector dataset obtained from French researchers working in southern Zimbabwe.The polygons were verified using the 2011 pan-sharpened August image.In order to investigate the impact of the agricultural activities on the MLC results, various thresholds were applied to classifications with and without a shadow class and/or masking the agricultural areas.On the basis of all the above investigations, ROIs and band combinations for use in the final supervised classification process were refined.

Confidence levels
In recent years the acknowledgement and visualisation of uncertainties in classification results received widespread attention amongst researchers (Brodlie et al., 2012, Ge et al., 2009, Griethe and Schumann, 2005, Smith et al., 2013).These authors stressed the importance of stating and/or illustrating the limitations of data that may be used in important decision-making processes.Appropriate ways of visualisation involves a thorough understanding of the user requirements and effective communication techniques (Davis and Keller, 1997).However, the use of uncertainty data may influence the level of trust that the user will attach to the information (Schiewe and Schweer, 2013).In the absence of suitable reference data, thresholds were used to designate varying confidence levels within classes.

Results and discussion
Results indicated that, when applied to all four SPOT 5 multispectral bands, the ROIs created from the April image resulted in good pair separation (J-M distance >1.9) between 24 of the 45 pairs and less favourable pair separation between 21 of the 45 pairs.When the same ROIs were applied to the August image, results were reversed (Figure 2).This trend remained consistent when tested on training areas of various sizes and applied to different band combinations.Similar trends were apparent when the process was inverted and ROIs created from the August image was applied to the April image.Pair separation improved (with more pairs illustrating a J-M distance value of 2) when the derived NDVI and MSAVI 2 index bands were added to the separability calculations -even when applied to a temporally different image (Figures 3 and 4).Results displayed in Figures 3 and 4 also suggest that even in the semi-arid and often overgrazed study area, NDVI may have a larger impact on the separability between classes than MSAVI 2 .Figures 3 and 4 were constructed from results obtained when similar sized ROIs were used on different images or image bands.However, the size and homogeneity of the training areas were not yet tested.In Figure 5 the results from only two sets of ROIs are used to illustrate the impact of size, heterogeneity and image bands on pair separation in the April image.One of these sets consisted of small very concise and homogeneous ROIs (30 pixels per class).The other training set allowed for more varied ROIs including larger areas.In this case the number of pixels varied according to the size of the class (1000+ pixels) as perceived visually and incorporated a wider variety of possibly inclusive pixels.As expected, Figure 5 illustrates that separation values improve across all image band combinations when small homogeneous training areas are selected as opposed to larger and more heterogeneous ROIs.Equal to 2 > 1.9 but not 2 1 to <1.9 <1  3. The small uniform ROIs applied to the April image (Case 1) shows that, when a threshold of 0.8 is applied, only about 2% of the image pixels are statistically close enough to the mean of the training sets to be classified.Even a small threshold of 0.01 still resulted in only a quarter of the pixels being classified.This may illustrate that the reasonably good pair separation achieved by this set of ROIs during pair separation tests (Figure 5), will not necessarily result in a good overall classification.This may be because all types of cover may not be captured within these small Big ROIs 1000+ pixels per class Small ROIs 30 pixels per class training plots and/or the small ROIs may not include enough pixel values to successfully represent some of the target classes.
When substantially larger and more heterogeneous ROIs were applied to the April image (Case 3) a much larger percentage of the image pixels were close enough to the statistical mean of the training sets to be classified.The heterogeneousness of these large ROIs in turn resulted in lower pair separation recorded during pair separation tests (Figure 5).The lower pair separation may negatively impact on the success of the MLC and confuse the classification output.No threshold 0 0 0 Figure 6 illustrates a noticeable variation in the number of pixels per class that were classified at the various threshold levels in Case 3. A small variation (as shown for Water, Bare Soil, Riverine Forests and Grass) most likely indicates that the corresponding class may be well extracted.Conversely, a big difference in the number of pixels classified between the highest and lowest thresholds (as for Open Woodland, Open Bushland, Woodland and Bushland) points towards a higher reliance on probability statistics and possibly a significant amount of confusion in the target class delineations.
Case 2 (Table 3) illustrates the impact of using more sub-classes instead of larger ROIs.In this instance ROIs with a varying pixel count in the region of 100 pixels each were used when classifying the August image.Sub-regions were created in an attempt to strive towards the use of fairly homogeneous ROIs but to "fill in the gaps" by adding additional training areas.Results from the Case 2 investigation suggest that the establishment of sub-ROIs for some classes may offer an improvement in "catching" class pixel values that would otherwise be lost.During post classification procedures these sub-groups are then combined as required.Similarly, results indicated a reduction in the percentage of unclassified pixels when a shadow class was added and when agricultural areas were masked out before the image pixels were classified (Table 4).8).A further 28 pairs illustrated a J-M distance above 1.99 but not 2.The lowest J-M distance value was recorded between two sub-class ROIs that were in any case destined to be combined during post-processing (Ironwood and Woodland).However, another 9 pairs with J-M distance values between 1.7 and 1.98 showed less favourable pair separation.
When compared to threshold results listed in Table 4, the classification runs using this refined set of 14 ROIs with various thresholds, resulted in lower percentages of unclassified pixels (Table 6).This may suggest the potential of somewhat improved classification outcomes when using these ROIs.When applying a refined set of 14 training ROIs specially adapted to the April SPOT 5 image (including the indices but with the agricultural areas masked out) less favourable pair separation statistics showed that only 33 out of the 91 resultant pairs achieved a J-M distance of 2 (Figure 8).A further 30 pairs illustrated a promising J-M distance above 1.99.The lowest J-M distance of 1.48 was recorded between two sub-class ROIs of the same land cover class (Riverine Forest and Open Riverine Forest).Another 11 pairs with J-M distance values between 1.5 and 1.90 showed less favourable pair separation.The remaining 15 pairs displayed fair pair separation values between 1.9 and 1.99.7) resulted in lower percentages of unclassified pixels than observed in the August image analysis results (Table 7).Although these threshold results may seem like an improvement on the August image classification, the lower confidence indicated by the pair separation statistics may suggest more confusion between classes and therefore further investigation will be required.When the two sets of ROIs created for each image were subsequently tested on band stacks incorporating all bands and indices from both the temporally different images, the percentage of unclassified pixels increased significantly (>10%) in all cases.This may confirm that the ROIs are essentially image specific due to seasonal variations.
Figure 9 illustrates possible ways of visualizing confidence levels in two vegetation structural classes (Riverine Forest and Sparse Vegetation).It is however debatable if the uncertainty information as conveyed in the map should be visualized at all as it may confuse rather than enlighten the map user.Smith (2013) reported that there is still ample scope for research which on the one hand cognitively assesses the appropriate levels of precision in pronouncing uncertainty and on the other hand measures the impact of uncertainty visualisation on decision makers.

Conclusion
From the results it became apparent that the inclusion of vegetation indices may improve the possibility of statistically distinguishing between vegetation classes.Results also suggested that even in the semi-arid and overgrazed areas, NDVI may have a more significant impact on the separability between classes than MSAVI 2 .Furthermore it was deducted that training ROIs may be image or season specific.It was proved that although small homogeneous ROIs may show good pair separation ability between classes, they will not necessarily produce a good overall supervised result.Conversely it was illustrated that larger ROIs in turn may result in lower pair separation ability between training ROIs.However, when applying thresholds to the supervised classification, a higher percentage of the image pixel values may then be close enough to the statistical mean of the bigger training sets to be classified.Results also suggest that the establishment of sub-ROIs, a shadow class and the use of an agricultural mask may be conducive to improved classification outcomes.Finally, it was shown that probability thresholds may provide a measure of confidence in the absence of suitable reference data for accuracy assessments.Future research may focus on methodologies for deriving optimal training area characteristics and threshold levels relevant to individual target classes in this savanna biome and the evaluation of various approaches to the dissemination of uncertainty information.

Figure 2 .
Figure 2. Pair separation between ROIs created from the April image as applied to the same image and the August image

Figure 3 .
Figure 3. Pair separation between ROIs created from the April image as applied to the same image and the August image using various band combinations

Figure 4 .
Figure 4. Pair separation between ROIs created from the August image as applied to the same image and the April image using various band combinations

Figure 6 .
Figure6.The variation in the number of pixels per class that were classified at the various threshold levels for Case 3 as given Table3

Figure 8 .
Figure 8. Graph summarising the separation statistics for 91 pairs in the final selected ROI sets for the April and August images respectively Classification runs using the final set of 14 ROIs on the April image with various thresholds (Table7) resulted in lower percentages of unclassified pixels than observed in the August image analysis results (Table7).Although these threshold results may seem like an improvement on the August image classification, the lower confidence indicated by the pair separation statistics may suggest more confusion between classes and therefore further investigation will be required.

Figure 9 :
Figure 9: Example of the visualisation of confidence levels based on classification thresholds within the two vegetation structural zones

Table 2 .
Target classes with approximate canopy cover ranges and vegetation structural characteristics * These ranges may overlap between height differentiations

Table 3 .
Illustration of the effect of ROI characteristics on the percentage of unclassified pixels when using thresholds in a maximum likelihood classification

Table 4 .
The effect of shadow and an agricultural mask on the percentage of unclassified pixels.All classifications are based on the four SPOT 5 bands plus the NDVI and MSAVI 2 indices

Table 6 .
Extracts of the results from the August image bands using refined ROIs.Masked agricultural areas are shown in grey on the classification results

Table 7 .
Extracts of the results from the April image bands using refined ROIs.Masked agricultural areas are shown in grey on the classification results