Accuracy of automatic aerial triangulation with bundle block adjustment on large historical image blocks in mountainous terrain

Achieving accurate orthorectification is a major constraint to upscaling the use of historical aerial imagery for 20th century change detection. This paper presents a series of aerial triangulation bundle block adjustment post-processing model tests to determine the planimetric accuracy obtainable for large historical image blocks, which inherently contain film deformations e.g. warpage and shrinkage. Photo jobs with a range of photo numbers (12-237 images) and area coverage (>33 000-~150 000ha) were included. Self-calibration with 44-parameters and fixing the model to the ground control network achieved the highest final planimetric accuracy (total root mean square error [RMSE] ranging from 18.6 – 25.7px [7.1 – 21.8m] at ground control points and 20.5 – 27.7px [7.8 – 23.5m] at checkpoints). Allowing movement in the model by increasing standard deviations at ground control points or automating the removal of blunders in the model significantly reduced final planimetric accuracy. Removing automated points and running postprocessing with no self-calibration also increased the error at checkpoint locations. This study shows that automatic aerial triangulation can assist towards reducing the number of ground control when orthorectifying large blocks of historical aerial photos. However, the study highlights the importance of post-orthorectification accuracy assessment because aerial triangulation accuracy results were not a correct reflection of the error in final orthoimages. Further work should focus on attempting to increase final planimetric accuracy by adjusting the accuracy and number of manual tie points and ground control in combination with altering the amount and positioning of automatic tie points.


Introduction
High-resolution aerial and satellite images are increasingly being used to evaluate and monitor a wide array of environmental and climatic variables (Chen et al. 2016;Bianchetti & MacEachren 2015).Despite recent advances, historical temporal and spatial coverage and poor resolution limits the use of satellite images for determining landscape dynamics during the 20 th century i.e. >30-80 years ago (Gennaretti et al. 2011).
Historical analogue aerial photos provide a unique opportunity to document 20 th century landscape change (Abrate et al. 2013;Gennaretti et al. 2011;Ma & Buchwald 2012;Asiyanbola 2014;Palandro et al. 2003;Tekle & Hedlund 2000;Schiefer & Gilbert 2007).However, a major constraint to upscaling the use of historical aerial photos is challenges associated with accurately spatially referencing a large set of historical aerial images in order for them to be overlaid and/or quantitatively compared with other geographic datasets (Morgan, Gergel & Coops 2010;Rocchini 2004;Nagarajan & Schenk 2016;Wang & Ellis 2005a).
It is well described in the literature that topographically complex and highly mountainous areas present greater challenges for accurate spatial referencing than areas of flat and simple terrain (Rocchini & Di Rita 2005;Rocchini et al. 2012;Wang & Ellis 2005a, 2005b).Film or print shrinkage or warpage is another cause of geometric distortion and particularly of relevance to historic photographs and film (Morgan, Gergel & Coops 2010;Asiyanbola 2014).Errors associated with film deformations are independent to terrain roughness errors; however, these two sources of error combine in complex ways to have much larger impacts on planimetric accuracy (Morgan, Gergel & Coops 2010).
Current literature and country initiatives aimed at spatially referencing and geometrically correcting historical aerial images for change detection have mainly been driven through the standard procedures of manually locating an extensive number of ground control points that can also be located on an existing orthoimage or spatially referenced satellite image (Hughes, McDowell & Marcus 2006;Ma & Buchwald 2012;Nagarajan & Schenk 2016;Wang & Ellis 2005b).This includes approximately 4-20 ground control points per photo (or photo pair) or approximately 30-60 (or more) ground control points for approximately 10 000ha (Gennaretti et al. 2011;Hughes, McDowell & Marcus 2006;Abrate et al. 2013;De Rose & Basher 2011;Marignani et al. 2008;Pulighe & Fava 2013;Rocchini et al. 2006).
Automatic aerial triangulation with bundle block adjustment potentially provides an opportunity to increase the automation of orthorectification of historical aerial imagery and to reduce the amount of ground control required across a large set of aerial images (Addo 2010;Linder 2006).This is through automatic tie point image matching using feature based and least squares matching techniques, bundle block adjustment, and blunder removal (Chen et al. 2016;Verhoeven et al. 2012;Abrate et al. 2013;Addo 2010;Schenk 1996).However, the advantages of these methods for geometrically correcting historical aerial images have not been fully explored.This paper presents an assessment of the potential for using digital automatic aerial triangulation with bundle block adjustment for orthorectifying damaged and degraded historical aerial images covering 30 000 -150 000ha of mountainous and rugged terrain.The main aim of the paper is to describe the effects of different aerial triangulation post-processing models on final planimetric orthorectification accuracy with a reduced ground control network.

Study area
Automatic aerial triangulation post-processing model parameters were tested for a mountainous area in the Western Cape of South Africa known as the Groot Winterhoek Mountains (Figure 1).

Orthorectification and accuracy assessment
The study comprised four main components detailed in Figure 3. Aerial triangulation in the form of bundle block adjustment and orthorectification was conducted respectively using MATCH-AT and OrthoMaster within Applications Master from Inpho (Trimble/Inpho 2014).The final accuracy assessment of the orthoimages generated was manually determined using ArcGIS software (ESRI 2015).

Input: historical and reference data, images and DEM
High-resolution scans of historical aerial photo negatives (21 microns per pixel) and camera calibration reports were obtained from South Africa's National Geospatial Information (NGI) office for 1948/9 and 1971/2 (Table 1).The analogue film negatives were warped to different degrees and these film deformations were also visually evident after the scanning process.

Aerial triangulation with bundle block adjustment
Aerial triangulation in the form of bundle block adjustment using a rigorous camera model was For this study, warped images necessitated manual interior orientation to align image and camera coordinates using the fiducial marks present on the images.Automatic tie point extraction was achieved using a combination of feature-based and least-squares matching procedures, but was only possible after the relative positions of the images in the block were refined by placing three manual tie points in Von Gruber positions for each image (Trimble/Inpho, 2014a).
Eight post-processing models were then tested using the results of the functional model used to triangulate each photo job (Table 2).The procedure started with a standard model run (std0.1),which included no blunder removal, no self-calibration and with fixed standard deviations set for planimetric and height ground control points (namely, 0.1m and 0.1m).This standard model run was then post processed using the seven different options described in Table 2.

Orthorectification
After each post-processing run, the exterior orientation for each photo job was imported into OrthoMaster.Orthographic corrections were performed using the Ortho Rectification tool and a 25m Digital Elevation Model (absolute accuracy 2.5 -5m) (Trimble/Inpho 2014).
Table 2 A list of post-processing models that were selected and run to evaluate the success of aerial triangulation accuracy and final orthorectification accuracy.

Accuracy assessment
For assessing final orthoimage planimetric accuracy, orthoimages generated in OrthoMaster and ground control points captured in the current orthoimage aerial triangulation project files were imported into ArcGIS.A manual procedure was then used to plot the deviation of each control point in the historical orthoimages from the ground control point in the current orthoimages (reference images).Deviations were measured for ground control points in all historical orthoimages including overlapping images.For example, for photo job 266 there were 40 ground control points with 117 photos covering at least one of these.The "deviation" is the root mean square error (RMSE) and can be expressed as follows: where is each ground control point and and are the residuals in the x and y axes.The total root mean square error for each post-processing run was derived using the following equation.
where is the error associated with each th ground control point (Rocchini et al. 2012).
Planimetric accuracy was also assessed at checkpoints for all photo jobs but only for the top two performing models and the standard model run (std0.1).
Spearman rank correlations were then performed to investigate correlations between accuracy results from the aerial triangulation process (Table 3) and final planimetric accuracy (

Planimetric accuracy of orthoimages
The self-calibrated post-processing model, c0.1, consistently showed the highest final planimetric accuracy at ground control points for all four historical aerial photo jobs (Figure 4).
This model can be viewed as achieving reasonably high accuracy especially given the limitations of the study area (complex geomorphology) and image quality (warpage across image surfaces) (Rocchini et al. 2012).The model also managed to retain this accuracy at additional checkpoints which were not included in the aerial triangulation process (Figure 5).
The non-automated model, which only included manual tie points (i.e.noauto0.1),showed the next highest accuracy at ground control for three of the photo jobs (226, 225 and 699).However, in contrast to the self-calibrated model, this model showed inconsistent planimetric accuracy with three out the four photo jobs showing decreased planimetric accuracy at additional checkpoints.In certain cases, this included an increase of more than 30 meters of planimetric error (Figure 5).Postprocessing models that allowed the aerial triangulation process more room to adjust around ground control points (i.e.setting standard deviation of five and two meters) were ineffective at reducing the final root mean square errors in orthoimages.This is despite being a far more realistic standard deviation estimate.For example, it is rather unrealistic to assume the standard deviation for ground control digitised from an orthoimage as being within 10cm.Constraining the standard deviations, however, clearly resulted in higher planimetric accuracy.

Aerial triangulation accuracy compared with final planimetric accuracy
The overall aerial triangulation accuracy for most post-processing models was much lower than the recommended one third of a pixel.Average Sigma0 was 38µ (1.8 pixels) and ranged from 11µ (0.5 pixels) to 147µ (7 pixels) (see AT Sig0 in red in Figure 4).Nonintuitively, the overall accuracy of the aerial triangulation process as measured by the Sigma0 was significantly negatively correlated with final planimetric accuracy at ground control (i.e.or.XY and or.max.XY) (Figure 6).
A similar relationship was found for the residuals in automatic points x (auto.x),and no significant correlations were found between all other aerial triangulation accuracy results and final planimetric accuracy in historical orthoimages (Figure 6).
When considering aerial triangulation accuracy results for ground control there were, again, no significant correlations present for post-processing models, which were set at a standard deviation of 0.1m (Figure 7 A).For more flexible models (standard deviation at 2m and 5m), a more positive relationship seemed to develop which included significant positive correlations between maximum planimetric accuracy in orthoimages (or.max.XY) and maximum aerial triangulation root mean square errors and residuals for height ground control (Figure 7 B).However, flexible models also showed the lowest final planimetric accuracy and therefore this is not an entirely useful measurement.
Figure 4 Final planimetric accuracy of orthoimages generated and direction of error for all ground control points summarised by the total and maximum root mean sqaure error (RMSE).Aerial triangulation accuracy measurements are included for comparison purposes including total and maximum XY root mean square error results shown in brackets and the overall triangulation accuracy in red (AT Sig0) .
Figure 5 Final planimetric accuracy at check points for orthoimages for all four photo jobs for c0.1, no-auto0.1 and std0.1 post-processing models.Limited and negative correlations between aerial triangulation results and final planimetric accuracy could be linked to a combination of aspects.Poor interior orientation (Figure 8) because of warpage and degradation across the entire image surface could have reduced the overall accuracy of the aerial triangulation process for all calibrated as well as non-calibrated post-processing models.It is also likely that the poor interior orientation process prevented a fully automated tie point matching process and the need for manually placing points in the Von Gruber positions for all images in all photo jobs.Poor interior orientation likely resulted in large residuals of the corresponding image and object coordinates.Therefore, although manually and automatically measured points were visually correctly measured, when using blunder detection they were eliminated because of poor transformation parameters of the interior orientation (Table 5).
Furthermore, for post-processing models in which blunder detection was not used, the accuracy calculated for image and object coordinates was low, and this affected the overall Sigma0 and resulted in misalignment between aerial triangulation results and final planimetric accuracy in orthoimages.
Blunder detection removes observations based on standard deviations specified.The smaller the standard deviation specified the higher the weight and the more accurate observations have to be, not to be eliminated.The higher the standard deviation values, larger observation error is accepted and the observation will not be eliminated.Therefore, when blunder detection was used in combination with a standard deviation of 0.1m the process removed manually placed points (including ground control), which were correctly placed.This was then determined by the aerial triangulation model as an increase in accuracy whereby "incorrect" tie points or control points (which in fact were actually correct) had been removed.As a result, post-processing models, including blunder detection and fixed to the ground control showed some of the highest accuracy for the aerial triangulation process i.e. lower Sigma0 and ground control root mean square errors (e.g.b0.1 and cb0.1).This however did not translate into similar levels of accuracy in the final orthoimages generated as achieved when using the self-calibrated model, c0.1, which showed lower aerial triangulation accuracy but higher final orthoimage planimetric accuracy at ground control (see b0.1 and c0.1 in Figure 4).One way of avoiding manual and automatic measurements from being eliminated is by giving them a larger standard deviation, thus allowing bigger residuals.
However, post-processing models with planimetric and height standard deviations of five and two meters respectively did not achieve acceptable levels of final planimetric accuracy (Figure 4).

Discussion
Errors from warpage are cumulative during the aerial triangulation and orthorectification process.This study has shown however that it is not completely restrictive with certain postprocessing models being able to achieve relatively acceptable accuracy.The self-calibration process in combination with constrained standard deviations of 0.1m consistently resulted in improved final planimetric accuracy across the four different photo jobs tested which included different ground control points, area coverage and number of images.
Although the process could not be fully automated, a substantially reduced number of ground control points was used in all photo jobs in comparison to the recommended 4-20 per image or image pair.For example, for a photo job of 237 images (covering ~150 000ha), this study used 40 ground control points and placed three manual tie points per image.Manual tie points between images were required, however, these are much quicker to find and measure in comparison to ground control points.Despite the assumed random nature of the error, self-calibration proved effective for increasing final planimetric accuracy to acceptable levels at both ground control and furthermore at additional check points indicating that there may have been some level of systematic error in the aerial images (Kaichang, Ruijin & Rong Xing 2003).
The total and maximum root mean square errors achieved for ground control and checkpoints for the constrained self-calibrated model, c0.1, are on par with orthorectification results achieved in other geomorphologically complex areas with higher quality original analogue photos and better scanning procedures (Rocchini et al. 2012).Local studies have a tendency to not report on georeferencing error with most dismissing the need for checkpoints and therefore rendering the total root mean square errors (when reported) an inadequate reflection of the positional error across the images.In studies that have reported errors, these have been relatively large, despite the flat terrain, When considering aerial triangulation of historical aerial imagery, it is important to understand that high aerial triangulation accuracy does not always translate into high planimetric accuracy in final orthoimages generated.Aerial triangulation accuracy results are highly influenced by the standard deviations set as well as other post-processing parameters linked to blunder detection and calibration.It is possible that the addition of more manual tie points for post-processing models that do not include blunder detection could reduce final planimetric errors in orthoimages.However, this would likely reduce the aerial triangulation accuracy results, an artefact of the post-processing model assuming manually placed points are incorrectly placed because of poor interior orientation, but also increase the time required for each independent photo job.The influence of the accuracy and number of points in the control network is a theme for further research.

Conclusion
This study has shown that automatic aerial triangulation can be used for orthorectifying historical aerial photos with a reduced number of ground control for large blocks of aerial imagery.Selfcalibration in conjunction with constraining standard deviations around the ground control was required for achieving acceptable levels of planimetric accuracy in final orthoimages comparable to other studies working in geomorphologically complex areas as well as an improvement to georeferencing studies in South Africa.
The national mapping department in South Africa has an extensive collection of historical analogue aerial photos with a large coverage across the country dating from the early 1900s (Figure 2).This is not unique to South Africa with many other countries (including least developed and developing) also having extensive analogue photos in storage (Palandro et al. 2003;Tekle & Hedlund 2000;Schiefer & Gilbert 2007)
Historical aerial photos were sourced from South Africa's national mapping organisation, the National Geospatial Information (NGI) office, a Chief Directorate of the Department of Rural Development and Land Reform (DRDLR).

Figure 1
Figure 1 Study area: the Groot Winterhoek Mountains in the Western Cape (WC) of South Africa (A-E).E is linked to the horizontal line in D and is an indication of the complexity of the topography (NGI-DEM 2016; MDB 2013; DEA 2016; SUDEM 2014).

Figure 2
Figure 2 National coverage of non-rectified aerial imagery available for South Africa from the NGI with scales ranging from 1:5000 and 1:50 000.The image on the right shows the number of overlapping years (NGI (2015a, 2015b)).

Figure 3
Figure 3 Study workflow consisting of four main components i) input images, data and DEM; ii) aerial triangulation with bundle block adjustment; iii) orthorectification; and iv) accuracy assessment.Current 2013/2014 digital orthoimages were also obtained from NGI including the respective aerial triangulation project files (Intergraph project files) and a photogrammetrically compiled 25 m Digital Elevation Model (DEM) (absolute accuracy 2.5 -5m) for use in the orthorectification process.Ground control points and Check points (XYZ) were measured directly from the current (2013/4) orthoimages' aerial triangulation Intergraph project.This was achieved by converting these files for use in Inpho and then using MATCH-AT to measure the coordinates for each respective ground control point.When placing a point in the existing orthoimages in MATCH-AT it provides the coordinates based on the block bundle adjustment used to generate the orthoimages.Ground control points were chosen based on their visibility across historical aerial photograph sets and the current orthoimages.
conducted to compute the exterior orientation of individual images.The rigorous camera model entails using physical parameters about the camera including focal length, principle point location, pixel size, and lens distortions and orientation parameters to represent the imaging geometry (Kaichang, Ruijin & Rong Xing 2003; Ma & Buchwald 2012).The rigorous camera model requires far less ground control points than rational function models which are generally appropriate for small areas with gentle terrain (Wang & Ellis 2005b).The bundled triangulation approach determines the exterior orientation using the collinearity condition (Nagarajan & Schenk 2016; Kaichang, Ruijin & Rong Xing 2003; Schenk 2004).Collinearity is expressed by the following equations: where , and are the ground coordinates, and are the image coordinates, and are the coordinates of the exposure centre in the ground coordinate system, is the calibrated focal length of the camera, and are the image coordinates of the principle point, and are the elements of the rotation matrix with the three angles ( (Omega, Phi and Kappa -yaw, pitch and roll).and represent the interior orientation parameters which were measured using the camera calibration report, while Xs, Ys, Zs, are the exterior orientation parameters (Nagarajan & Schenk 2016; Kaichang, Ruijin & Rong Xing 2003; Schenk 2004).

Figure 6 Figure 7
Figure 6 Spearman rank correlation coefficients between final orthoimage planimetric accuracy (or.XY and or.max.XY) and aerial triangulation accuracy measurements (Sig0, manual.xand manual.y,auto.x,auto.y).P values > 0.05 are indicated by a cross through the coefficient values.A (n= 20): SD set at 0.1m B (n= 12): 2m and 5m respectively

Figure 8
Figure 8 Boxplot of interior orientation results.Jitter points indicate individual images per photo job.The mean Sigma0 (in mm and pixels) is shown for each photo job.
. Ideally, imagery could be provided on an open source platform ready for viewing by the public and for review for potential change detection studies by applied scientists and other researchers.GeoMemories is an example of an initiative working towards this goal in the Italian landscape (Abrate et al. 2013).Libraries of Brock University, Santa Barbara and Stanford Universities have also been conducting similar work towards creating portals for national aerial photo archival data collection and distribution (Ma & Buchwald 2012).
NGI contributed the scanning of analogue aerial photos.Raoul Duesimi (NGI) assisted with access to storage areas and provided Intergraph aerial triangulation project files of current orthoimages and the data for generating Figure 2. Additional individuals that assisted and supported the study at NGI include Sue Kirschner, Rabelani Lidzhade and Aslam Parker.Maps throughout this article were created using ArcGIS® software by Esri.The research was funded by the Plant Conservation Unit, the African Climate Development Initiative and the National Research Foundation.

Table 1 .
Historical film based aerial photos including relevant scales and dates flown.

no. Date flown Scale Area (ha) No. of images (strips) Focal length (mm) Image size (cm) No. of gcps 1 /cps 2
1 Ground Control Points: Directly used for aerial triangulation and bundle block adjustment to generate orthoimages.2CheckPoints: Not used for aerial triangulation and bundle block adjustment but used to assess the consistency of planimetric accuracy across orthoimages generated.

Table 3
Accuracy parameters generated in Inpho photogrammetric software for the aerial triangulation process with bundle block adjustment.

Table 4
Accuracy assessment variables derived from manually measured deviations between historical orthoimages generated and reference orthoimages in ArcGIS.

Table 5
Ground control points, automatic tie and manual points for different model runs.