Accuracy assessment of smart devices for Geoscience field mapping

We present the measurement of fractures near the town of Beaufort West, South Africa. A field visit was conducted to examine the dip and azimuth of rock outcrops in and around the town. The locations of these various fractures were mapped and their orientation, which included the dip and strike of the rock surface, was measured using a geological compass (i.e., Brunton Truarc 15 Compass). The geological compass measurements were then compared to three mobile devices. These mobile devices, namely an iPad 2 and two smartphones (Samsung S8 and Huawei P10 Lite), all had the same application for standardization and the mobile device results were individually compared to the geological compass. The data stemming from the various mobile devices and the geological compass were then compared in terms of their variance. This statistical analysis was performed using the Correlated T-test method, as well as the Pearson Correlation Coefficient formula. To visually examine the main fracture orientations, the data obtained using the geological compass was plotted on a rose diagram. Results show that the relationship between the geological compass and the mobile device readings had little to no correlation, when using both the correlation and t-tests as combined determinants. This highlights the importance of ensuring measurement accuracy in the field as well as instrument calibration in situ.


Introduction
Fieldwork is a core component of any geological study, as it assists in understanding the evolution and formation of the Earth's long and complex history. Field visits and mapping form a fundamental and exciting time for any geological undergraduate degree (Lundmark et al., 2020). Rapid technological advances over the past decades have greatly improved the ease and accessibility of mapping tools and their subsequent outputs. Modern smartphones have become digital Swiss Army knives with their ability to track, manage, edit, and store data. A smartphone can be used to determine their locations via a GPS, perform complex calculations and take photos for field mapping purposes as shown by Weng et al (2012). Smartphones have become an extension of students' arms due to their utility and ease of use. In terms of fieldwork, one of the physical devices which have stood the test of time is the geological compass. Modern smartphones now have powerful gyroscopes and accelerometers which aid the device in determining orientation in any 3-dimensional plane.
Applications, referred to simply as 'apps', available on mobile devices may start to phase out the classic geological compass if they can prove to be accurate and reliable (Allmendinger et al., 2017).
The increasing popularity of mobile devices means that many students have access to a smartphone or tablet of some kind (Gadzama et al., 2019). This helps in terms of aiding the learning process and has a major impact on the delivery of material and the way the students learn (Giles et al., 2020). It should however be noted that the reliability of smartphones is questionable and the results, particularly those stemming from mobile apps, should be examined with caution (Meskini et al., 2019). Despite these facts, mobile device use is growing in emerging economies (Figure 1). Almost two-thirds of all South Africans have access to a smartphone with an even higher percentage of youth, aged 18 to 35, owning these devices (Silver et al., 2019). , which is an independent collaboration of software developers, has allowed a spatial dimension to be added and the integration of apps and various pieces of software into one location (De Filippis et al., 2020).
With all these advantages it should however also be noted that a comparison between the digital and manual devices as well as their precision and reliability is needed in an African context. This is particularly true across types of mobile devices and software as well as for various operating systems (Android vs Apple). Thus, highlighting the importance of a study of this nature.

Literature review
Field mapping has formed a critical component of Earth Science studies and research. This practice also plays an important role in the delineation of resources and understanding their distribution in the subsurface.
Roger Tomlinson spearheaded the transition from paper based to digital maps and he also went on to develop the Canadian Geographic Information System which made it possible to manage, model and analyse large quantities of geospatial data (Rura et al., 2014).
As GIS and technology becomes more advanced, geologists and cartographers now make use of satellite, airplane, and drone imagery in conjunction with GIS software such as ArcGIS and QGIS. This is further complimented by photogrammetry software such as PIX4D and Agisoft Metashape to easily create high resolution maps and 3D models. Digital cameras and smartphones allow for effortless documentation and storage of field observations in the form of high-definition photographs (Weng et al., 2012). On 18 March 2020 Apple unveiled the new iPad Pro (4th generation) with a LiDAR Scanner; thus allowing these mobile devices to aid in the collection of remotely sensed data which can be used in mapping. Traditional field equipment such as the Brunton Truarc 15 Compass is slowly being replaced by mobile apps such as FieldMove Clino, regardless of the questionable reliability of smartphones and tablets (Meskini et al., 2019).
The further development of mobile devices and the consequent increase in the number of mobile apps has led to increased advances in terms of usability, particularly in the field. Nowak et al. (2020) have extensively reviewed examples of these and highlighted the fact that citizen science is playing an increasingly important role in documenting our natural environment. Some of these include: This is further complemented by a timestamp and the generation of a single file to compile data in various formats which has been collected at one location (Weng et al., 2012). The combination of data into one location shows that the use of a mobile devices allows for the simple and easy collection of information and organizes this data into a usable format. Therefore, this study aimed to compare the use of a single mobile app across Android and IOS devices and cross-correlate the measurements with those from a Brunton Truarc 15 Compass, due to no study of this nature having been conducted in an African context. This is of the utmost importance as previous studies have shown ease of use of mobile devices in the field, but the combination of digital and traditional tools is preferred by users (Lundmark et al, 2020).
The use of Unmanned Aerial Systems (UAS) has taken the fore in terms of mapping in recent times. These UAS typically provide a higher resolution image than satellites, and allow for a larger area to be mapped than on foot, when compared to location specific mapping exercises such as this study ( Figure 2) by Manfreda et al (2018). Even though these UAS are finding widespread applications, Manfreda et al (2018)  The reality is that users are combining the aforementioned tools in order to capture measurements across spatial and temporal scales (Manfreda et al., 2018). This is important for understanding uncertainty during data collection.

Study area
The study area is situated in the town of Beaufort West, South Africa. The Nuweveld Mountains are located North of the town, and are characteristically flat-topped with thick dolerite caps, which are resistant to erosion. The study area is relatively flat with multiple rock outcrops and roadcuts, thus making it an ideal area to test the efficacy of mobile apps for earth sciences. This is complimented by the fact that several previous studies have measured the orientation of fractures in the area and these can then be cross-correlated.
Beaufort West is situated approximately 930m above sea level. At this altitude, precipitation mainly occurs during summertime (Tyson and Preston-Whyte, 2000). The study area consists predominantly of cold-dry winters and hot-wet summers. In winter, there have been occurrences of temperatures reaching below 0°C and above 40°C in summer. Most of the rainfall during summer occurs in the form of thunderstorms, turning the dried-out rivers into temporarily raging torrents (Willis, 2014). The regional geology of Beaufort West (Figure 3) is largely characterized by the Abrahamskraal and Teekloof Formation, which belong to the Adelaide Subgroup and Beaufort Group (Rose and Conrad, 2007). The Karoo Supergroup is extensively intruded by dolerite dykes and sill ring complexes (De Wit and Linol, 2016). Woodford and Chevallier (2002) identified a number of oblique lineaments, fracture-sets and master-joints with a northwest-southeast strike orientation occurring within the Beaufort West Area.

Devices and software used
Three different mobile devices, which all contain gyroscopes and accelerometers were used. All devices were up to date with their latest software prior to taking measurements (Table 1).
Unfortunately, no information on the type of accelerometer and gyroscope could be found for the specific mobile devices. The software used to take measurements on the three mobile devices was FieldMove Clino developed by Petroleum Experts Limited, as also used by Lundmark et al. (2020). The software is free for download on both Android and Apple devices. Before readings were taken all devices were calibrated to improve reading accuracy. This calibration functionality is built into the devices and the mobile apps in order to ensure accuracy.

Data collection
Physical measurements were taken on the rock surface using a Brunton Truarc 15 Compass in the immediate vicinity of Beaufort West (Figure 4). This included the location of the examined fractures, using a handheld GPS, and was further complemented by dip and azimuth readings on the surface of the rock to measure fracture orientation with the geological compass. These readings were also taken using the three mobile devices. All dip and azimuth readings of the rocks were taken along flat planes and each device was placed in the same location to minimize variance.

Statistical Analysis
Correlation coefficient and the T-test analysis were employed to determine the accuracy and precision of the mobile devices. The correlation coefficient determines the relationship strength between two variables, on a scale from 1 to -1, with 1 indicating a strong positive correlation and -1 indicating a strong negative correlation. There are several correlation coefficient formulas which are applied in various fields with numerous applications and requirements for their specific use (Helsel et al., 2020), with the Pearson correlation coefficient being the most common. [1] Where x and y indicate the sample mean between the independent and dependent values respectively (Lane, 2003).
where T is the T-value, mean1 and mean2 refer to the mean of each data group, s(diff) is the standard deviation between the data groups and n is the sample size (Boslaugh, 2013).  Table shown in Beyer (2017).
The Sum of Squares within (SSW) and Sum of Squares total (SST) was computed by comparing the means of each device with the geological compass measurements. Thereafter the Sum of Squares between (SSB) was determined. The SSB value is divided by the SSW in order to determine the final value and if this is less than the minimum acceptable T-value then the hypothesis is accepted, provided that the correlation value is also greater than 0.5.

Fracture orientation
Two major fracture orientations were noted during field observations in the study area, shown in Figure 5 (a). The 125 fracture readings taken in Beaufort West using a Brunton Truarc 15 Compass were used to produce the rose diagram in figure 5 (b), illustrating the strike orientation of the fractures in this area. This rose diagram indicates major fractures have a northwest-southeast strike orientation, and secondary fractures have a west-east strike orientation. Thus, confirming observations related to fracture orientation for the majority of measurements (Woodford & Chevallier, 2002). This is critical for ensuring reproducibility of results and measurements as well as calibration of digital tools in the field. Figure 5. a) Fractures observed in the study area; and b) a rose diagram illustrating strike orientation of the fractures seen in (a).

Variance analysis
For this study, the geological compass readings are viewed as the 'true' values. Values taken by mobile devices are compared to the true values to determine their overall accuracy. For the correlation coefficient, only devices with a R-value > 0.7 will be accepted as this indicates a strong positive relationship.
For the t-test analysis, the critical value was determined to be 1.86 by making use of the distribution table (table 2). T-values greater than 1.86 mean the null hypothesis was rejected. If both the correlation coefficient and T-values fell within an acceptable range, then the readings were accepted, and the device is acceptable to use. Correlation coefficient and t-test analysis were performed on the azimuth and dip readings independently.

Road Cut 1
Looking at the correlation coefficient and t-test analysis results for the azimuth and dip readings at road cut 1 the only device which had an acceptable correlation coefficient was the Huawei (table   3), however the t-value was rejected indicating the device is unacceptable to use. Samsung had an acceptable t-value but its correlation coefficient fell below the allowed limit.

Road Cut 2
The measurements for the readings taken at the road cut 2 (table 4) shows that all devices had acceptable azimuth t-values. The Samsung device was the only one to have an acceptable correlation coefficient and t-value. Dip values for all devices had low correlation coefficients, whilst both the iPad and Samsung had acceptable t-values, which can be seen from the accepted null hypothesis, however due to their weak correlation coefficients they were rejected. From table 3 and 4 it can be seen that all devices were almost unanimously rejected based on the poor relationship between the t-value and the Pearson correlation, the only exception being the azimuth readings for the second road-cutting. Overall, these devices are not yet reliable enough to be used for fieldwork. Furthermore, field observations during the first 5 recorded observations with the mobile devices were vastly different from the measurements taken with the geological compass.

Conclusion
The data presented, highlights the fact that issues are evident in terms of using technology, like mobile devices and mobile apps used in the field for geoscience mapping. This does pose some problems for earth scientists who are doing field work and creates issues in terms of accuracy of measurements, as shown in this study. The outcomes clearly highlight the importance of calibrating digital measurements with manual ones in order to correct and better understand issues around instrument calibration. Allmendinger et al (2017) have outlined the following precautionary measures to be taken in order to help alleviate this issue in the future: 1. One should make multiple measurements of the same surface using each type of instrument and compare the averages of the measurements.
2. When evaluating planar orientations, one should always compare the angular difference between poles to the planes and not the difference in strike or dip.
3. The user should invest in a program that monitors the device sensors over time and experiment with the effect of proximity of external metallic objects on the phone magnetometer.

4.
Where possible, the user should compare their phone measurements, not only to the Brunton geological compass measurements, but also to data independent of the magnetic field such as LiDAR topographic contours and Google Earth images.
The aforementioned points are critical for future studies and need to be taken into consideration when completing field work using mobile devices. This should be complemented with calibration from the outset as it was a major issue in the initial phases of this study.
The changing landscape of geoscience, which is impacted by technology at every turn, needs to be taken cognizance of (Giles et al., 2020). Thus, the marriage of technology with classical methods into a transition of solely digital tools needs to be carefully undertaken and analog methods should not entirely be removed. The limitations of technological tools in the field should be further examined (Manfreda et al., 2018). Therefore, these digital tools can be seen as supplemental and complementary to the classical analog devices (Giles et al., 2020). This is due to the fact that uncertainty and calibration play a major role in digital devices, such as the smart ones used in this study, as shown by the data collected.

Acknowledgements
We would like to thank Nelson Mandela University for funding aspects of the field work.
Furthermore, the NRF and DST are thanked for providing bursaries for the students. The farmers and people of Beaufort West are also mentioned for their hospitality during our field visits. We would also like to remember the late Professor Maarten De Wit, who supported our research financially and always reminded us to be critical of the data we collect. Lastly, we would like to pay our gratitude and respects to our co-author and colleague, Keegan Stokes, who sadly passed away on 30 January 2022. The Iphakade publication number for this publication is 271 and the AEON publication number is 205.