Processing Pix4D imagery with GCPs

Introduction

When working with a project that demands Ground Control Points (GCP), knowing how to correctly position, measure and record GCPs can make all the difference. In this update, I will delve into this topic. This week's focus lay on using the same techniques from last week but with the "small" difference of including GCPs. Also, the class was about to use images over another area, the Wolf Paving site in Wisconsin. A point cloud data set, a few maps and a video was generated. We were supposed to process our image data with the Pix4DMapper and then make maps with the generated data in ArcGIS Pro. In contrast from last week, this week the GCPs was part of the workflow.

Ground Control Points are used to georeference models while checkpoints (CPs) are used in the after-process to assess the absolute accuracy of the model. GCPs are marked targets on the ground, distributed throughout the area of interest. Ideally one should have at least 5 GCPs, preferably 10. Using more than necessary is good for quality checks. Additional GCPs can be used as checkpoints to improve both the relative and absolute accuracy. To avoid confusion, one would have to make sure the GCPs are positioned far enough apart. Generally, if one sees more than one GCP in an image, they are too close together. The GCPs coordinates are used to help Pix4DMapper accurately position the map in relation to the real world.

The quality of GCPs have an impact on the accuracy of the math model and that itself determines the result of the project. The person who is dealing with the UAS data has to rely on that the collector of the GCPs has done their work correctly. For the collector of GCPs, it helps to choose features that are possible to identify at the resolution of the raw image. He or she should choose features that are close to the ground, for example, buildings or big rocks. Then identify the features in the raw image that is wanted as GCP before collecting the actual GCP coordinates in the field. Hiring a surveyor to measure the location of the GCPs is an option. In one of the courses, I have done this myself using a handheld GPS unit. Also, it is important to collect GCPs from a variety of elevations in a broad distribution over the whole image. Preferably one spot in each corner and then one in the sweet spot of the area of interest.

Methods

The project images for this week's tasks were collected from this site, The Wolf Paving, Figure 1.

Figure 1: The site of Wolf Paving

Some metadata, that needs to be addressed. Notice the additional information about where exactly we used to measure the GCPs, Table 1.

Table 1: Metadata. Notice the additional information

The process, which I used to work this data set will be covered below. It started with the import of chosen images like to work with. The images were nadir straight down view, Figure 2.

Figure 2: Nadir viewing

Figure 3 illustrates, the preferred option is to use the Linear Rolling Shutter model, which will be displayed in Figure 4.

Figure 3: Linear Rolling Shutter model

Since the GCPs (blue crosses) are clustered between the lower rows, to the pilot and decision-makers, one could argue to exclude flying the top row line. If one decides to not cut it, one run into the risk of having a bias on the sampling of GCPs and by that bringing in inaccuracies in the dataset.

Figure 4:  GCPs and the flying route.

Next step, Figure 5, was to import our GCPs from a text file. An important reminder here is to never just accept the defaults. One always have to doublecheck that the GCPs are in the same coordinate systems and use correct datum.

Figure 5: If needed, this is where one can change the GCP coordinate system.

The text file we received from the field operator included these numbers, Figure 6. The order was not properly done, which needs to be taken care of.

Figure 6: The XYZ fields were in an incorrect order.

Some alterations were needed for Pix4DMapper to read them correctly, which is the Northern, Eastern and Elevation. Therefore one has to switch order, Figure 7.

Figure 7: Changing the order of the values of Northern, Eastern, and Elevation.

Figure 8 displays how the software tool looks, right after a proper change and the import of the GCPs.

Figure 8: The imported GCPs, in the correct order.

Figure 9, 10 and 11 all display steps in the process highlight settings regarding the Point cloud and Ortho-creation. There are some features that are optional but, which in the end, can give interesting aspects of the work. Also, it is possible to create new shapefiles, ready to be used in GIS-softwares like QGIS or ArcGIS.

Figure 9: Resolution and point cloud options.

Figure 10: Digital Surface Model and Orthomosaic options.

Figure 11: Additional outputs.

Below, Figure 12, one discovers there is an offset between the GCPs (blue vertical lines) and the ground. Since the GCPs are floating in space the Pix4D will have to bring the UAV data to reality. These errors often occur because they are done in a different datum.

Figure 12: Offset between the GCPs and the ground.

Working with GCPs are not always easy since misplacements of them occur, which will reduce the accuracy later on in the project. In order to facilitate this work and properly see the GCPs distribution, it is a good idea to open an ArcMap project with the same GCPs. Figure 13 and 14 displays this procedure.

Figure 13: Work side by side between Pix4DMapper and ArcMap.

Figure 14: A way to doublecheck where the markers are is to use ArcMap on the side.

Also working with ArcMap one has to consider the actual coordinate system. As Figure 15 shows, the projected coordinate system was WGS84 UTM 16N. Be aware of the metadata file to see the orders of the fields, some alterations were needed here.

Figure 15: Choose the correct coordinate system.

After these steps, in Figure 16, ArcMap displays the GCPs in green. It is good to use the Identifier-tool, so one knows which one is which.

Figure 16: GCPs in green and the Identifier-tool.

Below in Figure 17, the Identifier-tool gives us the value for point 101.

Figure 17: Data for point 101.

Ater the correct GCPs are identified, we are going back to work in Pix4DMapper. Below in Figure 18, is the initial difference between the chevron (the red marker up left) and the GCP-data (blue dot in the bottom right). In the next step, this is something which is going to be altered for all GCPs.

Figure 18: Difference between the marker and one GCP.

Right after the GCP data has changed it looks like in Figure 19. Also, according to field operators, always go on the inside of the chevron. Dropping this phase can weaken the quality significantly, especially in areas where high elevation variances occur. This is important information and should go into the metadata.

Figure 19: Be meticulous when deciding correlation points.

After these changes and the GCPs are brought down to the ground, the imagery has to recalibrate itself to where those GCPs are now. This is done in a re-optimization process, shown in Figure 20.

Figure 20: Re-optimization

As mentioned before, the GCPs were brought down to the correct location on the ground and after that, we did a re-optimization. Figure 21 displays the result of this process. Now it is time to create the Point cloud, the Digital Surface Model and the Orthomosaic. This process takes a while.

Figure 21: After the re-optimization is done, it is time for a break. It will take some time so it will be allowed to listen to Dr. Hupy's favorite song of all time. No kidding, behold. https://www.youtube.com/watch?v=9vST6hVRj2A

Welcome back! In Figure 22, it is possible to see the GCP marks and the cloud matching one another in the mesh of the Point cloud data.

Figure 22: The result of the images has been processed.

Figure 23, displays that the GCPs match up with the ground surface.

Figure 23: This is where I chose the GCP to be for Point 101 earlier.

As illustrated below in Figure 24, the GCPs are on the ground and the DSM and Ortho are created.

Figure 24: Final product ready to be used for other purposes.

This is a better angle to illustrate the above mentioned regarding the GCPs, Figure 25.

Figure 25: The ground welcomes the GCPs.

Results and Discussion

Below is the video I created with Pix4DMapper, Video 1. This time the video includes Ground Control Points.

Video 1: My video, created with Pix4DMapper includes GCPs.

Instead of using ArcScene this time to create a 3D version of the place I chose to use the 3D report we chose to create earlier from Pix4DMapper. See Figure 26 below, a meshed version of the site was created.

Figure 26: Results from the 3D report.

My goal was to create maps that showed both the GCPs and the elevation data in an appealing way. Both for the Orthomosaic and the Digital Surface Model, with the latter as the biggest challenge, Figure 27.

Figure 27: Digital Surface Model, with interposed visuals from Pix4DMapper.

Figure 28, this week's orthomosaic illustrates the details in better quality than last weeks. Using GCP's has an impact on the quality, however, I do not really know the impact the different sensors play here. This week's images were taken with Zenmuse X5 while last week's imagery was taken by an E90.

Figure 28: This week's map has higher accuracy and greater

Last week's results are shown below in Figure 29.

Figure 29: Last week's map

In Figure 30, just to illustrate that pattern, even when zooming in to the exact same scale as last week, the same conclusion is met. With the usage of GCPs, the accuracy increase and has an impact on the quality. An important note is that the usage of GCP does not automatically improve the quality.

Figure 30: This week's map, zoomed in.

Conclusion

GCPs are a vital tool for precision mapping, but the majority of UAV maps do not require GCPs. In order to save money and time, it is of importance to understand when to use GCPs and when to skip them. In general, if the specific project requires a high degree of global accuracy one should use GCPs. This is important when orthorectifying the map.

Since a high level of global accuracy is important for surveyors, they usually use GCPs. For instance in property boundary surveys. Occasions, when GCPs are not necessary could, for instance, be when one uses a map to take volumetric or linear measurements within a map to measure the stockpile volumes or the length of a fence.

A conclusion is that GCPs may not always do much to increase the quality of the measurements, but either way, they are useful to prove that the measurements are accurate. There is a positive correlation between high-resolution images and stitching accuracy. Thus, as long as the resolution of a map is good, the margins of errors for measurements are going to be the same with or without the use of GCPs.

When Pix4DMapper processes imagery into a map, it uses camera GPS information on the images. The quality of a map is statistically expressed as the average error between where the GPS said the camera was and where Pix4DMapper calculates the camera needed to be in order to make the overlapping images stitched together into a map.

In contrast to last week's report when we worked without GCPs, this week really got us some statistical numbers to examine. There are a great number of statistical numbers to compare accuracy in the report. The internal orientation that regards the camera and external that also take the camera positioning into account. I would focus on the Root Mean Square Error (RMSE). The X,Y and Z values that refer to the RMSE for all of the individual cameras in the X, Y or Z directions and the total RMSE are the errors across all three different directions.

GSD (Ground Sample Distance) refers to the number of pixels per centimeter. The higher the number of pixels per centimeter, the greater the accuracy of the map. The report gave me the value of 1 x GSD (2.09 [cm/pixel]) for the DSM and Orthomosaic Resolution. That is the equivalent of 2.09 cm of accuracy.



Processing time

In Table 2, last week's orthomosaic generation, without GCPs, took about 30 minutes.

Table 2 Last week's orthomosaic generation, without GCPs, took about 30 minutes.

As Table 3 displays, this week's processing an orthomosaic took just below 15 minutes.

Table 3: This week's orthomosaic, with GCPs. Processing time stopped just below 15 minutes.

Quality in terms of accuracy

Relative Accuracy is generally for projects like taking small-scale measurements, visual inspections, and monitoring stuff. Absolute accuracy becomes important when you need a high degree of confidence that the latitudinal, longitudinal and elevation measurements are correct. The Relative accuracy is as follows from Table 4 below. The Absolute accuracy was according to Pix4DMapper like presented in Table 5. As a rule of thumb, the elevation accuracy of a map will be around 3 times worse than its horizontal peers.

Table 4: Relative Geolocation Variance

Table 5: Table 2: Absolute Geolocation Variance

Not sure how important this statistical measure is when it comes to quality reporting but Table 6 and 7 displays the relative vs absolute uncertainties respectively.

Relative XYZ uncertainties are at about 4 cm, 3 cm, and 2 cm respectively.

Table 6: Relative camera position and orientation uncertainties

Absolute XYZ uncertainties are at about 7 cm, 6 cm, and 14 cm respectively.

Table 7: Absolute camera position and orientation uncertainties

Table 8 illustrates that all 69 images were used and calibrated. A key take in comparison from last week was that the relative difference between the initial and the optimized parameters than was 15.59% in comparison to 0.84% now. This says that working with GCPs improve the data significantly. The need for optimization is much bigger working without GCPs. 6 GCPs were used this week and in terms of georeferencing, the RMS error is less than 3 centimeters. Since we did not use GCPs last week, sadly there are no numbers som compare with here. I would imagine that this number is the key average quality of the work done.

Table 8: All 69 images were enabled and calibrated. Accuracy landed on 2.9 centimeters.

Processing Image data in Pix4D (without GCPs)

Introduction

Today our focus was on generating a point cloud data set, a true orthomosaic, and a digital surface model using Pix4Dmapper. We processed our image data with the Pix4D software and then made maps with the generated data in ArcGIS Pro. Also, ArcScene was used to create 3D effects.

What is Pix4D?

Pix4D is a company that produces software tools for a variety of different fields. For our case, the drone mapping & photogrammetry software tool that includes a flight app was the most relevant. Their products are widely used in surveying, construction, mining etc since Pix4D products are very user-friendly for these purposes.

What Products does it generate?
It can create video animation trajectories. Like we will see below, the user has the option to choose his own trajectory by deciding the order of waypoints. The quality of the video rendering can also be chosen, fitting the need. If one is out in the field but just want the result of the imaging collection, one could go for the fast but low-resolution option, the Rapid Check. Saving the real encoding and high quality for later processing. The software also gives the option of saving the coordinates from the flying route in a specific csv-file.

Pix4DMapper can process imagery from multiple flights. However, the pilot needs to pay attention that each plane captures the images with enough overlap. Also, weather conditions are key, big differences will cause trouble.

Not only nadir images is possible to use, but also oblique images is doable. especially in terrain with elevation, it could be useful. Using oblique images, the angle should be between 0 and 45 degrees in both tracks to the center of the corridor.

Why is it so integral to UAS data processing?

With Pix4DMapper one can do so many things in an easy way. This will be demonstrated throughout this report. To start with some basics. For most image collections, it is suggested a frontal overlap of at least 75% and no less than 60% side overlap between the flying tracks. For the sake of quality, it benefits if the camera is maintained at a constant height to ensure desired ground sample distance. Reservations have to be made, sometimes it is hard to map terrains like sand and snow. Also, areas with great elevation differences should be carefully selected.

Methods

This section will illustrate the process, working with Pix4DMapper. In Figure 1, below is the basic workspace.

Figure 1: The workspace in Pix4DMapper.

The key metadata we are working with, presented in Table 1 below.

Table 1: Table with metadata for this UAS mission.

As one can see in the specification tree in Figure 2 below, Pix4DMapper structure the content in terms of Cameras, Rays, Tie points, Point Clouds, Triangle Meshes and Objects with their associated subfolders. This structure makes it very effortless to localize.

Figure 2: Pix4DMapper structure

Examining the Quality report, one can find many interesting features. At the bottom of Figure 3 below one can see the differences between the initial image positions (blue dots) and the green, which are the new computed tie points positions. The ellipses display the uncertainty.

Figure 3: The initial blue and the processed green tie points displayed.

In terms of camera specifics, one could name a few things. If the camera has a global shutter, all light information is read at the same time for the whole sensor. The linear rolling shutter process the light information differently in the way that its image is scanned line by line. In Figure 4 below, one can see the linear rolling shutter processed for this flight. The green lines are the computed image positions while the blue dots are the camera position at the start of the exposure. Bottom left is a hole, this is where the one image that was not successfully calibrated belongs. It can be spotted as the red dot in Figure 3 above and in Figure 5 below.

Figure 4: The linear rolling shutter processed for this flight. The green lines are the computed image positions while the blue dots are the camera position at the start of the exposure. Bottom left is a hole, this is where the one image that was not successfully calibrated belongs. It can be spotted as the red dot in the Figure below.

Figure 5: Watch out for red dots, which indicate disabled or uncalibrated images.

GCPs are not necessarily required for processing imagery with Pix4Dmapper. However, they would have increased the absolute accuracy significantly. The usage of GCP is highly recommended when processing projects no image geolocation. GeoTIFF is preferred using as a metadata standard when working with UAS data. It allows georeferencing information to be embedded in the TIFF image files.

Results and Discussion

Below is the result of the video I created with Pix4DMapper.

Video 1: My video, created with Pix4DMapper

In order to display come caveats of the processing in Pix4DMapper, I decided to use ArcScene. My purpose was to see in what degree Pix4DMapper handled areas with elevation or dense wood. By using the Hillshade effect in ArcScene, troubled areas, where the processing wasn't that good are easier to discover. Figure 6 re-illustrates the process from an earlier lab.

Figure 6: I dedicate an image of this great ArcScene feature for the memory. 

ArcScene will reveal the drawback of Pix4DMapper. As it comes out from Figure 7, without many images with good overlap, good elevation data will simply not get displayed properly. Looking at those steep and plain areas to the left and right of the house acknowledge this. Also, the uncalibrated image, remembering the red dot, was located in this area.

Figure 7: Plain area(s) are problematic in terms of elevation is not processed correctly.

Last week we learned how to create orthomosaics and maps in ArcGIS Pro. This week we used Pix4D for point cloud creation and from there stitch the imagery together, which proved to be superior quality. Before using Pix4D, one could think that the quality and the accuracy for ArcGIS Pro was good.

Figure 8 displays the two maps that I created out of the data processed through Pix4DMapper. On the grey scale, Digital Surface Model one see the problematic areas very clearly, this is where Pix4DMapper failed to process the data. Where the software has done a great job, there is also very detailed information about the ground. Vegetation, cars and even human are spotted.

Figure 8: Digital Surface Model, with interposed visuals from ArcScene, showing Dr. Hupy's house from different angles.

The bottom-left image in Figure 9, one sees clear differences in quality in comparison to Figure 10. However, we haven't worked with GCPs as of yet nor oblique imagery so the quality and accuracy may continue to improve.

Figure 9: Imagery and map creation with Pix4D resulted in a high-caliber quality.

Figure 10 represents the results from last weeks processing, which we made in ArcGIS Pro.  In reality, there were no trees covering the red car. With greater accuracy of the process, these blunders were corrected.

Figure 10: Imagery accuracy with ArcPro was not so good in comparison with Pix4D.

This looks good, right? But delving deeper in the statistics it turns out that it is not as good as one would have hoped. The relative geolocation variance is as follows from the table below. According to Table 2, the accuracy on the ground is set to 5 meters, while the elevation may vary as much as 10 meters. Also, this number increases when flying in areas with greater height variances, since the uncertainty increases.

Table 2: Elevation accuracy (z-value) is a real problem.

Table 3 displays the processing time to be just over 30 minutes. Table 4 illustrates that 67 images were used but only 66 was calibrated.

Table 3: Processing time stopped at half an hour.

Table 4: 67 images were enabled, 66 calibrated.

Conclusions

Focus lay on generating a point cloud data set, a true orthomosaic, and a digital surface model using Pix4Dmapper. We processed our image data with the Pix4D software and then made maps with the generated data in ArcGIS Pro.

Without many images (5+) with good overlap, elevation data simply will not get displayed properly. In this report, the process has been gone through very thoroughly. Looking at plain areas acknowledge this.

Figure 11: Processing an orthomosaic in ArcGIS Pro took about 8 minutes.

Remembering from last week, Figure 11 processing an orthomosaic in ArcGIS Pro took about 8 minutes. A drawback using Pix4DMapper is that in order to create greater results, it takes a longer time to process. Is it worth it? The user's need has to tell.