Consistent processing failure immediately after substep Camera Calibration?

1498
6
03-30-2019 02:00 AM
JasonKovacs
New Contributor

So far in my trial of D2M, I've only managed to complete one 2D project. Since then, I consistently get an Unknown Exception Error which according to the log occurs immediately after (or during?) substep Camera Calibration during the initial processing phase. It happens whether I am attempt to produce purely 2D products at about the 29% completion mark, or 3D products, although the exception error seems to occur later in the processing of a purely 3D products attempt at 80% completion. I've attempted multiple slight variations of processing options, mostly adjusting keypoint imaging scale and targeted number of keypoints, but also with the full default options as used with my one and only successful processing attempt (the first dataset processed). I have no yet attempted to reprocess an identical dataset as my one successful processing attempt to see if the problem now occurs there too, as that dataset is quite a bit larger and it would take roughly 5 hours or so to reach the failure point.

Here is the relevant section from the log: 

2019-03-30 00:56:20,513 INFO - {"exit":{"code":"0","errorMessage":""},"status":{"currentStep":"initial","detailedMessage":"Compute number of matches.","overallProgress":"24","substep":"Georeference the model"},"timestamp":"1553921780513"}
2019-03-30 00:56:20,746 INFO - {"exit":{"code":"0","errorMessage":""},"status":{"currentStep":"initial","detailedMessage":"Substep Georeference the model finished.","overallProgress":"24","substep":"Georeference the model"},"timestamp":"1553921780746"}
2019-03-30 00:56:20,750 INFO - {"exit":{"code":"0","errorMessage":""},"status":{"currentStep":"initial","detailedMessage":"Substep Camera calibration started.","overallProgress":"29","substep":"Camera calibration"},"timestamp":"1553921780748"}
2019-03-30 00:56:24,973 INFO - {"exit":{"code":"0","errorMessage":""},"status":{"currentStep":"initial","detailedMessage":"Substep Camera calibration finished.","overallProgress":"29","substep":"Camera calibration"},"timestamp":"1553921784972"}
2019-03-30 00:56:25,099 INFO - {"exit":{"code":"1","errorMessage":"Caught unknown exception during initial processing!"},"status":{"currentStep":"initial","detailedMessage":"Substep Camera calibration finished.","overallProgress":"29","substep":"Camera calibration"},"timestamp":"1553921785099"}
2019-03-30 00:56:25,123 INFO - {"exit":{"code":"1","errorMessage":"Caught unknown exception during initial processing!"},"status":{"currentStep":"initial","detailedMessage":"ProjectLoaded","overallProgress":"29","substep":"Camera calibration"},"timestamp":"1553921785123"}
2019-03-30 00:56:25,150 INFO - {"exit":{"code":"1","errorMessage":"Caught unknown exception during initial processing!"},"status":{"currentStep":"initial","detailedMessage":"Aborting ...","overallProgress":"29","substep":"Camera calibration"},"timestamp":"1553921785150"}

Any help is greatly appreciated. Thanks --Jason

Tags (1)
0 Kudos
6 Replies
CodyBenkelman
Esri Regular Contributor

Jason

Can you tell us more about your project data?  And/or, is it possible to send us a complete set of images?

A few specific questions:

1) You've said "consistently get error" - How many different projects have you tried?  e.g. "I've flown 10 drone projects, #1 works, the other 9 all fail" or do you have one project that you've tried and tried, changing configurations, and that one project always fails?

2) are you using ground control?

3) can you look at your one successful project, open the Processing Report, and let us know the "Camera Optimization" results?   e.g. "XX% relative difference between initial and optimized internal camera parameters

4) is this a DJI drone?

Some of the DJI drones report very poor values for flight altitude, and if you have that AND you don't have ground control, the software has to guess at the focal length and it can lead to some wildly erroneous values.  (I'm not trying to blame DJI for the software errors, but just seeking to give you advice on how to move forward)

Thx

Cody

JasonKovacs
New Contributor

Hi Cody, thanks for the reply. I've been traveling internationally, and the internet at my location has just gotten sorted out (for the better), hence my delay in reply.

1) I have flight coverage of two areas at the moment (new DJI Phantom 4 Pro V2.0, trialing Drone2Map), with one of the areas flown with two different profiles, one for 2D and the other for 3D products. This area is not large, perhaps 30m by 30m, with a grid flight that yielded 90 pictures at 75% front/side overlap. An additional ~60 photos were acquired at oblique angles to capture a target for 3D testing. Five Ground Control Points were utilized. The success processing was purely for a 2D orthomosaic, in an area roughly 6 acres in size and consisted of around 300 pictures with 75%side/70%front overlap (if I remember correctly), with 11 GCPs. However, some glitch in Collector caused all elevation values to become 0.0 in both areas. I did in fact see some unusual altitude measurements from the DJI drone; it turns out, this Phantom take a barometric measurement to obtain the altitude recorded in the metadata (or so I am told), so I need to build an IMU calibration into my checklist workflow prior to mapping flights. I definitely noted a horizontal misalignment of the calculated Control Points in the final orthomosaic versus the actual GCPs in the imagery, roughly 60-80 cm, which I attribute to the very poor elevation data as reported in the metadata (over by some 15 or so meters). Obviously, the calculated elevation of the landform was problematic as well, but relative to points within the orthomosaic, the results seemed reasonably accurate. We're still learning the ropes, obviously.

I do have another, older flight without GCP and on an older DJI Phantom which I could try, but have not done so yet. The failures occur in both 2D mapping imagery and 3D imagery at another, much smaller location than the successful attempt. I have a second batch of pictures from the first area, taken via a different flight planning app, which I also need to test still to see if they fail or succeed.

2) I am using GCPs, however, as noted, elevation data was recorded as 0.0m for all points due to some unknown flub in Collector. Our GIS person is still investigating. For the trial which failed to process, I obtained elevation data via an ESRI service for the individual GCPs, which I then used to update the GCP table in Drone2Map. 

(Perhaps related, I also corrected the elevation data in the drone pictures' metadata using a similar method to bring the elevation down to what was actually flown. I tried processing both before and after editing the metadata, without success either way.)

3) The successful processing report showed 90.58% camera optimization, which yielded a red triangular warning symbol. Obviously, I figured out that that cannot be a good thing to see, but I don't yet know why, exactly, or what influences that rating. As I said, we're still figuring this all out. It also gives a georeferencing RMS error of 0.249m, which also gave the same red warning symbol. All other quality checks were green.

4) Yes, it is a DJI Phantom 4 Pro V 2.0

I'll have to inquire as to whether I can post the photos to a sharing site for outside parties. I don't own them; my company does.

Thanks for your help,

Jason

0 Kudos
JasonKovacs
New Contributor

Follow-up: I have reprocessed my originally successful dataset, and now also get the error at the same time stamp, just after (or possibly during) camera calibration. Attached is a screenshot showing where in the in-app log the error occurs and the processing aborts.

I'll be running a test of an orthomosaic I made via DroneDeploy, using photos from an older DJI Phantom 3 and without ground control, see if the error is persistent there as well.

Again, any help would be greatly appreciated. I am demo'ing this software for my employer, and thus far, despite ESRI being the basis of ALL our other GIS applications, I'm having a hard time convincing myself to make an affirmative recommendation to purchase D2M. It seems like it'll do everything we'll ever need and more, but if I can't run it to test it, it's like buying a vehicle sight-unseen and hoping for the best.

Thanks,

Jason

0 Kudos
MikiYAMAMOTO
New Contributor II

Hello Jason,

 I wonder...

- if you locate images and project in your local drive. 

- if you have enough free spaces in your storage which the project located.

- if you have run other big applications at same time (CPU). 

I cannot explain well at this moment why the project once processed successfully fails other time but some conditions would be different all the time and the differences may cause some issues even though you use same PC to process.

Regarding the answer 3) posted on 2019/04/08 15:19,  I wonder (if you use a dataset of field like above) ...

- if you have processed with using "Calibration Method: Alternative" ("Use Geometry Verified Matching" under "Matching Strategy" will be checked-on automatically btw) that is optimized for dataset captured flat and homogenous areas such a grass field. (related to an error appearance of "Camera Optimization" under "Quality Check" in Processing Report).

- if all GCPs are used in the process. If the accuracy of GCPs is low or images that linked with GCPs are blurred, it sometimes fail to verify using GCPs  (related to an error appearance of "Georeferencing" under "Quality Check" in Processing Report).

Hope these tips would be helpful,

Good luck!

Miki

0 Kudos
JasonKovacs
New Contributor

Thanks for the tips Miki!

In regards to file storage, my laptop (not ideal; a proper workstation is in the works) has around 100 GB left. The successfully processed project folder is roughly 15.7 GB in size. So unless D2M is using around 80 GB of temp disk storage in it's processing, I should be OK.

All other applications are turned off to maximize CPU and RAM availability for D2M, including Chrome and usually internet all together. I also verify that most everything that can be turned off is so, prior to attempting to process.

The area in the attached screencapture is indeed a mostly flat, grass field. Some relief along it's northern edge, but not much, less than 3 meters total elevation gain from highest spot in the field to lowest. I will attempt a processing run using "Calibration Method: Alternative" as you suggested and see what happens.

All of the GCP have been manually linked to 7-10 photos each. As for their recorded location accuracy.... no idea. I utilized Collector with a sub-meter Bad Elf GPS unit. For quick points, accuracy is usually observed as +/- 4 m. I set it up to record an average over 30 seconds, which generally brings the accuracy error down significantly, according to our GIS person, although I do not know the magnitude. We did have a weird error where the elevations for all GCP came back as 0.0 m; I was able to calculate a ground surface elevation for each GCP from an ESRI DEM online service, however.

The individual images aren't blurry, but zooming in on the GCPs does show some loss of resolution. If I'm manually linking GCPs in photos, dropping an X myself to indicate the center of each target, is D2M still attempting to calculate where the crosshairs should be? It seems redundant and unnecessary if this is the case, but if so, I can see the program having issues deciding where exactly the center of the GCP is.

As to whether D2M is using each of the eleven GCPs .... How would one find that out for certain? 

Thanks again,

Jason

0 Kudos
JamesStewart-Moore1
New Contributor III

Were you able to resolve the issue?

0 Kudos