Compute Accuracy For Object Detection

[RDAltarez]

Hi there!

I'm just wondering if anyone has encountered a problem with the "compute accuracy for object detection" tool.

I am new to object detection. I have tried to detect ships/marine vessels using a Sentinel-1 VH Sigma0 in an 8-bit GeoTIFF file. Using the export training samples, I produced over 700+ training chips (256 tile size and 128 stride) and labels. I successfully created a model using FasterRCNN - ResNet50 backbone, 20 epochs, batch size 16, validation 10%, and a learning rate of 0.0001 (I chose this so I can compare the result with other model types).

When I used the "compute accuracy for object detection" tool to check the validity of the output, it showed poor results (see below). Note: I have 84 validation polygons, which is actually equivalent to almost all the total ships found in the imagery.

Inspecting the results visually shows that the model is fairly accurate at detecting the ships/marine vessels (see below, the red rectangle indicates the detection, and the smaller filled rectangles represent the validation samples):

I used Non-Maximum Suppression, but it only resulted in a fairly decent improvement in accuracy. I also tried other model types with the same parameters as above, but the Faster R-CNN has provided better results so far.

Any suggestions to improve accuracy would be highly appreciated.

Thank you!

[RDAltarez · Accepted solution]

Update: I found a way to improve the accuracy. So the tool is very sensitive to the size of the validation samples vs the size of the detected object (features produced from the "detect object..." tool). These links are good read: website 1, website 2, and website 3.