topic Re: Can't Re-Train ESRI Model in ArcGIS Image Analyst Questions

Can't Re-Train ESRI Model

rescobar — Wed, 22 May 2024 21:21:47 GMT

Hi Folks,

I'm trying to retrain this model: Land Cover Classification (Aerial Imagery) - Overview (arcgis.com)

I run label features for deep learning and then export training data for deep learning.

I then run train deep learning model, but get this error:

Exception: Could not infer dataset type. Please specify a supported dataset type or ensure that the path contains valid exported training data from ArcGIS.


UnboundLocalError: local variable 'data_bunch' referenced before assignment

Is there a key parameter I'm missing?

Re: Can't Re-Train ESRI Model

PavanYadav — Tue, 28 May 2024 19:57:31 GMT

@rescobar Can you please confirm that you followed steps outlined here https://doc.arcgis.com/en/pretrained-models/latest/imagery/finetuning-the-land-cover-classification-aerial-imagery-.htm?

how many image chips do you have in your training data?

thanks
Pavan

Re: Can't Re-Train ESRI Model

rescobar — Thu, 30 May 2024 22:26:23 GMT

Yes, I cannot get "Export Training Data For Deep Learning" to work out of the toolbox

How many image chips do I need? I have 500 chips using "Export Training Data For Deep Learning" from the "Label Objects for Deep Learning window".

If I use export training data for deep learning out of the tool box from a shapefile I get a different error:

ERROR 002088: Feature class has not enough information for supervised training.

Re: Can't Re-Train ESRI Model

PavanYadav — Fri, 07 Jun 2024 18:30:20 GMT

@rescobar your feature class looks right to me. Try to use the Class Value Field parameter. If it does not work, please share it as a zip, I can try to investigate the issue at my end.

Re: Can't Re-Train ESRI Model

rescobar — Mon, 03 Jun 2024 15:57:21 GMT

Hi @PavanYadav , see the data here:

rescobar_data

Thank you!

Re: Can't Re-Train ESRI Model

PavanYadav — Fri, 07 Jun 2024 18:38:37 GMT

@rescobar thanks I will look into it.

Re: Can't Re-Train ESRI Model

PavanYadav — Wed, 12 Jun 2024 17:53:03 GMT

@rescobar looks like rescobar_data has expired. can you please share again?

Re: Can't Re-Train ESRI Model

rescobar — Tue, 18 Jun 2024 15:02:47 GMT

Hi @PavanYadav here is an updated link. Thank you!

rescobar_data

Re: Can't Re-Train ESRI Model

PavanYadav — Mon, 29 Jul 2024 22:27:24 GMT

@rescobar I'm so sorry but I cannot access the data .. I am sure it's at my end.. can you please email me at pyadav AT esri dot com.. I will share my ftp or you can just try to share it as an email attachment? thanks

Re: Can't Re-Train ESRI Model

rescobar — Thu, 08 Aug 2024 22:16:18 GMT

Sorry, I've emailed you and shared with google drive rather than one drive:

Just following up on your comment on my post in the ESRI forum, I'm now able export training data, but am getting a different error.

My Training sample is here: https://drive.google.com/file/d/1VtMmMAqEKoerdlwOLfy0Ugq_ac__3EBo/view?usp=sharing

And the original data I was trying to send is here: https://drive.google.com/file/d/1OA_fkg4Q7BGwtJxO__RbxV2XpzFtmaDv/view?usp=sharing

Runtime Error:Error(s) in loading state_dict for DynamicUnet: size mismatch for layers.0.0.weight: copying a param with shape torch.Size([64, 3, 7, 7]) from checkpoint, the shape in current model is torch.Size([64, 4, 7, 7]). size mismatch for layers.10.layers.0.0.weight: copying a param with shape torch.Size([49, 99, 3, 3]) from checkpoint, the shape in current model is torch.Size([50, 100, 3, 3]). size mismatch for layers.10.layers.0.0.bias: copying a param with shape torch.Size([49]) from checkpoint, the shape in current model is torch.Size([50]). size mismatch for layers.10.layers.1.0.weight: copying a param with shape torch.Size([99, 49, 3, 3]) from checkpoint, the shape in current model is torch.Size([100, 50, 3, 3]). size mismatch for layers.10.layers.1.0.bias: copying a param with shape torch.Size([99]) from checkpoint, the shape in current model is torch.Size([100]). size mismatch for layers.11.0.weight: copying a param with shape torch.Size([9, 99, 1, 1]) from checkpoint, the shape in current model is torch.Size([4, 100, 1, 1]). size mismatch for layers.11.0.bias: copying a param with shape torch.Size([9]) from checkpoint, the shape in current model is torch.Size([4]).Training not successful.

Traceback (most recent call last): File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 397, in <module> execute() File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 392, in execute del training_model_object UnboundLocalError: local variable 'training_model_object' referenced before assignment

Thank you for your help!

Re: Can't Re-Train ESRI Model

rescobar — Thu, 22 Aug 2024 16:56:37 GMT

I was able to retrain the model by the following:

My raster had 4 bands, the model required 3, so I extracted the bands with the "make raster" tool.

I then classified the raster with the "label objects for deep learning" tool. I used this interface to save the training data to a GDB; this creates all the fields needed. I wasn't able to create a training data any other way. If there are fewer classes than what are required by the model, then this will cause an error later. The classes in the schema must also be numbered and named to match the classes specified in the model's emd.

With this feature class and my raster I ran "Export Training Data for Deep Learning". The resulting training data then worked for the "train deep learning model tool".