topic Re: CROSS VALIDATION COMPARISON in ArcGIS GeoStatistical Analyst Questions

CROSS VALIDATION COMPARISON

EifLeffie — Sat, 22 Oct 2011 07:57:31 GMT

Dear All,

I am modelling a phenomenon using Geostat Analyst particularly the Kriging technique. I generated several models and is using the cross validation comparison to determine the best model (out of more than 20 models that I have generated). What I am doing is comparing the prediction error parameters (e.g. root mean square, everage std. error, etc.) of one model against the other. However, as I have many models, the process is taking some time. I would compare them two by two (manually) and then compare the best output with the other models.

I wonder if there is any other way (technique or method - using all the models at the same time and not two by two as I am currently doing) to facilitate the comparison between models that will result with the best option.

Thank you so much for any help you could give.

Cheers,
Eif

Re: CROSS VALIDATION COMPARISON

SteveLynch — Sun, 23 Oct 2011 12:27:20 GMT

If you are using ArcGIS 10 you could write a short Python script

see http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//00300000000z000000.htm

Steve

Re: CROSS VALIDATION COMPARISON

EifLeffie — Mon, 24 Oct 2011 00:05:37 GMT

Thanks Steve for the reply. I have not used this utility yet as I am not that familiar with programming but I think it's worth a try.

Best regards,
Eif

Re: CROSS VALIDATION COMPARISON

SteveLynch — Mon, 24 Oct 2011 14:26:23 GMT

You can also, if multiple geostatistical layers are in ArcMap, right click on a geostatistical layer and choose Compare.

Re: CROSS VALIDATION COMPARISON

EifLeffie — Sat, 29 Oct 2011 03:12:35 GMT

Hi Steve,

Thanks a lot for the reply. What I did was to copy all the parameters in the prediction error of each model and encode in excel. The process is a bit tedious but I can be sure of which model I could choose (eg. model with lowest mean, Root Mean Sqr and Ave. Std. Error is a bit large but gap is small, etc.).

It would be easier if only one parameter (e.g. mean) is to be compared, however, in choosing the best model, we have to consider the RMS and ASE values, gaps, etc.).

This I guess cannot be done in ArcGIS as this needs manual analysis and would be based from the best jugdment of the user.

Thanks a lot and have a great weekends!

Cheers,
Eif

Re: CROSS VALIDATION COMPARISON

MeganWolf — Fri, 24 Feb 2012 16:11:43 GMT

I need to do a similar batch cross validation. I am having trouble with writing a script that incorporates all of the statistics associated with cross validation. I am able to print the Root Mean Square error but am not able to produce Mean Prediction, Average Standard Error, Standardized Prediction Error and the Root Mean Square Prediction Error.

Thanks in advance for the help!

Re: CROSS VALIDATION COMPARISON

SteveLynch — Fri, 24 Feb 2012 16:48:23 GMT

These are interpolation method dependant. IDW, for example on has error. Kriging has many more.

Steve