nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How to normalize feature data between 0-1 in ArcGIS Pro?

HuangleiPan1

Hi,

I have a feature table, and want to normalize the data in a field between 0-1, and I know the normalization method should be "(x-min(x))/(max(x)-min(x) ". I have tried "add field---field calculator", but I have no idea how to use the functions. Could anyone help me to achieve it in ArcGIS Pro?

Thanks

Encontre mais publicações marcadas com

Accepted answers

DanPatterson_Retired

To atone for recommending a spreadsheet.... I have provided a quick demonstration using numpy and the builtin arcpy.da module TableToNumPyArray and NumPyArrayToTable functionality... which I wish was used and advertised more. It is lonely in the numpy/arcpy world... must develop a web widget for this stuff

/blogs/dan_patterson/2019/03/18/math-and-stats-with-numpy-normalize-data

All comments

HuangleiPan1

Dan, I find a field of my data couldn't been processed by the script, the results for it were all Null. However, the datatype(double) and other properties of this field are same as the others . The only difference is that the data of this field were calculated from some other fields using Field Calculator Tool. The warning message is below:

Warning (from warnings module):
File "C:\Users\pan\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone2\Lib\site-packages\numpy\core\fromnumeric.py", line 83
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
RuntimeWarning: invalid value encountered in reduce

I only know some basics of Python, so I can not figure out the problem here by myself.

Do you know why the error came?

DanPatterson_Retired

Curtis the statistical and image image stretch are both forms of Normalization - Wikipedia

I prefer to think of it as the process by which "a basic standard by which value is computed" (Numéraire like currency) since an upper range is not necessarily required but a 'base' of comparison is required

curtvprice

A more traditional approach (still a lot less work IMHO, sorry Dan):

1. Find the min and max values, let's say they are 10 and 900. (right click field in table view, Statistics)

2. Add a field

3. Run Calculate field with this expression (VFLD is your field that ranges from 0 to 900) (Use Python parser)

min + (max-min) * (value - 10.0) / (max-min)

<SPAN class="number token">10</SPAN> <SPAN class="operator token">+</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="number token">900</SPAN> <SPAN class="operator token">-</SPAN> <SPAN class="number token">10</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="operator token">*</SPAN> <SPAN class="punctuation token">(</SPAN>!VFLD! <SPAN class="operator token">-</SPAN> <SPAN class="number token">10.0</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="operator token">/</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="number token">900</SPAN> <SPAN class="operator token">-</SPAN> <SPAN class="number token">10</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

I should mention that what you are suggesting is not "normalizing" the data but instead doing a min-max "stretch" of the data. To Normalize the data you would convert the values to Z scores using the mean and standard deviation (replace mean and std with the numbers you get from the Statistics run):

<SPAN class="punctuation token">(</SPAN>!VFLD! <SPAN class="operator token">-</SPAN> mean<SPAN class="punctuation token">)</SPAN> <SPAN class="operator token">/</SPAN> std<SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

DanPatterson_Retired

Glad it worked..

I updated the blog to add examples if you needed to normalize data by column (as you need), but also by row or overall, depending on how your data are aranged or what it represents. Keep it in mind. It would be a bit beyond the field calculator, but arcpy and numpy interplay quite nicely to solve problems

HuangleiPan1

Thank you very much, Dan! After reading your script, I have solved the problem !

DanPatterson_Retired

/blogs/dan_patterson/2019/03/18/math-and-stats-with-numpy-normalize-data

DanPatterson_Retired

If the fields are numeric... and you have a very large number, then I would suggest you export the table to a spreadsheet, normalize the data there, then rejoin the data back to the original file. You will have to make sure that after you normalize there, that you copy the results with non-duplicate field names... and no formula etc to a fresh clean spreadsheet then use the Excel To Table tool to bring it back into pro.

There is no 'batch' normalize function in Pro and scripting would take you far longer than copying and pasting a formula in excel to do the work.

My preference would be to do the normalization using numpy and python, but I suspect that isn't an easier option for you.

HuangleiPan1

Thank you for you kind reply, Evgency. However, I have tens of fields to be normalized. If I convert all of them to rasters and then use the Raster Calculator Tool, it would be tedious. Could there be other methods to excute directly on feature dataset?

EvgenyPanchenko1

Hello Pan,

you'd need to use the Raster Calculator tool to employ this formula.

Participe da Conversa

Membros conectados podem postar, seguir atualizações e mais. Novo aqui? Registre uma conta gratuita.

Links Rápidos

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.