topic Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy in Python Questions

perform IDW interpolation to select multiple Z field from attribute table using ArcPy

ShouvikJha — Sat, 11 Dec 2021 08:54:45 GMT

I have 12 point feature datasets for different month. I want to perform IDW interpolation.

I have created the python code but its taking only one "Z" field for different month feature datasets from the attribute. Below I have mentioned the code. How to assign here multiple Z field from each month folder (JANUARY, FEBRUARY......DECEMBER).

Point feature class from different month folder (e.g. For January Merged_001, for FEBRUARY Merged_002.......for DECEMBER Merged_012 ) and want to perform IDW interpolation and save it as on same month folder as it Z field name (Max_Temper, Min_Temper, Precipitatat, Wind, Relative_H, Solar) with month (e.g for January 001_Max_Temper, 001_Min_Temper, 001_Precipitatat, 001_Wind, 001_Relative_H, 001_Solar).

After Whatever Solar raster for different month (for January 001_Solar....for DECEMBER 012_Solar) we will get, I want to do some calculation (001_Solar * 30) *0.5 and save it 001_sr ........012_sr in Month wise folder.

Below i have attached Point shapefile also for reference purpose

Here is my code.

import arcpy
from arcpy import env
from arcpy.sa import *
arcpy.env.parallelProcessingFactor = "100%"
arcpy.env.overwriteOutput = True
# Check out the ArcGIS Spatial Analyst extension license
arcpy.CheckOutExtension("Spatial")
env.workspace = "D:\SWAT-WEATHER-DATA2\APRIL"
# Set local variables
inPointFeatures = "D:\SWAT-WEATHER-DATA2\APRIL\Merged_004.shp"
zField = "Min_Temper" #Z_Filed = Max_Temper, Min_Temper, Precipitatat, Wind, Relative_H, Solar
cellSize = 0.002298707671
power = 2
searchRadius = RadiusVariable(10, 150000)
#Mask region of interest
mask="D:\Gujarta Shape file\GUJARATSTATE.shp"

# Execute IDW
outIDW = Idw(inPointFeatures, zField, cellSize, power, searchRadius)
# Execute Mask
IDWMASk = ExtractByMask(outIDW, mask)
# Save output, except Solar Radiation raster
IDWMASk.save("004_Min_Temp.tif")
# 004_Max_Temp.tif, 004_Min_Temp.tif....Solar
#Only for Solar Radiation raster
#PAR = (IDWMASk * 30) * 0.5
#Save output, Only for Solar Radiation raster
#PAR.save("003_sr.tif")
#print done
print 'done'‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

DanPatterson_Retired — Sun, 21 Aug 2016 18:36:20 GMT

just a side note... you file paths need to be in raw format, at least (r'c:\path' ), they shouldn't contain spaces nor any character other than letters, numbers and the underscore... dashes can be problematic. The rules regarding grid workspaces are equally onerous, but I will leave that for a bit

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

XanderBakker — Sun, 21 Aug 2016 20:04:38 GMT

I downloaded the shapefile that contains the data for month 004 (April), but I noticed that you have about 514 point locations and each point location contains data for each day of the month. In this case you have 30 points at each location. When you do an IDW interpolation these values will overlay and also may make the interpolation process slower than necessary. You may want to consider summarize the data (average min and max temp, etc) for each unique location.

Also the distribution makes me think that the data originally comes from a raster format.

Interpolating this data as if they were individual stations, doesn't feel right. Perhaps converting them to raster (after the aggregation per location) with one location per pixel, would be a better idea. If you require a more detailed resolution you could use the Resample—Help | ArcGIS for Desktop tool, but remember that you really don't have more detail then those input points.

The get an example of how to process and calculate based on lists of data, you could have a look at this thread: Calculate list of raster from two different subfolder using ArcPy where some examples are provided.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

ShouvikJha — Mon, 22 Aug 2016 09:00:49 GMT

Xander Bakker‌. Thank you. the point data comes from model generated output and they provides the datasets in csv format.

I don't understand, how to perform (after the aggregation per location with one location per pixel) on those data to calculate Mean value of each location for different parameters. My kind request, please guide me to the task with accurately.

Regarding interpolation daily data into a single pixel, Arc GIS has the default setting in Envs Setting: Geo-statistical Analysis : Coincident Points : Mean. So during interpolation i think its calculate the Mean value of different parameter for each location to crate a single pixel from daily data.

When i use the Arc GIS IDW tool for performing IDW then its taking some time but when i run the code in pyscripter, its not taking much time to perform interpolation.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

DanPatterson_Retired — Mon, 22 Aug 2016 09:32:33 GMT

Xander correctly points out that there is no need for interpolation since the input areal units and their output values have been previously defined from the source program... they just happen to be represented by points, but they reflect an area.

...

Perhaps converting them to raster (after the aggregation per location) with one location per pixel, would be a better idea

...

If you need a finer resolution, then that should be done in the source program. Interpolation of the data you have is not necessary or appropriate.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

ShouvikJha — Mon, 22 Aug 2016 15:35:12 GMT

xander_bakker‌ Dan_Patterson‌. Thank you for your kind cooperation.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

NeilAyres — Mon, 22 Aug 2016 16:01:55 GMT

To convert the points to raster, you would need to do a little fiddling with the point spacing and the origin.

Inputs to the Point to Raster tool, you would have to specify in the environments the exact cellsize of the output (equal to the spacing of the points) and also an extent which was exactly 1/2 of a cellsize offset to the west. So, in the output raster all the input points were at the centre of each output pixel.

But if you just wanted a quick picture of these points, I don't think using something like Natural neighbour interpolator would actually mess up the input data so much. And would save you a whole lot of messing about.

Perhaps Dan_Patterson would like to comment on the implications of this interp....

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

DanPatterson_Retired — Mon, 22 Aug 2016 16:16:15 GMT

One would have to use the same interpolator as that used to generate the points in the first place. There is no assumption given that the actual points represent just that, but could be a representation of an interpolation in the first place. I always get a bit squimish about interpolating interpolations which might have been interpolated in the first place, hence, my comment about going back to source and do it again.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

NeilAyres — Tue, 23 Aug 2016 06:12:44 GMT

If I get time, I might do some testing of interpolating an interpolation....

Will feed back here..

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

XanderBakker — Tue, 23 Aug 2016 15:44:45 GMT

I had a quick look at the IDW interpolation and if I use the following settings.

I get this result

The individual points are clearly visible and generate an undesired effect. To reduce this effect you can increase the search distance:

It becomes a little more smoother, but still the individual points are there. When the power is reduced to one the effect is this:

When you convert the points to raster with the "original" resolution you get this (hardly appealing) result:

If you resample this using Cubic Spline you will get this result:

The question is; which one is closer to the "truth"...

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

DanPatterson_Retired — Tue, 23 Aug 2016 17:49:19 GMT

leaving philosophy out of it... can one assume the original points represent the truth? and what errors does using that data introduce in itself.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

NeilAyres — Tue, 23 Aug 2016 19:30:22 GMT

What is the "truth".

Interpolation is interpolation. "Estimating", "predicting", values which are not actually there in your original data but putting some mathematical function through the input points to create a continuous surface raster output. Lots of stuff in the middle. IDW, NN, Splines, even Kriging

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

NeilAyres — Tue, 23 Aug 2016 19:32:04 GMT

By the way, thanks xander_bakker for giving us some nice pictures of the various "interpolations"

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

XanderBakker — Tue, 23 Aug 2016 20:33:13 GMT

You're welcome, but I'm sure there are many options to explore that will yield different results and there are many interpolations methods that will yield "better" results.

Also, very true your remark about estimating and predicting. If possible it is preferable to get back to the source (meteorological stations?). Hard to tell what kind of processes have been applied and on what data to obtain the results.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

DanPatterson_Retired — Tue, 23 Aug 2016 20:39:36 GMT

I will second that... especially it is climatalogical data

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

ShouvikJha — Sat, 11 Dec 2021 08:54:48 GMT

Earlier i had post the issues of taking multiple Z value to perfrom IDW. From IDW interpolation its seems to be not much accurate. I am trying to execute Universal Kriging (LINEARDRIFT). I have written one code to perform kriging, but its not running, And i am confused to select some variable for Kriging. Below i have attached point shape file, which i am performing Kriging.

Error massage from below code while performing Kriging

Message File Name Line Position
Traceback
 <module> D:\Arc-GIS-Python-Script\KRIGING_PAR1.py 31
 Kriging C:\Program Files\ArcGIS\Desktop10.1\arcpy\arcpy\sa\Functions.py 2405
 swapper C:\Program Files\ArcGIS\Desktop10.1\arcpy\arcpy\sa\Utils.py 47
 wrapper C:\Program Files\ArcGIS\Desktop10.1\arcpy\arcpy\sa\Functions.py 2397
 <lambda> C:\Program Files\ArcGIS\Desktop10.1\arcpy\arcpy\geoprocessing\_base.py 484
RuntimeError: Object: Error in executing tool‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Below is the code

import arcpy
from arcpy import env
from arcpy.sa import *
arcpy.env.parallelProcessingFactor = "100%"
arcpy.env.overwriteOutput = True

# Check out the ArcGIS Spatial Analyst extension license
arcpy.CheckOutExtension("Spatial")

env.workspace = r"D:\SWAT-WEATHER-DATA"

# Set local variables
inPointFeatures = r"D:\SWAT-WEATHER-DATA\SELECTED-EX-MERGE.shp"

zFields = ["Max_Temper", "Min_Temper", "Precipitat", "Wind", "Relative_H", "Solar"]

#Kriging Veriable
cellSize = 0.002298707671
lagSize = 0.5780481172534
majorRange = 6
partialSill = 3.304292110
nugget = 0.002701348
kRadius = RadiusFixed(20000, 1)

#Mask region of interest
mask="D:\Gujarta Shape file\GUJARATSTATE.shp"

# Execute Kriging
for zField in zFields:
    kModelUniversalObj = KrigingModelUniversal("LINEARDRIFT", lagSize, majorRange, partialSill, nugget)
    OutKriging = Kriging(inPointFeatures, zField, kModelUniversalObj, kRadius)
    # Execute Mask
    #IDWMASk = ExtractByMask(outIDW, mask)
    IDWKrig = ExtractByMask(OutKriging, mask)

    if not zField == "Solar":
    # Save output, except Solar Radiation raster
     IDWKrig.save("012_{}.tif".format(zField))

    else:
     #Only for Solar Radiation raster
     PAR = (IDWKrig * 30) * 0.5
     #Save output, Only for Solar Radiation raster
     PAR.save("012_{}.tif".format(zField))

#print done
print 'done'‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

ShouvikJha — Tue, 30 Aug 2016 11:20:47 GMT

Xander Bakker‌ Dan Patterson‌. Several researcher has been tested the issue of How effectively different interpolation techniques can predict the missing value based on known value, they found Kriging is working effectively to predict the missing value. And its very close to the observation. Thank you.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

DanPatterson_Retired — Tue, 30 Aug 2016 11:36:04 GMT

lets rule out potential sources of error first

file paths need raw format ie r":\c:\mypath" ... note the r
spaces are almost certain failure
dashes and other undesireables in the paths... like - dashes
indentation.... copy the code open up syntax highlighting, select python... then paste try using 4 spaces per indent level (we have blog posts on this issue... read them)
the input variables to kriging were derived appropriately and reflect the characteristics of the inputs? I can't test...
put in print statements to insure the input and output names are correct
check parallel processing factor, 100% as a string? or 100 as a number... don't have time to check
line 31 hmm off hand, doesn't like the mask (I think)

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

ShouvikJha — Tue, 30 Aug 2016 11:59:28 GMT

Dan Patterson‌. Thank you for the suggestions.. Regarding indentation issues with my posted code i don't know why indentation occurring even i posted code very carefully with syntax highlighting.

Re: perform IDW interpolation to select multiple Z field from attribute table using ArcPy

DanPatterson_Retired — Tue, 30 Aug 2016 12:06:10 GMT

try this /blogs/dan_patterson/2016/08/14/script-formatting