topic Pass a null result back to a dynamic map Service from a geoprocessing task in Geoprocessing Questions

Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Wed, 03 Oct 2012 05:23:08 GMT

Howzit All,

I have created a Geoprocessing Service that creates a Heat Map using the Kernel Density Tool. The service works well, and quickly when a feature selection is given to it. However, the tool fails when the feature layer selection returns no records and a null Raster Layer is passed back to the AGS Dynamic Map Service Layer.

Geoprocssing Overview:
- create a Feature Layer from a sql expression
- generate Kernel Density Layer
- set Layer into resulting Dynamic Map Service Layer as a Raster Layer

The task fails when a null Raster Layer does not exist and is passed back to the out Layer using.
arcpy.SetParameterAsText(6,outRaster)

The task succeeds when the heatMap is created from > 1 features in the selection.

Setting arcpy.SetSeverityLevel(0) ensures the task completes.

I think the problem occurs at the map services level because the task completes.

Question:
How can I pass a null Layer back to a Raster Layer in a AGS Dynmaic Map Service Layer from a Geoprocessing Result?

Susan

North South GIS
Auckland, New Zealand

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

ShingLin — Thu, 04 Oct 2012 16:40:44 GMT

Susan,

You can modify the code for output paramter so there is no result map service returning when the selections result is zero. Here are two options:

(1) If the selections result is zero, set the outputpath of the kernel density path to blank and do not execute the kernel density function. Also, set theoutput parameter as a derived output in the script tool.

Here is the code looks like

import arcpy, os
from arcpy.sa import *

inputval = arcpy.GetParameterAsText(0)

inputpts = r'c:\KernelDensity\data.gdb\Crime'
outputpts = os.path.join(arcpy.env.scratchGDB, "selectcr")
outraster = os.path.join(arcpy.env.scratchGDB, "KernelD_sele1")

query = "ID < " + inputval
arcpy.Select_analysis(inputpts , outputpts, query)

result = arcpy.GetCount_management(outputpts)
count = int(result.getOutput(0))

if count == 0 :
     arcpy.AddWarning("count = 0")
     outraster = ""
else:
     outKDens = KernelDensity(outputpts, "NONE", 45, 1200, "SQUARE_KILOMETERS")
     outKDens.save(outraster)

arcpy.SetParameter(1, outraster)

(2) Set the output parameter only when the the selection result is zero. Then set the output paramter as optional in the script tool.

import arcpy, os
from arcpy.sa import *

inputval = arcpy.GetParameterAsText(0)
outraster = arcpy.GetParameterAsText(1)

inputpts = r'c:\KernelDensity\data.gdb\Crime'
outputpts = os.path.join(arcpy.env.scratchGDB, "selectcr")

query = "ID < " + inputval
arcpy.Select_analysis(inputpts , outputpts, query)

result = arcpy.GetCount_management(outputpts)
count = int(result.getOutput(0))
arcpy.AddMessage(" The select count = " + str(count))

if count == 0:
     arcpy.AddWarning("count = 0")
     arcpy.SetParameter(1, "")
else:
     outKDens = KernelDensity(outputpts, "NONE", 45, 1200, "SQUARE_KILOMETERS")
     outKDens.save(outraster)

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Thu, 04 Oct 2012 21:38:51 GMT

Thanks for getting back about this one. I have tried passing back a null string. The tool completes by first setting the environment by arcpy.SetSeverityLevel(0)

#TODO: set Output
arcpy.AddMessage("***set The Output Layer")
if arcpy.GetCount_management("inFeatures1")>0:
    arcpy.AddMessage("SETTING VALID RASTER")
    arcpy.SetParameter(6,outRaster)
else:
    arcpy.AddMessage("SETTING INVALID RASTER")
    arcpy.SetParameter(6,"")

I am attaching the attached message when we run from the REST Endpoint.

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

ShingLin — Fri, 05 Oct 2012 19:51:30 GMT

Please note that the output parameter must be optional or derived, then setting the path to empty will work.

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Fri, 05 Oct 2012 21:20:21 GMT

Hi and Thanks for this,

We have been running with the derived output and then tried switching to the optional output with the same result.

There is no problem when the feature selection is greater than 0. The raster layer returns nice and beautifully into the dynamic map service layer at the conclusion of the script.

We have noticed that the following statement does not fire

#TODO: wont Fire
if arcpy.GetCount_management(lyr)==0:
arcpy.SetParameter(6,"")
sys.exit(0)

However, casting the parameter as a string will fire,

#TODO: will Fire
if str(arcpy.GetCount_management(lyr))=="0":
arcpy.SetParameter(6,"")
sys.exit(0)

It is probably another issue either with the syntax or the way the function is parsed through python.

I didn't get a chance to test this with ArcGIS Server 10.0, so cannot sayif this is new behaviour.

Susan

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

ShingLin — Mon, 08 Oct 2012 17:04:17 GMT

What version of server are you using?
My example is on 10.1 server. I have not tried previous version of server.

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Mon, 08 Oct 2012 18:43:54 GMT

We recently upgraded all the ArcGIS Infrastructure to ArcGIS 10.1 release.

I am not sure if the issue is in the scripting,but rather the way the parameter is parsed into the AGS Dynamic Map Service Layer which is giving problems.

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

ShingLin — Mon, 08 Oct 2012 23:42:15 GMT

Can you share the data/script you wrote so i can verify where may go wrong? Or just copy the script that I sent to you? I did not get any fails with the script that I replied to you.

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Wed, 10 Oct 2012 20:11:24 GMT

Attached is the complete script. There is a toolbox which I have not attached becaus eit is probably not necessary.

outRaster is a derived Raster Output Layer.

We pass a load of comma separated strings in the beginning and spend a bit of time building up the sql expression for inFeatures1.

Thanks for checking it out.

Susan

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

ShingLin — Fri, 12 Oct 2012 22:32:54 GMT

When you run your service with the query set to zero, did you see your "***set The Null Output" wriiten out? If not, can you try the approach in the code that I posted regarding to get count see if you get the message?

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Wed, 17 Oct 2012 02:47:38 GMT

Just checked the arcpy.SetParameter(6,"") using your recommendations. The output still generates an invalid return value back to the AGS Dynamic Map Service Layer. Pretty sure that the Geoprocessing serivce is successfully execution as it fails after the task has finished.

esriJobMessageTypeInformative: 0:00:02.254000
esriJobMessageTypeInformative: Completed script MEDKDInterpolatePOCwithparameters...
esriJobMessageTypeInformative: Succeeded at Wed Oct 17 16:45:28 2012 (Elapsed Time: 3.00 seconds)
esriJobMessageTypeInformative: Succeeded at Wed Oct 17 16:45:28 2012 (Elapsed Time: 3.00 seconds)
esriJobMessageTypeError: Invalid return value:

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Wed, 17 Oct 2012 18:51:53 GMT

Calling Script

--start Here --
arcpy.AddMessage(sqlQuery)

outRaster=arcpy.env.scratchGDB+os.path.sep+"heatMap"
#outRaster=arcpy.GetParameterAsText(3)
arcpy.MakeFeatureLayer_management(inFeatures,"inFeatures1",sqlQuery)
arcpy.AddMessage("feature Count:"+str(arcpy.GetCount_management("inFeatures1")))
descFeatures=arcpy.Describe("inFeatures1")
result=arcpy.GetCount_management("inFeatures1")
count = int(result.getOutput(0))
arcpy.env.mask=descFeatures.extent
arcpy.AddMessage(str(count))

#TODO: test For Selection
if count==0:
    arcpy.AddMessage("Error")
    arcpy.SetParameter(6,"")
else:
    arcpy.AddMessage("Continue")

    #TODO: process Kerenl Density
    processStart=datetime.datetime.now()
    arcpy.AddMessage("***generate Kernel Density Surface")
    arcpy.gp.KernelDensity_sa("inFeatures1", modelFld, outRaster, "", searchReadius, "SQUARE_MAP_UNITS")
    arcpy.AddMessage(str(datetime.datetime.now()-processStart))

    #TODO: check In License
    arcpy.AddMessage("***check In Spatial Analyst")
    arcpy.CheckInExtension("spatial")

    #TODO: set Output
    arcpy.AddMessage("***set The Output Layer")
    arcpy.SetParameter(6,outRaster)

Complete Messaging
Job Messages:
esriJobMessageTypeInformative: Submitted.
esriJobMessageTypeInformative: Executing...
esriJobMessageTypeInformative: Executing (MED_KDInterpolate_POC_withparameters): MEDKDInterpolatePOCwithparameters "Corinthian,Philip V" hok # # # Estimated_catch__gwt_ C:\Users\services\AppData\Local\Temp\med\medkdinterpolatepoc_demo_gpserver\je2f1a43da2ef49bc9585d464cf91e4c4\scratch\scratch.gdb\heatMap
esriJobMessageTypeInformative: Start Time: Thu Oct 18 08:44:54 2012
esriJobMessageTypeInformative: Executing (MEDKDInterpolatePOCwithparameters): MEDKDInterpolatePOCwithparameters "Corinthian,Philip V" hok # # # Estimated_catch__gwt_ C:\Users\services\AppData\Local\Temp\med\medkdinterpolatepoc_demo_gpserver\je2f1a43da2ef49bc9585d464cf91e4c4\scratch\scratch.gdb\heatMap
esriJobMessageTypeInformative: Start Time: Thu Oct 18 08:44:54 2012
esriJobMessageTypeInformative: Running script MEDKDInterpolatePOCwithparameters...
esriJobMessageTypeInformative: *** Kernel Density Generator with Parameters ***
esriJobMessageTypeInformative: ***check Out Spatial Analyst
esriJobMessageTypeInformative: "Vessel_Name" IN ('Corinthian', 'Philip V') AND "Species" IN ('hok')
esriJobMessageTypeInformative: feature Count:0
esriJobMessageTypeInformative: 0
esriJobMessageTypeInformative: Error
esriJobMessageTypeInformative: completed
esriJobMessageTypeInformative: 0:00:04.962000
esriJobMessageTypeInformative: Completed script MEDKDInterpolatePOCwithparameters...
esriJobMessageTypeInformative: Succeeded at Thu Oct 18 08:45:00 2012 (Elapsed Time: 6.00 seconds)
esriJobMessageTypeInformative: Succeeded at Thu Oct 18 08:45:00 2012 (Elapsed Time: 6.00 seconds)
esriJobMessageTypeError: Invalid return value: C:\Users\services\AppData\Local\Temp\med\medkdinterpolatepoc_demo_gpserver\je2f1a43da2ef49bc9585d464cf91e4c4\scratch\scratch.gdb\heatMap
esriJobMessageTypeError: Failed.

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Wed, 17 Oct 2012 20:43:43 GMT

Hi
perhaps the best workaround is to put some error trapping in to check for the failed task by the calling applicaton, in this case the javascript API.

It doesn't appear the GP Dynamic Map services can handle a null return.
The tool works fine when it is not a Geoprocessing Service, even when a null output is returned. It fails with Geoprocessing Service.

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

ShingLin — Fri, 19 Oct 2012 19:36:21 GMT

Only run the kernel density function when the get count is greater than zero. The map service is generated based on the result of gp service. When there is no input/output of gp service, the map service is destined to be failing. This is as designed for now. So the work around is not to call the function and set the blank in your output parameter. Not the best way, but at least won't fail when you run the service.

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Sun, 21 Oct 2012 19:30:59 GMT

HI,

The issue is in setting the null derived output in a Geoprocessing Service. Neither ArcMap, or the AGS Map Service Layer knows about the null part of it and fails.

The issue is not recreatable as a standalone tool (one that is not a service). It is just present in the Geoprocesing Service behaviour.

Susan

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

ShingLin — Mon, 22 Oct 2012 22:41:52 GMT

In the code u provided, are the following code executed when count = 0? It should not be. But the error message you sent seems showing this part of code was executed. If it does, it will fail because the output is overwritten outraster which is invalid.

#TODO: process Kerenl Density
processStart=datetime.datetime.now()
arcpy.AddMessage("***generate Kernel Density Surface")
arcpy.gp.KernelDensity_sa("inFeatures1", modelFld, outRaster, "", searchReadius, "SQUARE_MAP_UNITS")
arcpy.AddMessage(str(datetime.datetime.now()-processStart))

#TODO: check In License
arcpy.AddMessage("***check In Spatial Analyst")
arcpy.CheckInExtension("spatial")

#TODO: set Output
arcpy.AddMessage("***set The Output Layer")
arcpy.SetParameter(6,outRaster)

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Tue, 23 Oct 2012 17:24:20 GMT

That's correct. If the number of selected features is 0, Kernel Density will not run. Instead a null output is specified and the application bails when a null raster layer is being set as AGS Dynamic Map Service Layer.

It is almost as if the default setting from the publsihing mxd is overriding the null output passed back to the AGS Dynamc Map Service Layer.

I understand we are trialing the heat map sample using the javascript api and html5 framework. http://blogs.esri.com/esri/arcgis/2012/09/26/heat-up-your-webmaps/ This quite a big difference from the Geoprocessing Service. It is custom code with a mix of open source and ESRI technologies. It is very much a R&D exercise because we exhausted the Proof of Concept funding for the work.

It's kind of scary resorting to a non-tested, non-supported solution though.

Susan

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

SusanJones — Thu, 15 Nov 2012 22:23:34 GMT

Hi,

We picked up on this yesterday again and made some further progress.

We will have tio first initialize the outRaster as a none before with start with the Surface Generation function. After this, we can include a status variable

#TODO: THIS IS Important part �?? setting outRaster to None initially
outRaster=None
�?�
�?�
#TODO: Check Count
If cnt>0:
     #run The hotspot
   outRaster=�?�
   Status=�?�true�?�
Else:
     Status=�?�false
#assign Parameters
If Status=�?�true�?�:
    arcpy.SetParameter(6,outRaster)
else:
    arcpy.SetParameter(6,outRaster)

Susan

Re: Pass a null result back to a dynamic map Service from a geoprocessing task

ShingLin — Wed, 21 Nov 2012 00:09:44 GMT

Thanks for updating. Is this part of your code correct? It seems there is no need of 'if .. else' statement since both logic condition executes the same function and parameter arcpy.SetParameter(6,outRaster).

If Status=�?�true�?�:
arcpy.SetParameter(6,outRaster)
else:
arcpy.SetParameter(6,outRaster)