topic Re: Execution for Geo processing tool using custom python packages in ArcGIS Enterprise Questions

Execution for Geo processing tool using custom python packages

PrasadG — Wed, 14 Sep 2016 08:02:56 GMT

We have Geo processing tool to be executed in ArcGIS Server Linux.Tool execution failed due to import python module(seabird) in ArcGIS Server.We have written Python code using custom python packages(pip,seabird) in windows platform and published to ArcGIS Server as a Geo processing Service.

We have upgraded existing python version 2.6.6 to 2.7.10 in Linux server and installed both pip and seabird.But tool execution failed in ArcGIS Server.Please help us to provide solution for tool execution in Linux server using custom python packages.

Re: Execution for Geo processing tool using custom python packages

Anonymous User — Wed, 14 Sep 2016 09:49:25 GMT

In python version 2.6.6 to 2.7.10 some of basic function are changed
Try to compile python code and recreate your GP service again.

Re: Execution for Geo processing tool using custom python packages

PrasadG — Fri, 16 Sep 2016 06:39:53 GMT

Thanks Avinash it was exactly what i got problem. Now our problem is resolved...we did what you suggested ..Thanks once again...

Re: Execution for Geo processing tool using custom python packages

Anonymous User — Fri, 16 Sep 2016 06:47:17 GMT

Hello Prasad

Good to hear that
if its works then please close the thread.

Re: Execution for Geo processing tool using custom python packages

PrasadG — Thu, 22 Sep 2016 11:02:07 GMT

Hi Avinash,

We observed one different issue after tool published and executed on server.

Tool executed successfully for the first time.If we want to re-execute the tool, execution failed with below error message(collected from ArcGIS Server Logs)

We are using ArcGIS 10.3.1 Server, Python 2.7 and third party python packages PIP - 8.1.2 and Seabird-0.8.1 on RHEL Server 6.5. Both the third party python packages are installed on RHEL server.Please help us to resolve this issue.

Expecting your response with the solution soon...

Re: Execution for Geo processing tool using custom python packages

Anonymous User — Thu, 22 Sep 2016 11:48:17 GMT

Hello Prasad,

From Arcgis Server log file looks like you are trying to import some data in your DB with the help of this script,

Can you please share more details or sample script if possible.

if not then share overview of it.

if file with same name present on out put location its throws error
Please check.

Re: Execution for Geo processing tool using custom python packages

PrasadG — Thu, 22 Sep 2016 13:36:52 GMT

Hi Avinash,

Thanks for the quick response,Please find below details.

Please find below script to import .cnv files to SDE Database.

import os
... import re
... import csv
... import arcpy
... import unicodedata
... import numpy
... import seabird
... from seabird.cnv import fCNV
...
... instance= arcpy.GetParameterAsText(0)

... username= arcpy.GetParameterAsText(1)

... password= arcpy.GetParameterAsText(2)
...
connection=arcpy.CreateDatabaseConnection_management(arcpy.env.scratchFolder,"ncaor_linux","ORACLE",instance,"DATABASE_AUTH",username,password,"DO_NOT_SAVE_USERNAME")
...
... ctdinput = arcpy.GetParameterAsText(3)
... if ctdinput == '#' or not ctdinput:
... ctdinput = r"E:\Temp\CTDB12.cnv"
...
... file=os.path.basename(ctdinput)
... inputfile=os.path.splitext(file)[0]
... extension=os.path.splitext(file)[1]
...
...
... geographicprojection=r"GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]]"
...
... projection=r"PROJCS['WGS_1984_UTM_Zone_42S',GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Transverse_Mercator'],PARAMETER['False_Easting',500000.0],PARAMETER['False_Northing',10000000.0],PARAMETER['Central_Meridian',69.0],PARAMETER['Scale_Factor',0.9996],PARAMETER['Latitude_Of_Origin',0.0],UNIT['Meter',1.0]]"
...
... ctdprojection = arcpy.GetParameterAsText(4)
... if ctdprojection == '#' or not ctdprojection:
... ctdprojection = projection
...
...
... cruise_no = arcpy.GetParameterAsText(5)
... if cruise_no == '#' or not cruise_no:
... cruise_no = "SK14145" # provide a default value if unspecified
...
... file_id= arcpy.GetParameterAsText(6)
... if file_id== '#' or not file_id:
... file_id= "ctd"# provide a default value if unspecified
... #Replace the name of input file
...
... inputfile="A_"+file_id
...
...
... m=0
... longindex=0
... latindex=0
... depthindex=0
... dateindex=0
... timeindex=0
... csvvaluelist=[]
... csvfileheaderlist=[]
... data=[]
... featureclass_scratchL=arcpy.env.scratchGDB+os.sep+cruise_no+"_"+inputfile +"_fc"
... outputfcLine=str(connection)+os.sep+"SDE."+cruise_no+"_"+inputfile +"_fc"
... array=[]
...
... headerstring=""
... inputcnvfile= fCNV(ctdinput)
... keys=inputcnvfile.keys()
... rawdata=inputcnvfile.raw_data().values()
... headerdata=inputcnvfile.raw_header()
... deleteidentical=arcpy.env.scratchGDB+os.sep+"fc_projected"
...
...
... def writetocsvandSDE():
... datatocsv=[]
... CSVdir = arcpy.env.scratchFolder
... realCSVdir = os.path.realpath(CSVdir)
... foupath = os.path.join(realCSVdir+os.sep+inputfile+".csv")
... fou = open(foupath, 'wb')
... a = csv.writer(fou)
... for i in range(0,len(csvvaluelist)):
... datatocsv= [csvvaluelist]
... a.writerows(datatocsv)
... fou.close()
... csvfilepath=os.path.join(arcpy.env.scratchFolder+os.sep+inputfile+".csv")
... eventlayer=arcpy.MakeXYEventLayer_management(csvfilepath,long,lat,"fc_layerfile",geographicprojection,depth)
... outputfc=os.path.join(str(connection)+os.sep+cruise_no+"_"+inputfile+"_fc")
... projectedfc=arcpy.Project_management(eventlayer,os.path.join(arcpy.env.scratchGDB+os.sep+"fc_projected"),ctdprojection,"",geographicprojection,"NO_PRESERVE_SHAPE")
... # Process: Copy Features to output sde
... fc_copy=arcpy.CopyFeatures_management(projectedfc,outputfc,"","","","")
... deleteidentical=arcpy.DeleteIdentical_management(projectedfc,"\""+long+";"+lat+"\"", "", "0")
...
...
... for j in range(0,len(keys)):
... headerstring=headerstring+" "+keys
...
... data.append(headerstring.strip())
...
... for row in rawdata:
... array.append(str(row))
...
... rawdata=re.split('\n',str(array[0]))
...
...
...
... for raw in rawdata:
... raw=re.sub( '\s+', ' ', raw).strip()
... data.append(raw)
...
... for k in range(0,len(data)):
... celldata=[]
... rowdata=data.split()
... for t in range(0,len(rowdata)):
... celldata.append(rowdata)
... if re.match('long.*',rowdata,re.IGNORECASE):
... longindex=t
... long=rowdata
... elif re.match('lat.*',rowdata,re.IGNORECASE):
... latindex=t
... lat=rowdata
... elif re.match('depth.*',rowdata,re.IGNORECASE):
... depthindex=t
... depth=rowdata
... csvvaluelist.append(celldata)
... writetocsvandSDE()
...
... arcpy.SetParameterAsText(7,deleteidentical)
... arcpy.SetParameterAsText(8,headerdata)

Re: Execution for Geo processing tool using custom python packages

Anonymous User — Fri, 23 Sep 2016 08:57:31 GMT

http://www.linz.govt.nz/system/files_force/media/pages-attachments/Importing-LDS-CSV-data-into-your-database.pdf

Re: Execution for Geo processing tool using custom python packages

Anonymous User — Fri, 23 Sep 2016 08:58:24 GMT

Apart from this I am looking in your script
Let you know my findings

Re: Execution for Geo processing tool using custom python packages

JonathanQuinn — Fri, 23 Sep 2016 22:41:05 GMT

I'd say you can focus on the line mod = sys.modules[extant]. It says NoneType object has no attribute "modules" indicating that when you import sys, a NoneType object, ("None"), is returned. An easy way to test this is by running the python included with Server, importing sys, seeing what type of object it is, and then seeing what sys.modules gives you:

If printing sys gives you None, that's where to start troubleshooting.

Re: Execution for Geo processing tool using custom python packages

PrasadG — Thu, 29 Sep 2016 06:07:52 GMT

Thanks Jonathan,

Finally we resolved our issue after we installed set up tools 19.2 in python 2.7