Execution for Geo processing tool using custom python packages

1641
10
09-14-2016 01:02 AM
PrasadG
New Contributor II

We have Geo processing tool to be executed in ArcGIS Server Linux.Tool execution failed due to import python module(seabird) in ArcGIS Server.We have written Python code using custom python packages(pip,seabird) in windows platform and published to ArcGIS Server as a Geo processing Service.

We have upgraded existing python version 2.6.6 to 2.7.10 in Linux server and installed both pip and seabird.But tool execution failed in ArcGIS Server.Please help us to provide solution for tool execution in Linux server using custom python packages.

Tags (1)
0 Kudos
10 Replies
by Anonymous User
Not applicable

In python version 2.6.6 to 2.7.10 some of basic function are changed 
Try to compile python code and recreate your GP service again.

PrasadG
New Contributor II

Thanks Avinash it was exactly what i got problem. Now our problem is resolved...we did what you suggested ..Thanks once again... 

0 Kudos
by Anonymous User
Not applicable

Hello Prasad 

Good to hear that 
if its works then please close the thread.

0 Kudos
PrasadG
New Contributor II

Hi Avinash,

We observed one different issue after tool published and executed on server.

Tool executed successfully for the first time.If we want to re-execute the tool, execution failed with below error message(collected from ArcGIS Server Logs)

We are using ArcGIS 10.3.1 Server, Python 2.7 and third party python packages PIP - 8.1.2 and Seabird-0.8.1 on RHEL Server 6.5. Both the third party python packages are installed on RHEL server.Please help us to resolve this issue.

 

Expecting your response with the solution soon...

0 Kudos
by Anonymous User
Not applicable

Hello Prasad,

From Arcgis Server log file looks like you are trying to import some data in your DB with the help of this script,

Can you please share more details or sample script if possible.

if not then share overview of it.

if file with same name present on out put location its throws error
Please check.

0 Kudos
PrasadG
New Contributor II

Hi Avinash,

Thanks for the quick response,Please find below details.

   

Please find below script to import .cnv files to SDE Database.

import os
... import re
... import csv
... import arcpy
... import unicodedata
... import numpy
... import seabird
... from seabird.cnv import fCNV
...
... instance= arcpy.GetParameterAsText(0)

... username= arcpy.GetParameterAsText(1)

... password= arcpy.GetParameterAsText(2)
... 
connection=arcpy.CreateDatabaseConnection_management(arcpy.env.scratchFolder,"ncaor_linux","ORACLE",instance,"DATABASE_AUTH",username,password,"DO_NOT_SAVE_USERNAME")
...
... ctdinput = arcpy.GetParameterAsText(3)
... if ctdinput == '#' or not ctdinput:
... ctdinput = r"E:\Temp\CTDB12.cnv"
...
... file=os.path.basename(ctdinput)
... inputfile=os.path.splitext(file)[0]
... extension=os.path.splitext(file)[1]
...
...
... geographicprojection=r"GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]]"
...
... projection=r"PROJCS['WGS_1984_UTM_Zone_42S',GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Transverse_Mercator'],PARAMETER['False_Easting',500000.0],PARAMETER['False_Northing',10000000.0],PARAMETER['Central_Meridian',69.0],PARAMETER['Scale_Factor',0.9996],PARAMETER['Latitude_Of_Origin',0.0],UNIT['Meter',1.0]]"
...
... ctdprojection = arcpy.GetParameterAsText(4)
... if ctdprojection == '#' or not ctdprojection:
... ctdprojection = projection
...
...
... cruise_no = arcpy.GetParameterAsText(5)
... if cruise_no == '#' or not cruise_no:
... cruise_no = "SK14145" # provide a default value if unspecified
...
... file_id= arcpy.GetParameterAsText(6)
... if file_id== '#' or not file_id:
... file_id= "ctd"# provide a default value if unspecified
... #Replace the name of input file
...
... inputfile="A_"+file_id
...
...
... m=0
... longindex=0
... latindex=0
... depthindex=0
... dateindex=0
... timeindex=0
... csvvaluelist=[]
... csvfileheaderlist=[]
... data=[]
... featureclass_scratchL=arcpy.env.scratchGDB+os.sep+cruise_no+"_"+inputfile +"_fc"
... outputfcLine=str(connection)+os.sep+"SDE."+cruise_no+"_"+inputfile +"_fc"
... array=[]
...
... headerstring=""
... inputcnvfile= fCNV(ctdinput)
... keys=inputcnvfile.keys()
... rawdata=inputcnvfile.raw_data().values()
... headerdata=inputcnvfile.raw_header()
... deleteidentical=arcpy.env.scratchGDB+os.sep+"fc_projected"
...
...
... def writetocsvandSDE():
... datatocsv=[]
... CSVdir = arcpy.env.scratchFolder
... realCSVdir = os.path.realpath(CSVdir)
... foupath = os.path.join(realCSVdir+os.sep+inputfile+".csv")
... fou = open(foupath, 'wb')
... a = csv.writer(fou)
... for i in range(0,len(csvvaluelist)):
... datatocsv= [csvvaluelist]
... a.writerows(datatocsv)
... fou.close()
... csvfilepath=os.path.join(arcpy.env.scratchFolder+os.sep+inputfile+".csv")
... eventlayer=arcpy.MakeXYEventLayer_management(csvfilepath,long,lat,"fc_layerfile",geographicprojection,depth)
... outputfc=os.path.join(str(connection)+os.sep+cruise_no+"_"+inputfile+"_fc")
... projectedfc=arcpy.Project_management(eventlayer,os.path.join(arcpy.env.scratchGDB+os.sep+"fc_projected"),ctdprojection,"",geographicprojection,"NO_PRESERVE_SHAPE")
... # Process: Copy Features to output sde
... fc_copy=arcpy.CopyFeatures_management(projectedfc,outputfc,"","","","")
... deleteidentical=arcpy.DeleteIdentical_management(projectedfc,"\""+long+";"+lat+"\"", "", "0")
...
...
... for j in range(0,len(keys)):
... headerstring=headerstring+" "+keys
...
... data.append(headerstring.strip())
...
... for row in rawdata:
... array.append(str(row))
...
... rawdata=re.split('\n',str(array[0]))
...
...
...
... for raw in rawdata:
... raw=re.sub( '\s+', ' ', raw).strip()
... data.append(raw)
...
... for k in range(0,len(data)):
... celldata=[]
... rowdata=data.split()
... for t in range(0,len(rowdata)):
... celldata.append(rowdata)
... if re.match('long.*',rowdata,re.IGNORECASE):
... longindex=t
... long=rowdata
... elif re.match('lat.*',rowdata,re.IGNORECASE):
... latindex=t
... lat=rowdata
... elif re.match('depth.*',rowdata,re.IGNORECASE):
... depthindex=t
... depth=rowdata
... csvvaluelist.append(celldata)
... writetocsvandSDE()
...
... arcpy.SetParameterAsText(7,deleteidentical)
... arcpy.SetParameterAsText(8,headerdata)

0 Kudos
by Anonymous User
Not applicable

Apart from this I am looking in your script
Let you know my findings

0 Kudos
JonathanQuinn
Esri Notable Contributor

I'd say you can focus on the line mod = sys.modules[extant].  It says NoneType object has no attribute "modules" indicating that when you import sys, a NoneType object, ("None"), is returned.  An easy way to test this is by running the python included with Server, importing sys, seeing what type of object it is, and then seeing what sys.modules gives you:

If printing sys gives you None, that's where to start troubleshooting.

0 Kudos