topic Re: Error message even though script finishes with expected results. in Python Questions

Error message even though script finishes with expected results.

AlexGray — Thu, 06 Jun 2013 17:08:16 GMT

I am working on a script trying to plot lines from csv point data file..
It all seems to work until the end where I get an error stating that the table is not found..

when I open the FC in the map view, I see that I am getting the result I was expecting..

any help would be greatly appreciated..

here is the error...

Traceback (most recent call last):   File "C:\temp\GIS_projects\Sample_tripData\test.py", line 27, in <module>     "TELATDD","GEODESIC","TRIPSEQNUM")   File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\management.py", line 2913, in XYToLine     raise e ExecuteError: ERROR 999999: Error executing function. The table was not found. [trip_lines] Failed to execute (XYToLine).

here is my code..

# Import system modules import arcpy print "importing modules and set environment...." from arcpy import env env.workspace = "c:/temp/gis_projects/sample_tripdata/test.gdb" fc = "trip_lines" #overwrite pre-existing files arcpy.env.overwriteOutput = True  # Set local variables print "setting data input and output variables..." input_table = "C:/temp/GIS_projects/Sample_tripData/A6_TEMP_ATL_TRIP_HAULx.csv"  # check to see if FC exists, if so, delete. if arcpy.Exists(fc):     print "trip_lines exists..."     print "deleting trip_lines..."     arcpy.Delete_management(fc)     print "trip_lines deleted..."  out_lines = "C:/temp/GIS_projects/Sample_tripData/test.gdb/trip_lines"  #XY To Line print "running XY to Line Geoprocesseing tool..." arcpy.XYToLine_management(input_table,out_lines,                          "TSLONGDD","TSLATDD","TELONGDD",                          "TELATDD","GEODESIC","TRIPSEQNUM")

Re: Error message even though script finishes with expected results.

Anonymous User — Sun, 12 Dec 2021 16:17:22 GMT

Your code looks good to me...I am guessing there is something wrong with your path. When you set your environment, you have the some things capitalized...But then when you have your table path the capitalization does not match...Case matters in Python. Is this supposed to be lowercase?

# Import system modules
import arcpy
print "importing modules and set environment...."
from arcpy import env
env.workspace = "c:/temp/gis_projects/sample_tripdata/test.gdb"
fc = "trip_lines"
#overwrite pre-existing files
arcpy.env.overwriteOutput = True

# Set local variables
print "setting data input and output variables..."
input_table = "C:/temp/GIS_projects/Sample_tripData/A6_TEMP_ATL_TRIP_HAULx.csv"

# check to see if FC exists, if so, delete.
if arcpy.Exists(fc):
    print "trip_lines exists..."
    print "deleting trip_lines..."
    arcpy.Delete_management(fc)
    print "trip_lines deleted..."

out_lines = "C:/temp/GIS_projects/Sample_tripData/test.gdb/trip_lines"

#XY To Line
print "running XY to Line Geoprocesseing tool..."
arcpy.XYToLine_management(input_table,out_lines,
                         "TSLONGDD","TSLATDD","TELONGDD",
                         "TELATDD","GEODESIC","TRIPSEQNUM")

I would double check the paths to make sure you have it capitalized correctly. Also, I find that it is easier to specify your paths this way:

r'G:\PROJECTS\Cedar\GeneralScripts\PyLibrary'

By inserting the "r" in front of single/double quotes, you can cast it as a raw string. When you do this you do not have to worry about escape characters and shortcuts. You can simply copy and paste the path inside the r' ' and you don't have to worry about changing your backslashes to double backslashes or forward slashes.

Re: Error message even though script finishes with expected results.

RhettZufelt — Sat, 11 Dec 2021 13:41:49 GMT

Not sure, but could be due to not using "pytonic" paths.

try:

out_lines = r"C:\temp\GIS_projects\Sample_tripData\test.gdb\trip_lines"

or

out_lines = "C:\\temp\\GIS_projects\\Sample_tripData\\test.gdb\\trip_lines"

Re: Error message even though script finishes with expected results.

T__WayneWhitley — Thu, 06 Jun 2013 20:56:53 GMT

You've already set the workspace you intend to write your output to which you named with your variable 'fc'... So 'output_lines' is redundant.
Should be able to get rid of the line where you set the 'output_lines' variable; simply substitute fc...like so:

arcpy.XYToLine_management(input_table, fc, ....etc., etc....)

...and ditto what Rhett said about pathnames.

Enjoy,
Wayne

Re: Error message even though script finishes with expected results.

AlexGray — Sat, 11 Dec 2021 13:41:51 GMT

Well, I've made the changes to the script that @Zufelt and @Wayne have mentioned..

Yet, I am still getting an error.. this time says "Cannot create Featureclass TripLines"

Could it be that I'm not specifying the license?

here is the error message...

Traceback (most recent call last):
  File "C:\temp\GIS_projects\Sample_tripData\test.py", line 20, in <module>
    arcpy.XYToLine_management(input_table,fc,"TSLONGDD","TSLATDD","TELONGDD","TELATDD","GEODESIC","TRIPSEQNUM")
  File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\management.py", line 2913, in XYToLine
    raise e
ExecuteError: ERROR 000206: Cannot create FeatureClass tripLines
Failed to execute (XYToLine).

here is my modified code...

# Import system modules
import arcpy
from arcpy import env

print "importing modules and set environment...."

#overwrite pre-existing files
arcpy.env.overwriteOutput = True

#set workspace
env.workspace = "c:\temp\gis_projects\sample_tripdata\test.gdb"
fc = "tripLines"

# Set local variables
print "setting data input and output variables..."
input_table = r"C:\temp\GIS_projects\Sample_tripData\A6_TEMP_ATL_TRIP_HAULx.csv"

#XY To Line
print "running XY to Line Geoprocessing tool..."
arcpy.XYToLine_management(input_table,fc,"TSLONGDD","TSLATDD","TELONGDD","TELATDD","GEODESIC","TRIPSEQNUM")

# Process to delete identical line features...
# Set the field upon which the identicals are found
fields = "Shape;TRIPSEQNUM;Shape_Length"

# Set the XY tolerance within which to identical records to be deleted
xy_tol = "0.0002 Miles"

# Set the Z tolerance to default
z_tol = "0"

print "Deleteing identical features..."
# Execute Delete Identical 
arcpy.DeleteIdentical_management(fc, fields, xy_tol, z_tol)

print "Dropping leading character..."

cursor = arcpy.UpdateCursor(fc)
field = "TRIPSEQNUM"
row = cursor.next()
while row:
    # set value of field to only those values after the zero value location
    # I want to strip off the first character within the value of the field
    row.setValue(field, row.getValue(field)[0:])
    cursor.updateRow(row)
    row = cursor.next()

del fc, input_table, row, cursor, out_lines, input_table, field

print "Processing Complete!"

Re: Error message even though script finishes with expected results.

T__WayneWhitley — Sat, 11 Dec 2021 13:41:54 GMT

Apply the pathname convention to all your pathnames -- your workspace var pathname is wrong because "\t" is interpreted by python as a tab, see the test below, the 'jump' after the colon and again at the gdb name - this is invalid:

>>> testPathname = "c:\temp\gis_projects\sample_tripdata\test.gdb"
>>> print testPathname
c: emp\gis_projects\sample_tripdata est.gdb
>>>

Almost had it, try again!

The difference with 'r':

>>> testPathname = r"c:\temp\gis_projects\sample_tripdata\test.gdb"
>>> print testPathname
c:\temp\gis_projects\sample_tripdata\test.gdb
>>>

Enjoy,
Wayne

Re: Error message even though script finishes with expected results.

AlexGray — Sat, 11 Dec 2021 13:41:57 GMT

@Wayne, thanks.. I tried it both ways.. this time it ran through the process and executed the xytoline_management tool..

I checked the env and it was correct this time.. thanks for the heads up.. :0)

anyways, the tool ran and the csv was converted to lines.. but then it threw another error.. same as the first..

here is the error message...

Traceback (most recent call last):
  File "C:\temp\GIS_projects\Sample_tripData\test.py", line 22, in <module>
    arcpy.XYToLine_management(input_table,fc,"TSLONGDD","TSLATDD","TELONGDD","TELATDD","GEODESIC","TRIPSEQNUM")
  File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\management.py", line 2913, in XYToLine
    raise e
ExecuteError: ERROR 999999: Error executing function.
The table was not found. [tripLines]
Failed to execute (XYToLine).

here is my new updated code...

# Import system modules
import arcinfo
import arcpy
import sys
if arcpy.CheckProduct("ArcInfo") == "Available":
    print "ArcInfo available, processing script..."
    print "importing modules and set environment...."

    #overwrite pre-existing files
    arcpy.env.overwriteOutput = True

    #set workspace
    arcpy.env.workspace = "c:\\temp\\gis_projects\\sample_tripdata\\test.gdb"
    fc = "tripLines"

    # Set local variables
    print "setting data input and output variables..."
    input_table = "C:\\temp\\GIS_projects\\Sample_tripData\\A6_TEMP_ATL_TRIP_HAULx.csv"

    #XY To Line
    print "running XY to Line Geoprocessing tool..."
    arcpy.XYToLine_management(input_table,fc,"TSLONGDD","TSLATDD","TELONGDD","TELATDD","GEODESIC","TRIPSEQNUM")

    # Process to delete identical line features...
    # Set the field upon which the identicals are found
    fields = "Shape;TRIPSEQNUM;Shape_Length"

    # Set the XY tolerance within which to identical records to be deleted
    xy_tol = "0.0002 Miles"

    # Set the Z tolerance to default
    z_tol = "0"

    print "Deleteing identical features..."
    # Execute Delete Identical 
    arcpy.DeleteIdentical_management(fc, fields, xy_tol, z_tol)

    print "Dropping leading character..."

    cursor = arcpy.UpdateCursor(fc)
    field = "TRIPSEQNUM"
    row = cursor.next()
    while row:
        # set value of field to only those values after the zero value location
        # I want to strip off the first character within the value of the field
        row.setValue(field, row.getValue(field)[0:])
        cursor.updateRow(row)
        row = cursor.next()

    del fc, input_table, row, cursor, out_lines, input_table, field

    print "Processing Complete!"

else:
    print "ArcInfo license not available"
    sys.exit("ArcInfo license not available")

Re: Error message even though script finishes with expected results.

T__WayneWhitley — Fri, 07 Jun 2013 21:43:55 GMT

I am not certain, but did notice your spatial reference parameter is not set... you should try specifying that. Sorry that's all I can think of that looks suspicious to me - oh, and I'd also avoid writing to the temp directory. The 'table not found' error can be sort of cryptic...
If you are still having difficulty, you can try attaching a portion of your csv (or all if small enough to attach here) and I'll see what I can do with it later, possibly tomorrow...

Enjoy,
Wayne

EDIT
I was curious for you to test something - see what happens if you can specify shapefile output. This may not be allowed for this tool. You can set up a different kind of test to see if you can make an event layer out of the point coordinates, then copy to a shapefile... not sure, but if you can do this to create shapefile output but not file gdb output, chances are the spatial reference is problematic. Again, just guessing without experimenting.

Re: Error message even though script finishes with expected results.

RhettZufelt — Sat, 11 Dec 2021 13:41:59 GMT

Traceback (most recent call last):
  File "C:\temp\GIS_projects\Sample_tripData\test.py", line 22, in <module>
    arcpy.XYToLine_management(input_table,fc,"TSLONGDD","TSLATDD","TELONGDD","TELATDD","GEODESIC","TRIPSEQNUM")
  File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\management.py", line 2913, in XYToLine
    raise e
ExecuteError: ERROR 999999: Error executing function.
The table was not found. [tripLines]
Failed to execute (XYToLine).

here is my new updated code...

# Import system modules
import arcinfo
import arcpy
import sys
if arcpy.CheckProduct("ArcInfo") == "Available":
    print "ArcInfo available, processing script..."
    print "importing modules and set environment...."

    #overwrite pre-existing files
    arcpy.env.overwriteOutput = True

    #set workspace
    arcpy.env.workspace = "c:\\temp\\gis_projects\\sample_tripdata\\test.gdb"
    fc = "tripLines"

    if arcpy.Exists(fc):
      arcpy.Delete_management(fc, "FeatureClass")
    featclass = "c:\\temp\\gis_projects\\sample_tripdata\\test.gdb\\tripLines"    # Set local variables
    print "setting data input and output variables..."
    input_table = "C:\\temp\\GIS_projects\\Sample_tripData\\A6_TEMP_ATL_TRIP_HAULx.csv"

    #XY To Line
    print "running XY to Line Geoprocessing tool..."
    arcpy.XYToLine_management(input_table,featclass,"TSLONGDD","TSLATDD","TELONGDD","TELATDD","GEODESIC","TRIPSEQNUM")

    # Process to delete identical line features...
    # Set the field upon which the identicals are found
    fields = "Shape;TRIPSEQNUM;Shape_Length"

    # Set the XY tolerance within which to identical records to be deleted
    xy_tol = "0.0002 Miles"

    # Set the Z tolerance to default
    z_tol = "0"

    print "Deleteing identical features..."
    # Execute Delete Identical 
    arcpy.DeleteIdentical_management(fc, fields, xy_tol, z_tol)

    print "Dropping leading character..."

    cursor = arcpy.UpdateCursor(fc)
    field = "TRIPSEQNUM"
    row = cursor.next()
    while row:
        # set value of field to only those values after the zero value location
        # I want to strip off the first character within the value of the field
        row.setValue(field, row.getValue(field)[0:])
        cursor.updateRow(row)
        row = cursor.next()

    del fc, input_table, row, cursor, out_lines, input_table, field

    print "Processing Complete!"

else:
    print "ArcInfo license not available"
    sys.exit("ArcInfo license not available")

What happens if you make the above change in blue?

If that doesn't work, what happens if you make the changes above in red?

R_

Re: Error message even though script finishes with expected results.

AlexGray — Sat, 11 Dec 2021 13:42:02 GMT

I am not certain, but did notice your spatial reference parameter is not set... you should try specifying that. Sorry that's all I can think of that looks suspicious to me - oh, and I'd also avoid writing to the temp directory. The 'table not found' error can be sort of cryptic...
If you are still having difficulty, you can try attaching a portion of your csv (or all if small enough to attach here) and I'll see what I can do with it later, possibly tomorrow...

Enjoy,
Wayne

EDIT
I was curious for you to test something - see what happens if you can specify shapefile output. This may not be allowed for this tool. You can set up a different kind of test to see if you can make an event layer out of the point coordinates, then copy to a shapefile... not sure, but if you can do this to create shapefile output but not file gdb output, chances are the spatial reference is problematic. Again, just guessing without experimenting.

Wayne,

As it turns out, the script runs, although I did add the spatial reference.. I looked into the shapefile output.. but it looks as though the tool only outputs a feature class. I think I found the problem though, and it wasn't the script per sa.. I found that some of the entries in the csv file I was using had some empty cells in the data fields I was using. So, the short of it is that I needed to make sure my data was complete and didn't include Null values.. A little embarrassing.. Thanks for your help on this though.. really appreciate the effort.. now I need to figure out how to get it to check for null values..

here is the updated code...

# Import system modules
import arcinfo
import arcpy
import sys
if arcpy.CheckProduct("ArcInfo") == "Available":
    print "ArcInfo available, processing script..."
    print "importing modules and setting environment...."

    #overwrite pre-existing files
    arcpy.env.overwriteOutput = True

    #delete old feature class
    #if  arcpy.Exists("c:\\temp\\gis_projects\\sample_tripdata\\test.gdb\\haulLines"):
        #arcpy.Delete_management ("c:\\temp\\gis_projects\\sample_tripdata\\test.gdb\\haulLines")

    #set workspace
    arcpy.env.workspace = "c:\\temp\\gis_projects\\sample_tripdata\\test.gdb"
    fc = "c:\\temp\\gis_projects\\sample_tripdata\\test.gdb\\haulLines"
    fieldName = "HaulSeqNum"
    
    # Set local variables
    print "gathering input table data...."
    input_table = "C:\\temp\\GIS_projects\\Sample_tripData\\TEST_HAUL_DATA.csv"

    # tool variables
    print "setting up variables for XY to Line management tool..."
    in_table = input_table
    out_featureclass = fc
    startx_field = "DLONGDD"
    starty_field = "DLATDD"
    endx_field = "RLONGDD"
    endy_field = "RLATDD"
    line_type = "GEODESIC"
    id_field = fieldName
    
    print "setting the spatial reference..."
    spatialRef = arcpy.SpatialReference("C:\\temp\\GIS_projects\\Sample_tripData\\TRIPs_GOA_STAT7.prj")
    
    #XY To Line
    print "running XY to Line Geoprocessing tool..."
    arcpy.XYToLine_management(in_table, out_featureclass, startx_field, starty_field, endx_field, endy_field,line_type,id_field,spatialRef)
    # Process to delete identical line features...
    # Set the field upon which the identicals are found
    fields = '"Shape;' + fieldName + ';Shape_Length"'

    # Set the XY tolerance within which to identical records to be deleted
    xy_tol = "0.0002 Miles"

    # Set the Z tolerance to default
    z_tol = "0"

    print "Deleteing identical features..."
    # Execute Delete Identical 
    arcpy.DeleteIdentical_management(fc, fields, xy_tol, z_tol)

    print "Dropping leading character..."

    cursor = arcpy.UpdateCursor(fc)
    field = fieldName
    row = cursor.next()
    while row:
      # set value of field to only those values after the zero value location
      # I want to strip off the first character within the value of the field
      row.setValue(field, row.getValue(field)[1:])
      cursor.updateRow(row)
      row = cursor.next()

    print "Processing Complete!"

else:
    print "ArcInfo license not available"
    sys.exit("ArcInfo license not available")

Re: Error message even though script finishes with expected results.

T__WayneWhitley — Tue, 11 Jun 2013 16:07:52 GMT

Very well sounds good, you are on track. That's all that matters - knowing what/where to troubleshoot is more than half the battle!

If I may suggest this, based on what you said you may want to insert an 'intermediate' geoprocessing step where you write to a new table for better input into your tool...a kind of 'clean' processing for your csv. You could potentially do this with cursor processing, looping on your csv to writing to, say, a gdb table. Couple that with a 'try-except' within the loop to write the lineno for faulty lines to check, writing all the 'good' lines to the table. When you correct the 'bad' lines, you can append them. ....or, if you know how to handle them in the code, just correct them all in the same process. But if easier, it would probably greatly help to at least know the lines that need your attention.

Enjoy,
Wayne

Re: Error message even though script finishes with expected results.

RhettZufelt — Tue, 11 Jun 2013 16:23:49 GMT

Wayne,

now I need to figure out how to get it to check for null values..

Could probably do a TableToTable_conversion on your input file. the TableToTable allows an expression so you could only copy the rows without nulls/blanks to the new table (possibly in memory) and use that table for your input.

Just a thought that doesn't need an if/then statement for each iteration.

R_

Re: Error message even though script finishes with expected results.

AlexGray — Sat, 11 Dec 2021 13:42:05 GMT

Could probably do a TableToTable_conversion on your input file. the TableToTable allows an expression so you could only copy the rows without nulls/blanks to the new table (possibly in memory) and use that table for your input.

Just a thought that doesn't need an if/then statement for each iteration.

R_

yeah, that is what I've done.. but it would be nice to know where the nulls are or how many there are..
I'm working on trying to figure out how to run a cursor and check values..
but for now this works..

I first convert the csv file to a table within a gdb and then convert the gdb table to another table minus the nulls..
here is the code..

#convert csv to table
    table1 = "C:\\temp\\GIS_projects\\Sample_tripData\\TEST_HAUL_DATA.csv"
    gdb = "c:\\temp\\gis_projects\\sample_tripdata\\test.gdb"
    table2 = "testHaul"
    table3 = "cleantestHaul"
    fieldName = "HaulSeqNum"
    print "Converting CSV table to GDB table..."
    arcpy.TableToTable_conversion(table1, gdb, table2)
    expression = arcpy.AddFieldDelimiters(arcpy.env.workspace, "DLONGDD") + " IS NOT NULL"
    print "Excluding null values..."
    arcpy.TableToTable_conversion(table2, gdb, table3,expression)
    fc = "c:\\temp\\gis_projects\\sample_tripdata\\test.gdb\\haulLines"

Thanks..

Re: Error message even though script finishes with expected results.

RhettZufelt — Sat, 11 Dec 2021 13:42:08 GMT

You could do something like this:

import arcpy
# Overwrite pre-existing files
arcpy.env.overwriteOutput = True

# Local variables:


test_csv = "D:\\baks\\dont_use\\test.csv"
in_memory = "in_memory"
temp_gdb = "D:\\baks\\dont_use\\temp.gdb"
testHaul = "in_memory\\testHaul"

# Process: Table to Table
arcpy.TableToTable_conversion(test_csv, in_memory, "testHaul")

arcpy.TableToTable_conversion(testHaul, temp_gdb, "testHaul_nonnull", "\"COMMENT\" IS NOT NULL AND \"COMMENT\" <> '' AND \"COMMENT\" <> ' '")

arcpy.TableToTable_conversion(testHaul, temp_gdb, "testHaul_nulls", "\"COMMENT\" IS NULL OR \"COMMENT\" = '' OR \"COMMENT\" = ' '")

which would put your temp table in memory, which is much faster (since the expressions doesn't appear to work in table2table with csv files....another bug??), then convert to FGDB table using the expression on the second table2table.

My data, COMMENT is the field I am testing for NULL. Actually, my expression tests if NULL, or if equal to blank or if equal to single space (all not valid data). So, when run, I get one table "testHaul_nonnull" with the all rows where COMMENT is not Null or blank.

The last TableToTable is basically a reverse of the expression of the one before it. So, it will make a new table with all rows with COMMENT = null, blank, and single space as a new table with testHaul_nulls.

You could also iterate through and use a cursor, put expression on the cursor, etc., but I think that would take longer. the above seems like a quick/easy way to get both the non nulls to a "clean" table you can use for input as well as a table with all the "errors" that were stripped out.

R_

Re: Error message even though script finishes with expected results.

T__WayneWhitley — Sat, 11 Dec 2021 13:42:11 GMT

Rhett's approach is clean, I like it...this is just additional info, a little late, but here it is--- basically, a None type obj is returned by python...

You can access a csv with an arcpy search cursor as shown below, possibly with the 'data access' search cursor (arcpy.da.searchcursor). The below is a simple demo from 10.0 (da is new at 10.1).

This is my 'junk' csv file contents, set up just for this demo:

field0,field1,field2,field3
1254,text,1258,562
,,,
 ,   ,      , 
99.99,, 515, 621   
5424,mytext,2301.123,562

...and this is the simple code (entered from IDLE) showing cursor access (just printing to the screen):

>>> import arcpy
>>> arcpy.env.workspace = r'C:\Documents and Settings\whitley-wayne\Desktop'

>>> CSV = 'testCSV1.csv'

>>> fields = arcpy.ListFields(CSV)

>>> rows = arcpy.SearchCursor(CSV)

>>> lineTXT = ''

>>> for row in rows:
 for fieldObj in fields:
  lineTXT += '\'' + str(row.getValue(fieldObj.name))+ '\'\t'
 print lineTXT
 lineTXT = ''

 
'1254.0' 'text' '1258.0' '562' 
'None' 'None' 'None' 'None' 
'None' 'None' 'None' 'None' 
'99.99' 'None' '515.0' '621' 
'5424.0' 'mytext' '2301.123' '562' 
>>>

Anyway, that was short and sweet - maybe that'll help in getting started with cursors...
And by the way, the analogous test for Null in Python simply use the keyword None (no quotes), i.e., in an 'if' statement:

if row.getValue(field) is None:
     # then do something, for example...
     myFlag = True

Enjoy,
Wayne

Re: Error message even though script finishes with expected results.

KeithMcKinnon2 — Mon, 11 Aug 2014 16:54:03 GMT

I am getting the same error when running the XY to Line tool directly in ArcMap using a table in a file geodatabase.