MakeXYEventLayer_management

[AnthonyKeogh1]

Hi,

I am trying to create a feature class from a CSV file using the MakeXYEventLayer_management function.
The function works correctly the very first time the script is run but after the script has run and the function has output a Schema.ini file for the CSV file if it is ran again the function fails. The only way to make it run successfully again is to delete the schema.ini file each time.

Does any one know a reason for this.

Thanks,
Anto

[T__WayneWhitley]

Details of your problem?  If you haven't already solved your problem please post your code, any errors you have received, your system configuration, etc., in order to get timely and more effective help.

Thanks,
Wayne

[AnthonyKeogh1]

Hi Wayne,

The only details that I can offer are the code segment below and also the contents of the schema.ini file that seems to be preventing the function from working.

locatorType = sys.argv[1] # e.g."Postal" #
        gDBPath = sys.argv[2] # e.g. "C:/Users/akeogh/Documents/ArcGIS/CSV_fGDB.gdb" #
        csvFile = sys.argv[3] # e.g. "C:/Users/akeogh/Documents/ArcGIS/AddressBase.csv" #

    
        fullDataLayer = ConfigManager.getFullDataFCName()
        memLayer = "tempFullLayer"
        xCoords = ConfigManager.getXFieldName()
        yCoords = ConfigManager.getYFieldName()
        postcode = ConfigManager.getPostcodeFieldName()
        spRef = ConfigManager.getProjectionFile()

        print("Inputs -> ")
        print("Locator: "+locatorType)
        print("GDB Path: "+gDBPath)
        print("CSV File: "+csvFile)

        if locatorType.lower() == "postal":

            # Make a point feature class from CSV file, schema.ini file created here
            print ("Making in memory feature class")
            arcpy.MakeXYEventLayer_management(csvFile, xCoords, yCoords, memLayer, spRef)

The schema.ini file contains the following, one thing to note is that there are 25 columns in the CSV but only 3 are output to the schema file:
    [AddressBase.csv]
    Col8=SUB_BUILDING_NAME Text
    Col17=XCOORDINATE Text
    Col18=YCOORDINATE Text

Also XCOORDINATE  and YCOORDINATE are doubles in the data not text but it is always output as text.

Thanks for any help you can provide.

[T__WayneWhitley]

Yes, sounds like there is nothing wrong with your code - the error is when the file is accessed as if for the first time, the ini file is written by your system to facilitate reading the text file, only it seems to be stumbling and writing it incompletely or incorrectly...then, of course, successive executions will 'find' and use that improperly written ini to attempt reading the file.

Something you can do is 'override' or provide the ini before the system writes it for you - that way you control how the file is read, hopefully allowing a correct interpretation of the file so that your script runs without error every time.

Can you zip and attach your ini and a sample of your txt file?

Meanwhile, Kim Ollivier seems to be quite an expert on the matter, see this:
http://forums.arcgis.com/threads/70873-ArcGis-reading-field-as-integer-when-it-should-be-string.#4

Also, see this post by Chris Fox (ESRI), that at 10.1 a new engine is being used - however, ini files are still honored:
http://forums.arcgis.com/threads/64976-ArcGIS-10.1-schema.ini-file-causes-arcpy.CopyRows_management-...


I suspect something is simply interfering with the way your ini file is being written, probably has nothing to do with the new 'engine', but your data file may have a minor glitch...please zip and send that for closer examination.  Realize that when you write the file yourself, a 'sampling' of your rows that would take place for the system to write it is bypassed.  That's good and bad, good in a way that you should be able to get more feedback from the file; still bad in that you will probably still have an error in the file to find and fix.

Hope that points in the right direction...

Enjoy,
Wayne

[JeffBigos · Accepted solution]

Hi Anto,

The biggest trick to working with the layer tools in the management toolbox is that they create layers in the local memory.
This is the same concept as in ArcMap with FeatureLayers that are loaded in the map.

When the MakeXYEventLayer tool runs against a file, the result is a layer that is loaded into memory.
When the tool is run a second time, and the python IDE hasn't been closed, the layer from the first run is still in memory.
This can cause a failure point in the script if it is not handled.

To handle this case you can test for the existence of the layer, if it is there delete it, then regenerate the layer.
I have pasted an example below.

I also included a common error trapping technique that I use when I need to filter returned messages and need to observe failure points.



try:
   [INDENT] #imports   
    import arcpy, sys, traceback

    #Local Variable definitions
    inFile = r"C:\Data\textfiles\locations.csv"
    outlyr = "pireps_Layer"

    #Test to see if the layer resides in local memory already
    #Delete it if it is there
    if arcpy.Exists(outlyr):
        arcpy.Delete_management(outlyr)
   
    #Run the tool and load the layer into memory
    arcpy.MakeXYEventLayer_management(inFile,"long","lat",outlyr)[/INDENT]

except:
    [INDENT]# Get the traceback object
    #
    sysX = sys.exc_info()
    tb = sys.exc_info()[2]
    tbinfo = traceback.format_tb(tb)[0]

    # Concatenate information together concerning the error into a message string
    #
    pymsg = "PYTHON ERRORS:\nTraceback info:\n" + tbinfo + "\nError Info:\n" + str(sys.exc_info()[1])
    msgs = "ArcPy ERRORS:\n" + arcpy.GetMessages(2) + "\n"

    # Return python error messages for use in script tool or Python Window
    #
    arcpy.AddError(pymsg)
    #arcpy.AddError(msgs)

    # Print Python error messages for use in Python / Python Window
    #
    print "linenumber: {0}".format(tb.tb_lineno)
    print pymsg + "\n"
    #print msgs
    ####[/INDENT]    print "Processing completed"

[AnthonyKeogh1]

Thanks for the replies guys.

That code it is working well now Jeff!

Just out of curiosity is there a way to make the MakeXYEventLayer_management write straight to a Geodatabase instead of creating an in memory Feature Class.
I have tired it already but it doesn't seem to do anything so I stayed with the in memory Feature Class, there is a chance that the CSV files will be really big in the future so storing them in memory might not be possible.

Thanks,
Anto

[T__WayneWhitley]

No not directly, but what I've done in the past is follow up with CopyFeatures...
That should do it.

Enjoy,
Wayne

[AnthonyKeogh1]

Yeah that is what I have been doing alright Wayne, thanks.

[T__WayneWhitley]

Just one more point, related to this there is the XY to Line tool:
http://resources.arcgis.com/en/help/main/10.1/index.html#//0017000000tv000000

Not sure if there is an existing similar tool for that kind of direct conversion for points, probably there is, and at least you could even make your own...in fact, one of the deprecated tools written as a system py script tool was Write Features from Text File, I think it was called.  But that required a specifically formatted input txt.  It was still pretty cool.

That's all for now, I'm glad you are up and running with your script, good work.  I had to vote up the code from Jeff Bigos and will study it more later, that and more of Kim Ollivier's code contributions.  I always forget that about in_memory... must be something about my memory (lol).

Enjoy,
Wayne

[AnthonyKeogh1]

Good point Wayne, I might have too look into that more soon.

Thanks again,
Anto