Open DBase file, make XY-event, save shapefile... how?

MiriamDorigo · ‎08-05-2013

Hi all,

Today I've started using Python for the very first time and therefor I am not very adept. I've bought a book and I am using ArcGIS Help and the forum extensively, so I've solved many of the problems I was dealing with. But what should be the simplest of all I cannot solve. I am used to SPSS and use syntax a lot, and I'm no fool when it comes to data handling. I'm not new to ArcGIS either. Just the combination of ArcGIS/Python is new to me.

I want to do the following things:
1) Add data, namely add a .csv file to the opened Arcmap file
2) Create XY-events from the .csv file
3) Data: save as Shapefile
4) Add some characteristics to the Shapefile
5) Save again

I've succeeded in automating step 4 using Python. With steps 1, 2, 3 and 5 I have the following issues. If you would be willing to help me with one of more of these problems that would be amazing. Ps. I know all of the things I want to do are possible through menus but I am explicitly looking for Python code as so to automate the process.

1) I simply cannot find any code that will allow me to open a .csv file into an open ArcMap. My book nor the help pages are illuminating to me, perhaps because I don't use the correct search terms?
2) I can't create XY events from a .cvs file. I can be done from a shapefile, but sadly, at this point in the process I have no shapefile, I have only .csv file with the variables for latitude and longitude. When I work manually (through the menus), the order of execution is always: import the .csv file, create XY-events using the latitude and longitude variables, and save XY-events as a shapefile. That's why I'm confused why I can't create XY-events from the .csv file.
3) What is the code to save my XY-events as a shapefile?
5) And what's the code to save a shapefile again?

Thank you so very much,

Miriam (from the Netherlands)

T__WayneWhitley · ‎08-05-2013

Well, you could start here (the link below) - notice that csv (or other txt) input is allowed, although you may have to adjust how ArcGIS 'sees' this input with an accompanying schema.ini file. All of this is in the usage section. Work on that, then you can use Copy Features to make a permanent shapefile (this is noted on the help page for Make XY Event Layer as well).

Make XY Event Layer (Data Management)
Desktop » Geoprocessing » Tool reference » Data Management toolbox
http://resources.arcgis.com/en/help/main/10.1/index.html#//00170000006z000000

Welcome and have fun!
Wayne

SchoppMatthieu · ‎08-05-2013

Hi,

I've scripted something that does what you described excepted that no .csv file is added to the map. I'm not sure you really need the csv in the mapDocument but if you really do, try to google something like 'arcpy 10.1 AddTableView'

I let you read the comments in the code to understand the process.
Only 'toto.mxd' and 'test.csv' exist at the first place. 'toto.lyr' and 'toto.shp' are generated from the script

I've attached the folder that contains the whole thing just in case

Here you go :

import arcpy, os

from arcpy import env
env.overwriteOutput = True  # -----------------------------------------------------> IF YOU WANT THE OUTPUT TO BE OVERWRITED



# ------- DEFINE VARIABLES


directory = r'D:\GAIA\AG\script' # -------------------------------------------------> WHERE MY STUFF IS

mxd = arcpy.mapping.MapDocument(os.path.join(directory, r'toto.mxd')) # ------------> MXD REFERENCE

tableCSV = os.path.join(directory, r'test.csv') # ----------------------------------> TABLE REFERENCE

temp_Layer = os.path.join(directory,r'toto.lyr') # ---------------------------------> NAME FOR THE TEMPORARY LAYER TO BE GENERATED FROM 'MakeXYEventLayer_management'

temp_Layer_toShapefile = os.path.join(directory,r'toto.shp') # ---------------------> NAME FOR THE SHAPEFILE TO SAVE



# -------- CREATE XY EVENT TABLE AND SAVE IT AS A SHAPEFILE :

arcpy.MakeXYEventLayer_management(tableCSV, "X_COORD", "Y_COORD", temp_Layer) # ----> TABLE TO XY Event LAYER (temporary Layer) (inputTable, ColumnNameForX, ColumnNameForY, LayerToSave)

arcpy.CopyFeatures_management(temp_Layer, temp_Layer_toShapefile) # ----------------> COPY TEMP LAYER TO PROPER SHAPEFILE



# -------- ADD SHAPEFILE TO THE MAP :


layer_ToAdd = arcpy.mapping.Layer(temp_Layer_toShapefile) # ------------------------> CAST SHAPEFILE AS LAYER OBJECT

df = arcpy.mapping.ListDataFrames(mxd, "Layers")[0] # ------------------------------> DATAFRAME REFERENCE IN MXD (DEFAULT = "Layers")

arcpy.mapping.AddLayer(df, layer_ToAdd, "TOP") # -----------------------------------> ADD LAYER TO MXD



# -------- SAVE, CLEAN UP, EXIT

mxd.save() # -----------------------------------------------------------------------> SAVE MXD

del mxd
del temp_Layer_toShapefile # --------------------------------------------------------------------------> DELETE MEMORY REFERENCE TO THE MXD (kill lock)

print 'JOB DONE'

MiriamDorigo · ‎08-06-2013

Hi both,

thank you for the replies, it seems I can get much further then I thought 🙂 I will be able to implement them this afternoon and post the results asap.

Miriam

MiriamDorigo · ‎08-07-2013

The code that Matshopp has provided me with seems to work just fine, so thank you!

The only problem that remains for me is that I when I use it on the files that I have, this takes quite some time because they're too large. They are between 150 mb and 1 gig, text files. This is why I think the best solution is to modify them one by one, loading all the files seperately in a seperate arcmap document.

Do you guys have any other hints and tips on how to work with such large files in Arcgis? Other data-programs can handle them, such as SPSS and Matlab, but ArcGIS doesn't seem to, or at least, it's a lengthy process. Is it simply impossible for gis to handle these files quicker? I am using an I7 computer and so I thought that it should work...

Miriam