Using in_memory

[ScottBlankenbeckler]

I want to build my input table in memory to help speed the geocoding process up. However, when I try it tells me the table does not exsist. What am I doing wrong?

import arcpy

Input_address = arcpy.GetParameterAsText(0)
Input_city = arcpy.GetParameterAsText(1)
Input_state = arcpy.GetParameterAsText(2)
Input_zip = arcpy.GetParameterAsText(3)

address = Input_address
city = Input_city
state = Input_state
zipcode = Input_zip
print address + ', ' + city + ', ' + state, zipcode

#Table Creation
Distance_gdb = 'in_memory\\'
Test_DB = Distance_gdb + '\\temp.dbf'

arcpy.CreateTable_management(Distance_gdb, "temp.dbf", "", "")
arcpy.AddField_management(Test_DB, "address", "TEXT", "", "", "", "", "NON_NULLABLE", "REQUIRED", "")
arcpy.AddField_management(Test_DB, "city", "TEXT", "", "", "", "", "NON_NULLABLE", "NON_REQUIRED", "")
arcpy.AddField_management(Test_DB, "state", "TEXT", "", "", "", "", "NON_NULLABLE", "NON_REQUIRED", "")
arcpy.AddField_management(Test_DB, "zipcode", "TEXT", "", "", "", "", "NULLABLE", "NON_REQUIRED", "")

Add_field = 'address'
City_field = 'city'
state_field = 'state'
zip_field = 'zipcode'

#Table populate
cur = arcpy.InsertCursor(Test_DB)
row = cur.newRow()
row.setValue(Add_field, address)
row.setValue(City_field, city)
row.setValue(state_field, state)
row.setValue(zip_field, zipcode)
cur.insertRow(row)

#Geocode 
Input_Table = Test_DB 
US_StreetAddress_Enhanced = "Z:\\NewYork\\US_StreetAddress_Enhanced" #Locator
Geo_Address = 'in_memory\\Geo_Point' #Geocoded results
arcpy.GeocodeAddresses_geocoding(Input_Table, US_StreetAddress_Enhanced, "Street address VISIBLE NONE;City City VISIBLE NONE;State State VISIBLE NONE;ZIP zipcode VISIBLE NONE", Geo_Address, "STATIC")

The table creation works fine but the Add Field fails

arcgisscripting.ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000732: Input Table: Dataset in_memory\\temp.dbf does not exist or is not supported
Failed to execute (AddField).

[DarrenWiens2]

I'll go out on a limb and say you can't make a dBase file in the in_memory workspace. My understanding is that in_memory is like a GDB-lite (only GDB tables and feature classes allowed). I'd try deleting all ".dbf"s.

[ScottBlankenbeckler]

You can put tables in memory. In another script I am placing the results of arcpy.GenerateNearTable_analysis(Geo_Address_P, "Zip_p2", Table_of_Zipcode_Distances, "", "NO_LOCATION", "NO_ANGLE", "ALL", "0")
into memory and  then using it for further processes.

DOH now I understand what you are saying. If I tell it just \\temp and not \\temp.dbf if works.

Now my only problem is it still takes way to long to create and geocode. 2.7s to Create and 38.6 sec to geocode.

[MathewCoyle]

Aye, even though it lets you "create" the table in memory using .dbf, it will simply ignore that and reference the base name. Something like the following should work.

...
arcpy.CreateTable_management("in_memory", "temp")
arcpy.AddField_management(r"in_memory\temp", "address", "TEXT", "", "", "", "", "NON_NULLABLE", "REQUIRED", "")
...

[ScottBlankenbeckler]

$ python test4.py '301 Park Ave' 'New York' NY 10022
301 Park Ave, New York, NY 10022
Time elapsed Table Create:  2.65299987793 s
Time elapsed Table Populate:  0.0160000324249 s
Time elapsed Geocode:  14.6960000992 s

Any suggestions on how I can speed this process up?

[MathewCoyle]

Any suggestions on how I can speed this process up?

Don't know much about geocoding first hand, but came across this.

http://blogs.esri.com/dev/blogs/geocoding/archive/2011/02/09/tuning-a-locator-for-improved-performan...

[AnthonyFarndon]

I always thought that when you use in_memory, it creates a file geodatabase in your temp drive, so it is not really 'in memory', there will likely be minimal performance gains, it is more of a convenience factor (although with it being stated using this does improve performance, I may be wrong).

Some goods tips on improving performance from the 2011 UC here:

http://resources.arcgis.com/gallery/file/geoprocessing/details?entryID=3E582F8E-1422-2418-88A4-9DF6F...

[MathewCoyle]

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//002w0000005s000000

in_memory is in memory, if it were on disk there wouldn't be restrictions on what operations could or couldn't be written to it. It may reference a location on the hard drive for some file management reason over my head, but it is held directly in memory as well.

Also, from the UC performance tips slides you linked

When you write intermediate data
-Do use in_memoryworkspace

[AnthonyFarndon]

http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//002w0000005s000000

in_memory is in memory

Ah, so it is, I stand corrected.

[ChrisSnyder]

While there is probably some performance increase having the address list stored in_memory, the real perfomance gain would be any enhancements that could be made to your geocoding service... Have you set it up correctly and in the most efficient way possible?

Have you ever seen this? http://code.google.com/p/geopy

It's pretty dang cool...