Insert Cursor to Table In Memory

[ClintonCooper1]

I am trying to take a large dataset and import (export or append) it into an "in memory" table where I can then run the calculations:  I need to import three fields ( SOS_VOTERID, FEAT_SEQ and YEAR_Of_BIRTH).  For my code below, I am just working with 2.  I believe I need to run a search cursor on my orig table, and then run an insert cursor to import the data into my new table.  I am running into an error that says:

Runtime error  Traceback (most recent call last):   File "<string>", line 23, in <module> TypeError: sequence size must match size of the row 

import arcpy, collections  from arcpy import env env.workspace = r"C:\Users\cc1\Desktop\NEW.gdb\WAYNE" table = "WAYNE"  table2 = arcpy.CreateTable_management("in_memory", "WAYNE") arcpy.AddField_management(table2, "SOS_VOTERID","TEXT", field_length=25) arcpy.AddField_management(table2, "FEAT_SEQ","LONG")  newList = {row[0]: row[1] for row in arcpy.da.SearchCursor(table, ["SOS_VOTERID","FEAT_SEQ"])}                                         tbl = arcpy.ListTables("*")                       for table in tbl:     fieldList = arcpy.ListFields(table)     for field in fieldList:         newList.append([table,field.name])    #  this populates the new list with table and field to directly insert to new table with arcpy.da.InsertCursor(table2, ['SOS_VOTERID', 'FEAT_SEQ']) as insert:     for f in newList:                                          insert.insertRow(f) del insert 

Anyone know where I am going wrong?  Thanks!

Clinton

[ChrisSnyder]

Another option is to just use a dictionary... this way there is no dependency on 3rd party modules. Although I admit I need to check out pandas as it seems pretty cool.

The dictionary option though is a built-in, simple, and powerful one, and can be made to be quite robust with not a lot of code. A very simple example:

valueDict = {r[0]:[r[1],r[2]] for r in arcpy.da.SearchCursor(myFC, ["OID@","ID","NAME"])}
#add/calc a new "field" that is float(OID - 1) * OID
for key in valueDict:
   valueDict[key].append(float(key -1) * key)

However, the real power comes from building RDBMS-like table structures, and being able to run complex queries (especially recursive ones) crazy fast.

Adding calcing fields via Dictionaries or Numpy Arrays (pandas) will be orders of magnitude faster than any ESRI table/tool - even in the in_memory workspace.

Clinton, how many records are you dealing with here... 23 hours seems a bit over the top.

[JoshuaBixby]

I know this thread is older, but I figured I would chime in because I have been running into problems with using in_memory workspaces.

When it comes to using the in_memory workspace in the interactive python window when background processing is enabled; well, all bets are off.  There are numerous tools that don't actually create GPInMemoryWorkspaces when using 'in_memory' with 'Background Processing' in the interactive Python window.  I primarily work in ArcGIS 10.1, but I have also noticed the same is true with 10.2.1.

If 'in_memory' doesn't mean in RAM, then performance could be much lower than expected.  If using 'Background Processing' with the interactive Python windows, it pays to double check whether your 'in_memory' objects are actually in memory and not in disk-based temporary file geodatabases.