One of my users has a script that works in 2.7 and was working with 3.x but now breaks after the ArcGIS Pro 2.4.1 update. When copying an SDE table into memory using the arcpy.CopyRows_management all text field sizes are increasing from what there are to 5000. For example, text fields of size 1 and 100 become 5000. Our tables are a SQL 2012 / SDE 10.4.1 geodatabase. To see if I could replicate this issue I even created an empty table from scratch containing only two fields, the first is DataSource defined text length 100 and the other is Flag defined as text length 1.
Is anyone else seeing this? The user says it was working in version 3 before we pushed out Pro 2.4.1.
c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3>python.exe h:\treg\test_inmemory2.py
SDE table is : H:\Treg\(Prod) MyDatabase@Reader.sde\MyDatabase.DBO.WS_TestOBJECTID is a type OID with a length of 4
DataSource is a type String with a length of 100
Flag is a type String with a length of 1Copying table H:\Treg\(Prod) MyDatabase@Reader.sde\MyDatabase.DBO.WS_Test into memory...
temp table is : memory\tmpTable
OBJECTID is a type OID with a length of 4
DataSource is a type String with a length of 5000
Flag is a type String with a length of 5000
try:
##Import modules
import os, sys
import os.path as path
import traceback
import arcpy
from arcpy import env
arcpy.env.overwriteOutput = True
sdeTable = r"H:\Treg\(Prod) MyDatabase@Reader.sde\MyDatabase.DBO.WS_Test"
tmpTable = r"memory\tmpTable"
print('SDE table is : {}\n' .format(sdeTable))
##check field lengths in SDE text fields
fields = arcpy.ListFields(sdeTable)
for field in fields:
print('{0} is a type {1} with a length of {2}'.format(field.name, field.type, field.length))
print ('\nCopying table {} into memory...'.format(sdeTable))
arcpy.CopyRows_management(sdeTable, tmpTable)
print('\ntemp table is : {}\n' .format(tmpTable))
##check field length on in memory text fields
fields = arcpy.ListFields(tmpTable)
for field in fields:
print('{0} is a type {1} with a length of {2}'.format(field.name, field.type, field.length))
except:
print ('\n*** ERROR IN SCRIPT!')
print ('\n***PYTHON ERROR INFO: \n' + str(traceback.format_exc()))
Solved! Go to Solution.
What happens when you use the old/legacy "in_memory" in Pro instead of using the new "memory" workspace? It could be a difference in the in-memory workspace implementation more than Pro or ArcMap.
Reading over,
One of my users has a script that works in 2.7 and was working with 3.x but now breaks after the ArcGIS Pro 2.4.1 update.
The first part of the statement implies it is an ArcMap vs. ArcGIS Pro issue since ArcMap is Python 2.x and ArcGIS Pro is Python 3.x; however, the second part of the statement implies it is an ArcGIS Pro 2.4 vs Pro 2.4.1 issue.
Does the script work in Pro 2.4?
Hi Josh,
You are correct, I didn't mention Pro vs ArcMap because we run these scripts in a CMD window. Also, we upgraded all of our ArcGIS Pros to 2.4.1 so I haven't been able to test it with an early version of Pro. I changed the script to run with python 2.7 to verify that it, at least, worked correctly in ArcMap python. The only change needed was changing line 14 from memory to in_memory. Having said that, CopyRows shouldn't be changing field definitions in the first place.
What happens when you use the old/legacy "in_memory" in Pro instead of using the new "memory" workspace? It could be a difference in the in-memory workspace implementation more than Pro or ArcMap.
Joshua, that fixed the issue. I'll report this behavior to ESRI since it seems like a bug.
Please mark a comment/suggestion as correct to close out the question.
I ran a process in a Spyder window not the python window with Pro 2.4 installed on my desktop and while I don't have an answer for you, I can confirm that a text field from an egdb to in_memory stayed the same length....
I am experiencing something very similar. I copy parcels to 'memory', add fields using a dictionary that comes from a template feature class. Every text/string field comes out as 5000.
edit: python 3
I am experiencing the same issue in Arc Pro 2.5
I installed a fresh copy of Arc Pro 2.4 on a new laptop for some testing:
Copying into 'memory' changes all text fields to length 5000
Copy into 'in_memory' maintains the text field sizes.
Since the new capabilities of the 'memory' space are important to my project, going to 'in_memory' is not really a viable option.
Can anyone confirm that this has actually been entered as a bug at Esri, and is being worked on?
If it is a bug, it isn't in the posted bug list, so maybe it hasn't been reported
https://support.esri.com/en/Search-Results#search?q=memory%20workspace&content-type=Bugs
Kory Kramer 5000 sounds familiar, but not sure if it is associated with the memory workspace