nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Problem with CopyRows_management to in memory with Python 3

RandyKreuziger

One of my users has a script that works in 2.7 and was working with 3.x but now breaks after the ArcGIS Pro 2.4.1 update. When copying an SDE table into memory using the arcpy.CopyRows_management all text field sizes are increasing from what there are to 5000. For example, text fields of size 1 and 100 become 5000. Our tables are a SQL 2012 / SDE 10.4.1 geodatabase. To see if I could replicate this issue I even created an empty table from scratch containing only two fields, the first is DataSource defined text length 100 and the other is Flag defined as text length 1.

Is anyone else seeing this? The user says it was working in version 3 before we pushed out Pro 2.4.1.

c:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3>python.exe h:\treg\test_inmemory2.py
SDE table is : H:\Treg\(Prod) MyDatabase@Reader.sde\MyDatabase.DBO.WS_Test
OBJECTID is a type OID with a length of 4
DataSource is a type String with a length of 100
Flag is a type String with a length of 1
Copying table H:\Treg\(Prod) MyDatabase@Reader.sde\MyDatabase.DBO.WS_Test into memory...
temp table is : memory\tmpTable
OBJECTID is a type OID with a length of 4
DataSource is a type String with a length of 5000
Flag is a type String with a length of 5000

<SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
    <SPAN class="comment token">##Import modules</SPAN>
    <SPAN class="keyword token">import</SPAN> os<SPAN class="punctuation token">,</SPAN> sys
    <SPAN class="keyword token">import</SPAN> os<SPAN class="punctuation token">.</SPAN>path <SPAN class="keyword token">as</SPAN> path
    <SPAN class="keyword token">import</SPAN> traceback

    <SPAN class="keyword token">import</SPAN> arcpy
    <SPAN class="keyword token">from</SPAN> arcpy <SPAN class="keyword token">import</SPAN> env

    arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>overwriteOutput <SPAN class="operator token">=</SPAN> <SPAN class="token boolean">True</SPAN>

    
    sdeTable <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"H:\Treg\(Prod) MyDatabase@Reader.sde\MyDatabase.DBO.WS_Test"</SPAN>
    tmpTable <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"memory\tmpTable"</SPAN>    

    <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'SDE table is : {}\n'</SPAN> <SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>sdeTable<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>  
    
    <SPAN class="comment token">##check field lengths in SDE text fields</SPAN>
    fields <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>ListFields<SPAN class="punctuation token">(</SPAN>sdeTable<SPAN class="punctuation token">)</SPAN>
    <SPAN class="keyword token">for</SPAN> field <SPAN class="keyword token">in</SPAN> fields<SPAN class="punctuation token">:</SPAN>
        <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'{0} is a type {1} with a length of {2}'</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>field<SPAN class="punctuation token">.</SPAN>name<SPAN class="punctuation token">,</SPAN> field<SPAN class="punctuation token">.</SPAN>type<SPAN class="punctuation token">,</SPAN> field<SPAN class="punctuation token">.</SPAN>length<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>

    <SPAN class="keyword token">print</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'\nCopying table {} into memory...'</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>sdeTable<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
    arcpy<SPAN class="punctuation token">.</SPAN>CopyRows_management<SPAN class="punctuation token">(</SPAN>sdeTable<SPAN class="punctuation token">,</SPAN> tmpTable<SPAN class="punctuation token">)</SPAN>    
    

    <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'\ntemp table is : {}\n'</SPAN> <SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>tmpTable<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>    

    <SPAN class="comment token">##check field length on in memory text fields</SPAN>
    fields <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>ListFields<SPAN class="punctuation token">(</SPAN>tmpTable<SPAN class="punctuation token">)</SPAN>
    <SPAN class="keyword token">for</SPAN> field <SPAN class="keyword token">in</SPAN> fields<SPAN class="punctuation token">:</SPAN>
        <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'{0} is a type {1} with a length of {2}'</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>field<SPAN class="punctuation token">.</SPAN>name<SPAN class="punctuation token">,</SPAN> field<SPAN class="punctuation token">.</SPAN>type<SPAN class="punctuation token">,</SPAN> field<SPAN class="punctuation token">.</SPAN>length<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>

<SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
    <SPAN class="keyword token">print</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'\n*** ERROR IN SCRIPT!'</SPAN><SPAN class="punctuation token">)</SPAN>
    <SPAN class="keyword token">print</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'\n***PYTHON ERROR INFO: \n'</SPAN> <SPAN class="operator token">+</SPAN>  str<SPAN class="punctuation token">(</SPAN>traceback<SPAN class="punctuation token">.</SPAN>format_exc<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

Vind meer berichten getagd met

Accepted answers

bixb0012

What happens when you use the old/legacy "in_memory" in Pro instead of using the new "memory" workspace? It could be a difference in the in-memory workspace implementation more than Pro or ArcMap.

All comments

DanPatterson_Retired

If it is a bug, it isn't in the posted bug list, so maybe it hasn't been reported

https://support.esri.com/en/Search-Results#search?q=memory%20workspace&content-type=Bugs

Kory Kramer‌ 5000 sounds familiar, but not sure if it is associated with the memory workspace

ToddTurner

I am experiencing the same issue in Arc Pro 2.5

I installed a fresh copy of Arc Pro 2.4 on a new laptop for some testing:

Copying into 'memory' changes all text fields to length 5000

Copy into 'in_memory' maintains the text field sizes.

Since the new capabilities of the 'memory' space are important to my project, going to 'in_memory' is not really a viable option.

Can anyone confirm that this has actually been entered as a bug at Esri, and is being worked on?

DanDeegan

I am experiencing something very similar. I copy parcels to 'memory', add fields using a dictionary that comes from a template feature class. Every text/string field comes out as 5000.

edit: python 3

RandyKreuziger

Joshua, that fixed the issue. I'll report this behavior to ESRI since it seems like a bug.

JoeBorgione

I ran a process in a Spyder window not the python window with Pro 2.4 installed on my desktop and while I don't have an answer for you, I can confirm that a text field from an egdb to in_memory stayed the same length....

bixb0012

What happens when you use the old/legacy "in_memory" in Pro instead of using the new "memory" workspace? It could be a difference in the in-memory workspace implementation more than Pro or ArcMap.

RandyKreuziger

Hi Josh,

You are correct, I didn't mention Pro vs ArcMap because we run these scripts in a CMD window. Also, we upgraded all of our ArcGIS Pros to 2.4.1 so I haven't been able to test it with an early version of Pro. I changed the script to run with python 2.7 to verify that it, at least, worked correctly in ArcMap python. The only change needed was changing line 14 from memory to in_memory. Having said that, CopyRows shouldn't be changing field definitions in the first place.

bixb0012

Reading over,

One of my users has a script that works in 2.7 and was working with 3.x but now breaks after the ArcGIS Pro 2.4.1 update.

The first part of the statement implies it is an ArcMap vs. ArcGIS Pro issue since ArcMap is Python 2.x and ArcGIS Pro is Python 3.x; however, the second part of the statement implies it is an ArcGIS Pro 2.4 vs Pro 2.4.1 issue.

Does the script work in Pro 2.4?

Doe mee aan het gesprek

Aangemelde leden kunnen berichten plaatsen, updates volgen en meer. Nieuw hier? Registreer een gratis account.

Snelle Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.