topic Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python in Python Questions

Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

CCWeedcontrol — Sat, 11 Dec 2021 09:21:20 GMT

I am trying to test some of my 2.7 python scripts on 3.6 python and i noticed a huge difference on amount of time it takes to run the FeatureClassToFeatureClass_conversion process. so with 3.6 python the process took 18 min 5 secs, with 2.6 python it took only 8 seconds. Does anyone know why the it take much longer in 3.6 Python?

This is what i am using, there is more fields than this i just removed a lot of them.

PT1 = "C:/Temp/Scratchworkspace.gdb/Drain"

def Layers15(PT1):
    FieldMapString = "" \

                + """PIN "PIN" true true false 13 Text 0 0 ,First,#,""" + PT1 + """,Pin,-1,-1;"""\
                + """ACRES "ACRES" true true false 4 Double 0 0  ,First,#,""" + PT1+ """,ACRES,-1,-1;"""\
                + """Instrument "Instrument" true true false 10 Text 0 0 ,First,#,""" + PT1 + """,Instrument,-1,-1;"""\
                + """SiteAddres "SiteAddres" true true false 106 Text 0 0 ,First,#,""" + PT1 + """,SiteAddres,-1,-1;"""\
                + """SiteCity "SiteCity" true true false 32 Text 0 0 ,First,#,""" + PT1 + """,SiteCity,-1,-1;"""\
                + """SiteZip "SiteZip" true true false 10 Text 0 0 ,First,#,""" + PT1 + """,SiteZip,-1,-1;"""\
                + """SubName "SubName" true true false 20 Text 0 0 ,First,#,""" + PT1 + """,SubName,-1,-1;"""\       
                            

    fieldmappings = arcpy.FieldMappings()
    fieldmappings.loadFromString(FieldMapString)
    return fieldmappings


def main(args=None):
        if args is None:
                args = sys.argv

# Process: Feature Class to Feature Class
arcpy.FeatureClassToFeatureClass_conversion(PT1, "C:/Temp/Scratchworkspace.gdb", "PT_ALL","", Layers15(PT1),"")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

DanPatterson_Retired — Sat, 11 Dec 2021 09:21:22 GMT

PT1 = 'HELLO'

FieldMapString = '''
PIN "PIN" true true false 13 Text 0 0 ,First,#, {0}, Pin,-1,-1;
ACRES "ACRES" true true false 4 Double 0 0  ,First,#, {0}, ACRES,-1,-1;
Instrument "Instrument" true true false 10 Text 0 0 ,First,#, {0}, Instrument,-1,-1;
SiteAddres "SiteAddres" true true false 106 Text 0 0 ,First,#, {0}, SiteAddres,-1,-1;
SiteCity "SiteCity" true true false 32 Text 0 0 ,First,#, {0}, SiteCity,-1,-1;
SiteZip "SiteZip" true true false 10 Text 0 0 ,First,#, {0}, SiteZip,-1,-1;
SubName "SubName" true true false 20 Text 0 0 ,First,#, {0}, SubName,-1,-1;
'''
print(FieldMapString.format(PT1))‍‍‍‍‍‍‍‍‍‍‍‍

def Layers15(PT1, FieldMapString):
    """pass the variable and the constant into
    the function
    """
    fieldmappings = arcpy.FieldMappings()
    fieldmappings.loadFromString(FieldMapString)
    return fieldmappings‍‍‍‍‍‍‍

don't know about the speed thing, but that ugly string in a function surely had to go.

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

JoshuaBixby — Tue, 05 Feb 2019 02:31:51 GMT

Is this code running on the same machine?

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

CCWeedcontrol — Tue, 05 Feb 2019 16:14:41 GMT

It's always been the way i have done it. Thanks for the suggestion i will try it out.

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

CCWeedcontrol — Tue, 05 Feb 2019 16:15:07 GMT

Joshua,

Yes very same computer. None of the code was changed.

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

CCWeedcontrol — Tue, 05 Feb 2019 18:33:53 GMT

With Dan Petterson suggestions it took 16 min 40 secs in Python 3.6.

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

DanPatterson_Retired — Tue, 05 Feb 2019 19:05:07 GMT

is your field mapping necessary? Just try a copy features tool to compare speed. It may be either things are slower in your environment or it is now just slower.

I haven't noticed any tool slowness at all in Pro 2.3, but I have beefy machines and I use local data. It could be the way data is served/handled from sde or whatever if it isn't locally stored. It seems to be a common thread for those that are on the "slow train"

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

CCWeedcontrol — Tue, 05 Feb 2019 19:11:03 GMT

Yes because the field mapping rearranges the fields which is linked to a reporting tool. Unless there is another way to rearrange the fields?

All my data is on my local PC, i have noticed slowness with Pro 2.2. I haven't upgraded to Pro 2.3 yet.

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

DanPatterson_Retired — Tue, 05 Feb 2019 19:19:42 GMT

When you run Check My ability to run ArcGIS pro on the help page

ArcGIS Pro 2.3 system requirements—ArcGIS Pro | ArcGIS Desktop

does it pass... errrr more importantly, does it surpass the highest recommendations?

Do you run other stuff like browsers, social media and other unnecessary fluff when you are trying this?

Pro is going to be slower apparently since people are complaining about it, but I can assure you it has nothing to do with Python

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

CCWeedcontrol — Tue, 05 Feb 2019 21:39:17 GMT

My requirements pass.

When i run that block of code in 2.7 and 3.6 i had the same stuff open.

I restarted my PC and just run the script with python 2.7 the process took 10 secs. Then i closed python2.7 and opened python 3.6 and run the code it took 16 min 40 secs. Seem like the issue has to do with python 3.6 because i run the exactly same code. My python 2.7 is 32 bit, my 3.6 is 64 bit if that makes a difference.

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

DanPatterson_Retired — Sat, 11 Dec 2021 09:21:25 GMT

the arcpy and arcgisscripting and all the esri stuff in Pro is different, so when you call FeatureClassToFeatureClass, there is an arcmap and an ArcGIS pro version. If you will notice your pure python stuff in your code example very limited...

Try testing it yourself if you have an appropriate python ide that used line magics (ie IPython in Spyder etc)

def tester():
    PT1 = 'HELLO'
    FieldMapString = '''
    PIN "PIN" true true false 13 Text 0 0 ,First,#, {0}, Pin,-1,-1;
    ACRES "ACRES" true true false 4 Double 0 0  ,First,#, {0}, ACRES,-1,-1;
    Instrument "Instrument" true true false 10 Text 0 0 ,First,#, {0}, Instrument,-1,-1;
    SiteAddres "SiteAddres" true true false 106 Text 0 0 ,First,#, {0}, SiteAddres,-1,-1;
    SiteCity "SiteCity" true true false 32 Text 0 0 ,First,#, {0}, SiteCity,-1,-1;
    SiteZip "SiteZip" true true false 10 Text 0 0 ,First,#, {0}, SiteZip,-1,-1;
    SubName "SubName" true true false 20 Text 0 0 ,First,#, {0}, SubName,-1,-1;
    '''
    FieldMapString.format(PT1)
    return

# ----- timing section

tester()

%timeit tester()
2.36 µs ± 194 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

# ---- that is on my laptop, haven't tested on the desktop‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

So if yours is several orders of magnitude faster than 2ish microseconds then you can blame python 3... otherwise file a case with esri about the slowness of fc2fc with field mapping

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

CCWeedcontrol — Wed, 06 Feb 2019 15:31:57 GMT

ok i think i understand. so i tired to run what you posted to test but i am getting invalid syntax on line with %timeit Tester()

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

CCWeedcontrol — Sat, 11 Dec 2021 09:21:28 GMT

I upgrade to Pro 2.3 this morning and now i am getting this error on line 81 and i didn't change the script at all.

RuntimeError: ERROR 999999: Something unexpected caused the tool to fail. Contact Esri Technical Support (http://esriurl.com/support) to Report a Bug, and refer to the error help for potential solutions or workarounds.

import sys, arcpy, os, fnmatch

from datetime import datetime as d
startTime = d.now()
start_time = time.time()


# Local variables:
print ('Started')
arcpy.env.overwriteOutput = True
arcpy.env.workspace = "C:/Temp/Scratchworkspace.gdb"
arcpy.env.qualifiedFieldNames = False
arcpy.env.preserveGlobalIds = True

PT1 = "C:/Temp/Scratchworkspace.gdb/Drain"

FieldMapString = '''
DXF_TEXT "DXF_TEXT" true true false 11 Text 0 0 ,First,#, {0}, ACRES,-1,-1;
ACCOUNT "ACCOUNT" true true false 11 Text 0 0 ,First,#, {0}, ACCOUNT,-1,-1;
PIN "PIN" true true false 13 Text 0 0 ,First,#, {0}, PIN,-1,-1;
ACRES "ACRES" true true false 4 Double 0 0  ,First,#, {0}, ACRES,-1,-1;
Instrument "Instrument" true true false 10 Text 0 0 ,First,#, {0}, Instrument,-1,-1;
SiteAddres "SiteAddres" true true false 106 Text 0 0 ,First,#, {0}, SiteAddres,-1,-1;
SiteCity "SiteCity" true true false 32 Text 0 0 ,First,#, {0}, SiteZip,-1,-1;
SiteZip "SiteZip" true true false 10 Text 0 0 ,First,#, {0}, SiteZip,-1,-1;

'''
#print(FieldMapString.format(PT1))

def Layers15(PT1, FieldMapString):
    """pass the variable and the constant into
    the function
    """
    fieldmappings = arcpy.FieldMappings()
    fieldmappings.loadFromString(FieldMapString)
    return fieldmappings

def main(args=None):
        if args is None:
                args = sys.argv

# Process: Feature Class to Feature Class
arcpy.FeatureClassToFeatureClass_conversion(PT1, "C:/Temp/Scratchworkspace.gdb", "PT_ALL","", Layers15(PT1, FieldMapString),"")

print ("Finished")
print ('(Elapsed time: ' + str(d.now() - startTime)[:-3] + ')')

try:
    print ('(Elapsed time: ' + str(d.now() - startTime)[:-3] + ')')

except Exception as e:
    # If an error occurred, print line number and error message
    import traceback, sys
    tb = sys.exc_info()[2]
    print ("Line %i" % tb.tb_lineno)
    print (e.message)
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

DanPatterson_Retired — Wed, 06 Feb 2019 19:30:26 GMT

which means you don't have 'line magics' so you have an older IDE (try spyder, you will have to replace timing stuff with whatever you use)

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

DanPatterson_Retired — Wed, 06 Feb 2019 19:41:35 GMT

Over the last 10 years since python 2.x, python 3 has had a number of changes to the datetime module and dates in general

datetime — Basic date and time types — Python 3.7.2 documentation

you should check out some of the changes and/or new features

Re: Feature Class To FeatureClass conversion 2.7 python vs 3.6 Python

JoshuaBixby — Thu, 07 Feb 2019 23:35:46 GMT

You answered Dan's question about the field mapping being necessary, but did you try copying without using a field mapping? In terms of isolating the issue, it is important to know whether the field mapping is contributing to the problem or if it something else.