code very slow in jupyter notebook inside ArcGIS Pro 3.4?

[JohannesBierer]

Hello, 

The following code is very slow in jupyter notebook, why? Any Ideas? ArcGIS Pro 3.4.3

It's even many times faster to go through the steps manually with the same geoprocessing tools? An excel table with about 50 000 points takes an hour or more to process? To write in memory isn't really faster in the code? 

import arcpy
import os

input_folder = r"Folder"
gdb = r"File GDB"

x_field = "X"
y_field = "Y"
wgs84 = arcpy.SpatialReference(4326)
etrs32 = arcpy.SpatialReference(25832)

arcpy.env.overwriteOutput = True

def ensure_numeric_fields(table, x_field, y_field):
    """
    Converts X/Y fields to numeric FLOAT fields if needed.
    Creates new fields X_NUM and Y_NUM and fills them with numeric values.
    """
    # New safe numeric fields
    x_num = x_field + "_NUM"
    y_num = y_field + "_NUM"

    # Create numeric fields if not already present
    if x_num not in [f.name for f in arcpy.ListFields(table)]:
        arcpy.management.AddField(table, x_num, "DOUBLE")
    if y_num not in [f.name for f in arcpy.ListFields(table)]:
        arcpy.management.AddField(table, y_num, "DOUBLE")

    # Convert text numbers → float
    with arcpy.da.UpdateCursor(table, [x_field, y_field, x_num, y_num]) as cursor:
        for row in cursor:
            try:
                row[2] = float(str(row[0]).replace(",", "."))  # X
                row[3] = float(str(row[1]).replace(",", "."))  # Y
            except:
                row[2] = None
                row[3] = None
            cursor.updateRow(row)

    return x_num, y_num


excel_files = [f for f in os.listdir(input_folder) if f.lower().endswith((".xls", ".xlsx"))]

for excel in excel_files:
    excel_path = os.path.join(input_folder, excel)
    name = os.path.splitext(excel)[0]

    print(f"Processing: {excel}")

    # 1 — Convert Excel to table
    table_out = os.path.join(gdb, f"{name}_tbl")
    arcpy.ExcelToTable_conversion(excel_path, table_out)

    # 2 — Make sure X and Y are numeric
    numeric_x, numeric_y = ensure_numeric_fields(table_out, x_field, y_field)

    # 3 — Now create XY points from numeric fields
    fc_wgs = os.path.join(gdb, f"{name}_WGS84")
    arcpy.management.XYTableToPoint(
        table_out,
        fc_wgs,
        numeric_x,
        numeric_y,
        coordinate_system=wgs84
    )

    # 4 — Project to EPSG:25832
    fc_etrs = os.path.join(gdb, f"{name}_25832")
    arcpy.management.Project(fc_wgs, fc_etrs, etrs32)

    print(f" → Created {fc_etrs}")

print("All Excel files processed.")

If I use python console in QGIS it's finished under one minute?

[AlfredBaldenweck · Accepted solution]

Is it slow if you copy-paste into the Python Window or wrap it in a script tool?

Reason I ask is that there are several funky things between notebooks, python window, and toolboxes where they don't match up between the three.

[JohannesBierer]

If I use the Python Window it's faster I think, about 20 Minutes for my table?  Script Tool is really fast, about a minute ... I think that's strange?

[AlfredBaldenweck]

Bizarre but unfortunately kind of...

I ran into a case earlier this summer where it turns out that arcpy.ImportToolbox() works fine to import geoprocessing services in the python window and a notebook, but doesn't do anything if it's used in a toolbox (atbx or pyt). 

Not really sure what's happening back there as to why each one is so different, but...