Equivalent native python for Arcpy

Question

Hello everyone,

I have a script tool (A map-matching tool) in Arcmap prepared by a peer. I want to replace it with native python code since it does not work with the huge data set that I have (more than 6000 sets of trip trajectories), and I believe it's because of the low capacity of Arcmap. I would like to know what would be the native python equivalent for these lines:

file = arcpy.GetParameterAsText(0)

for row in arcpy.da.SearchCursor(infc, ["OID@", "SHAPE@"])

with arcpy.da.InsertCursor(fc, ['SHAPE@']) as cursor

Thank you very much!

JoshuaBixby · Answer

Regarding:

that is because apparently ArcMap cannot deal with the big number of data.

The original question mentions "6000 sets of trip trajectories," which could only qualify as "big number of data" if each trip trajectory had many many millions of vertices.  Even then, one could argue that still isn't "big" data.

ArcGIS Desktop/ArcMap is a 32-bit application, which comes with the typical limitations of 32-bit applications in terms of addressable memory space.  To augment the base application for larger data sets, Esri has the Background Geoprocessing (64-bit)—ArcMap | Documentation (arcgis.com).

ArcGIS Pro is native 64-bit and can process as much data as your machine can handle.  Since ArcGIS Desktop/ArcMap will be retired in a couple years, I strongly suggest you start using ArcGIS Pro.

Since you seem quite new to Python, I don't believe trying to convert ArcPy code to open-source equivalent is a good use of your time unless your career goal is to get into geoprocessing scripting and development.

blake.terhune · Answer

Pandas might be an option if you want to work with tabular data without arcpy. Could you tell us all the arcpy functions that are used and maybe we could suggest some alternatives.

Anonymous User · Answer

It sounds like you're trying to convert an arcpy tool to a tool that relies only on open source libraries. It depends on the task - but this is likely much more effort than a community forum can support (e.g. answering single questions). If you could post the entire tool or a complete code block, that would give others a better chance to help.

I realize this isn't a response to your original question, but there are likely better options here since you mentioned Python programming might be difficult. If the tool is slow, you could try to process your data in smaller chunks instead of all at once. There is no guarantee that swapping to open source will be faster. If we had the original code, users might be able to point out where it is running inefficiently.