topic Re: OD Cost Matrix Different results between ArcMap and Python in ArcGIS Network Analyst Questions

OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 13:07:20 GMT

Hi,

I am working on automating OD cost matrix runs and am trying to figure out why running the data in arcpy gives a significantly different result then when run in arcmap. I have checked every setting (i believe) and they all seem to be the same. It uses the same network and the same origins and destinations.

At first i thought it might have been a projection issue and i have tested the runs with everything in the same projection and still the results are different.

Does anyone have any idea on why this would be?

Thanks,

Matt

Re: OD Cost Matrix Different results between ArcMap and Python

FreddieGibson — Thu, 24 Sep 2015 13:36:48 GMT

Would you be able to upload a small sample of your data, a map document with your solved od cost matrix layer, and script that I can use to take a closer look at this?

Re: OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 13:38:40 GMT

I am sorry but I can't for this data. The only other variable i can think if is the data is being run is a query layer.

Re: OD Cost Matrix Different results between ArcMap and Python

FreddieGibson — Thu, 24 Sep 2015 13:53:11 GMT

I would assume that there is something different between your solver in ArcMap and your solver in your standalone python application. I would suggest running your standalone code in the python window of a new blank session of ArcMap so that you can verify the data in the same environment.

Re: OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 14:06:53 GMT

I have never done python in the python window. Would what you are suggesting just be a copy paste and run?

Thanks,
Matt

Re: OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 15:00:17 GMT

HI Freddie,

I did as you suggested and ran the script standalone, in the python window and in arcmap and it seems it may not be the solver. ArcMap loads and finds a few more destinations then when the loading happens in python.

Thanks,

Matt

Re: OD Cost Matrix Different results between ArcMap and Python

MelindaMorang — Thu, 24 Sep 2015 15:05:54 GMT

I'm guessing that the problem is most likely one of the following:

- In your python code, when you run arcpy.na.MakeODCostMatrixLayer(), you aren't feeding it exactly the same parameters as the layer you have in ArcMap (where many of the parameters are set to defaults). So maybe you have a different impedance attribute, or some restrictions are toggled differently.

- When you add your origins and destinations, maybe the field mapping isn't working the same as it is in ArcMap. In ArcMap, it's super easy to use the field mapping control on the Load Locations dialog. In python, you need to use either a correctly formatted text string or an arcpy.na.NAClassFieldMappings() object.

I know you can't upload your data, but can you upload a snippet of your python code? Otherwise, it's virtually impossible to diagnose this problem.

Re: OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 15:16:25 GMT

Hi Melinda,

here is the make OD cost matrix code -

/****************************/

#Sets the location of the network dataset and the name of the OD Cost Matrix

inNetworkDataset = "Drive:\NetworkAnalysisTesting\Network_w_Time_NAD83.gdb\Updated_LenMin_ND\Streets_Updated_LenMin_ND_ND"

outNALayerName = "OD_Cost_Matrix"

impedanceAttribute = "Length_Miles_Recalc"

searchTolerance = "5000 Meters"

odOrigins = "Origins"

#Create a new OD cost matrix analysis layer. For this scenario, the default

#value for all the remaining parameters satisfies the analysis requirements

print "I am about to make an OD Layer"

outNALayer = arcpy.na.MakeODCostMatrixLayer(inNetworkDataset, outNALayerName,

impedanceAttribute,"",3,"Length_Miles_Recalc;Minutes_recalc","ALLOW_UTURNS","Oneway")

here is the load locations code

/*******************************************/

#Load origins as destinations

print "I am about to load the destinations - " + inDestinationName

arcpy.na.AddLocations(outNALayer,destinationsLayerName,inDestinationFeatures,"Name UniqueRecordID #", searchTolerance,"UniqueRecordID","Streets_Updated_LenMin SHAPE;Streets_Updated_LenMin_ND_ND_Junctions NONE","MATCH_TO_CLOSEST","CLEAR","NO_SNAP","#","INCLUDE")

print "Load Destinations complete"

#Load origins

print "I am about to load the Origins - " + inOriginName

arcpy.na.AddLocations(outNALayer,originsLayerName,inOriginFeatures,"Name UniqueID #",searchTolerance,"UniqueID","Streets_Updated_LenMin SHAPE;Streets_Updated_LenMin_ND_ND_Junctions NONE","MATCH_TO_CLOSEST","CLEAR","NO_SNAP","#","INCLUDE")

print "Load Origins complete"

Thanks,

Matt

Re: OD Cost Matrix Different results between ArcMap and Python

MelindaMorang — Thu, 24 Sep 2015 15:40:58 GMT

Hmm, well, none of that looks problematic.

Can you describe the differences you're seeing between the python results and ArcMap? Is one showing considerably longer distances than the other, or are the results just different all over the place? Are the results in ArcMap identical each time you run Solve in ArcMap, and are the results in python identical every time you run Solve in python?

Re: OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 15:51:53 GMT

Melinda,

At the end of the process i split the name at the dash and join up other data based on the ID. Then I take the first destination and find all records that were within 5, 15 miles and 12 and 35 minutes and get a count. These counts are different between python and ArcMap. If i input the same origins and destinations i would expect that the distances returned would be the same.

Yes i get the same answer with different runs in python using the same data. I haven't checked this in ArcMap but based on past runs i believe i do.

Also not sure if you caught that i wrote when i load the data in python there are some records that are not located on the network but when i load them in arcmap they are located. This leads me to lean towards there being something with the add locations but i am not sure.

Thanks,

Matt

Re: OD Cost Matrix Different results between ArcMap and Python

MelindaMorang — Thu, 24 Sep 2015 16:29:27 GMT

What you just said in your last paragraph above sounds very suspicious to me, so I'm inclined to agree that it sounds like an Add Locations problem.

Maybe it's a search_criteria syntax problem.

Your input: "Streets_Updated_LenMin SHAPE;Streets_Updated_LenMin_ND_ND_Junctions NONE"

I would have thought that syntax would be okay (because often the semi-colon-separated strings are interpreted the same way as lists in arcpy), although the doc tells you to enter it in as a list of lists.

Doc: [["Streets_Updated_LenMin","SHAPE"], ["Streets_Updated_LenMin_ND_ND_Junctions","NONE"]]

Maybe try that?

Also, are you sure that "Streets_Updated_LenMin_ND_ND_Junctions" is actually the name of your junction source, or is that a typo?

Finally, when using search_criteria in Add Locations, you have to put something for all your network dataset sources. It's unclear what the behavior is if you have, for instance, multiple edge sources, but you only specify the behavior of one of them. So, are all your network dataset source feature classes accounted for in the search_criteria above? If not, put something for all of them, and see if that helps.

Re: OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 17:07:58 GMT

Melinda,

I will look into the syntax but that is the syntax that gets exported from model builder. If I am doing something with many options i will sometimes create a sample in modelbuilder to see what the syntax should be.

I think I may have found the issue and I think it has to do with the query layers as my inputs. I exported my data as file geodatabase feature classes and ran them through both arcmap and my python code and I got the same result. I want to try and perform this on my full datasets to confirm my results.

If this really is the case that query layers are causing the problem then this is going to add a bunch of processing time to my project.

Any thoughts on why query layers would cause a problem. They visually look good and perform pretty well in arcmap.

Thank you so much for looking into this with me.

Thanks,
Matt

Re: OD Cost Matrix Different results between ArcMap and Python

MelindaMorang — Thu, 24 Sep 2015 17:10:45 GMT

Not sure what tool/syntax you're using for your layer query. Can you post another python code snippet?

Re: OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 17:18:15 GMT

This is pretty simple tool to use so i don't see where there could be an issue but here it is. Also it fails if i have something wrong here like i have the database name wrong.

inOriginName = rec

inSQLQueryRec = "Select * from PCPToRecipRatio.dbo." + rec

print inSQLQueryRec

inOriginFeatures = arcpy.MakeQueryLayer_management(inOriginRecipientsFilePath,rec,inSQLQueryRec,"UniqueID","POINT")

inDestinationName = prov

inSQLQueryProv = "Select * from PCPToRecipRatio.dbo." + prov

print inSQLQueryProv

inDestinationFeatures = arcpy.MakeQueryLayer_management(inDestinationProvidersFilePath,prov,inSQLQueryProv,"UniqueRecordID","POINT")

Thanks,

Matt

Re: OD Cost Matrix Different results between ArcMap and Python

MelindaMorang — Thu, 24 Sep 2015 18:04:26 GMT

This might still be a syntax problem.

Technically, an arcpy geoprocessing tool call produces a "Result object". Sometimes you can pass that result object in as input for another tool, and the other tool is able to figure out what to do with it. However, a more correct thing to do would be to retrieve the tool's actual output and pass that output as input to another tool.

Your code:

inDestinationName = prov

inSQLQueryProv = "Select * from PCPToRecipRatio.dbo." + prov

print inSQLQueryProv

inDestinationFeatures = arcpy.MakeQueryLayer_management(inDestinationProvidersFilePath,prov,inSQLQueryProv,"UniqueRecordID","POINT")

Suggestion:

inDestinationFeatures = arcpy.MakeQueryLayer_management(blahblahblah).getOutput(0)

Other suggestion:

Instead of using the output layer object from MakeQueryLayer, try just passing the layer's string name. This also tends to work.

arcpy.na.AddLocations(outNALayer,destinationsLayerName,prov,...)

Re: OD Cost Matrix Different results between ArcMap and Python

MatthewCieri — Thu, 24 Sep 2015 18:13:08 GMT

i will try the getOutput(0) idea but the string name doesn't work. The passing of string name is what lead me down the query layer path when i first started this. It kept say layer was invalid (or something along those lines).

Thanks again for all your help and thoughts.

Some of these things seem so simple to implement but after running you notice issues popping up even though everything was "successful"

Re: OD Cost Matrix Different results between ArcMap and Python

MelindaMorang — Thu, 24 Sep 2015 18:19:43 GMT

Happy to try to help. I've never used the MakeQueryLayer tool before, so everything I said was just a guess. If that doesn't work for you, you're best bet is to contact Support and to get one of them working on it. I think I've about reached the limit of what I can advise on GeoNet.