Economizing OD Cost Matrix when A->B = B->A

[MelindaMorang1]

Hi!

I'm creating OD cost matrices of walking distance between transit stops in Chicago.  So, because my origins and destinations are the same files and because pedestrians don't care about one-way restrictions, the distance from Stop A to Stop B is always equal to the distance from Stop B to Stop A.

Is there a way to cut down the analysis time and output file size by having Network Analyst calculate the distance between A and B only once, instead of twice (A->B and B->A)?

Thanks!
~Melinda

PS. I have already cut down the run time and output file size by using cutoff distances.  Still, when your cutoff distance is 2 miles, it still takes forever and results in a very large output file.

[JaySandhu]

First of all there is no way to tell the OD compute one-half of the matrix, that is, just A to B.

Second, how many origins and destinations do you have for this solve? By any chance, does your network datset have hierarchy and it is set ON by default, then, if you are usinga 2 mile cutoff, set the hierarchy to OFF and the performance will increase. Note that hierarchy helps when you have long distance routes, but not when you have small cutoff's like 2 miles.

Jay Sandhu

[MelindaMorang1]

Thanks, Jay.

My hierarchy is set to "NO_HIERARCHY" since I don't really need it for anything.  Although, come to think of it, I was wondering if there was a way to use it to prevent pedestrians from walking on limited-access highways for network datasets that don't have anything like that in the restrictions.  But that's another topic...

I'm trying to calculate an OD matrix of bus stops in Chicago.  Since there are nearly 13,000 of them, it's obviously not practicable to calculate an OD matrix for the entire network.  However, even a matrix with a cutoff distance of 1.5 miles creates an output file on the order of several Gb because the network of stops is so dense.  I'd love to be able to cut that down.

[JaySandhu]

Few things to do: Are you running ArcGIS 10.0? If you have a machine with 64 bit OS then you can use ArcGIS 10.0 on that for it to use up to 4 GB RAM. As far as the 13,000 OD even with 2.5 mile cutoff can be quite large, how about solving this in smaller chunks? Divide the origins into four sets of 3250 each and then solve four 3250  by 13000 OD's and combine the results? If that is still too large, divide further!

Jay Sandhu

[MelindaMorang1]

Actually, what I'm doing is creating a custom tool to let people create walking/transit service areas using GTFS public transit data.  See http://www.transit.melindamorang.com/ for more details.  If I were just doing a single analysis myself, it would make sense to break up the data into chunks, but I don't think that would be practical for a tool used by a variety of people for a variety of datasets.  Currently the tool simply allows the user to enter a cutoff distance for the OD matrix to keep the file size down, but they have to make sure to use a cutoff distance greater than the maximum distance they expect their pedestrians to walk when they're calculating their service areas later.  It works fine, except for the output file size with a large transit network and a cutoff distance larger than, say, a mile.

Is ESRI working on incorporating a way to use GTFS data into Network Analyst?  To me, NA's public transit capabilities seem really lacking.  There's definitely room for improvement, and I'd love to see it be able to handle GTFS.  Because GTFS datasets are used in Google Maps trip planning, transit authorities actually keep them up to date and readily available, unlike transit-inclusive network datasets.  Plus, they actually have schedules, and the schedules make a huge difference as far as what areas are accessible and what aren't.

[JaySandhu]

I am sure that a tool you write can have limits put in so that end users cannot enter in more than, say, 1 mile cutoff. As far as I know from walking behaviour, most people will not walk even 1 mile! Also, I am trying to understand what are the origins/destinations that people will be solving on. That is, you mention 13,000. What are these locations? What is someone going to do with gigabytes of data comming from a web service?
Jay Sandhu

[MelindaMorang1]

To clarify: It's not a web service.  It's a downloadable custom ArcToolbox script tool created with python.

The ultimate goal of the tool is to be able to figure out the area reachable by walking and transit from a given point.  To do that, you need to know the network distance between that point and the transit stops you could walk to within the time limit.  However, you also need to know the network distance between the transit stops themselves.  Someone might, for instance, walk for one minute to get to a stop, get on the bus and ride for 10 minutes, get off the bus and walk for 5 minutes to another stop, and then get on a different bus and ride some more until the time runs out.  So, we also need to know the stop-stop network distance to calculate how long it takes to walk between stops.

We solve this problem by just calculating one big OD matrix for all the stops, within a cutoff distance.  We've separated this part of the tool so you only need to do it once for any city, and you can calculate many different service areas without having to run the long part over and over.  The OD matrix just sits in an SQL table that the code can refer to.