Save to Layer File on a OD Cost Matrix Layer takes forever

[DevinTaylor1]

I am trying to generate OD Cost Matrix in a model and generating the matrix for 5,000 origins and destinations takes about 15 minutes, but the next step is to Save to Layer File in order to persist the matrix, and is taking hours. The first two attempts I thought something had gone wrong and killed the machine and rebuilt the model after a couple of hours, the most recent attempt I have left it running and it is going on 9 hours for the Save to Layer File step.

How can saving the file take so much longer than generating the matrix?

Is there a better way to persist an OD Cost Matrix? Save to Layer was the only method I could find.

[JaySandhu]

What is the size of your Lines in the OD layer? That can give an indication to how long it may kae to write it out. But 9 hours does seem quite long. As you wait for the write to happen, is the layer file size growing? That is, is something being written out? The other choices you have to write things out is that you can save the Lines to a feature class (shape file or gdb) or you can also open the attribute table and save that (dbase or in a gdb, etc).

Jay Sandhu

[DevinTaylor1]

The Save To Layer is still running at 16 hours and the output file is 16k, so I'm going to guess there is some problem.

I am not sure of the Lines size, I have only tried to generate it within the Model. I will try the steps manually and take a look at it this evening. I will also explore the export to feature class. Is there an attribute table export tool I could use in Model Builder?

[JaySandhu]

You can use the Copy Rows tool in the Data Management Table toolbox to write out rows to a new table.
Jay Sandhu

[DevinTaylor1]

A final update on this.

The performance problems appear to be related to the set size and some limiting factor in arcgis or windows or somewhere in between. The solve would have been generating about 30M lines. I reduced the set so that about 500,000 lines are generated and it solves in a few minutes, so I must have run into some limit somewhere that dramatically slowed down processing or made it hang. The 500K line result was 36 MB so a 30M line result would have been close to 2 GB. The 30M process also never exceeded 750 MB of RAM usage, while the 500K process took over 200 MB, so there could be some other limit in software or the script engine that bogs it down.

[JaySandhu]

Its not clear from your last post if the slow down on the larger case (30 million) was during the solve or in writing out the results to disk. Solving the OD is quite fast but populating the OD lines feature class with 30 million rows can take some time! Generally if you are doing very large OD, then you should consider solving it via a standalone .net/arcobjects program to solve and not generate the OD lines but directly access the OD matrix object and write the distances to a disk file.

Jay Sandhu