Issue with Rebuilding Network Dataset

Narasimhapavanappalla

We working on big network dataset ( around 2GB) which is updated almost everyday and feeds our live Routing service. 

However, rebuilding of the network dataset takes a couple of hours ( ~ 3 hours) despite small updates.

As of now the Network dataset has these amount of features:

1. edges = 1 million
2. junctions = 45 millions
3. stair/elevator = 30 thousands

Is  this expected or we are missing something?

Any suggestions on how we can speed up the process would be appreciated.

Thank you!

Melinda Morang

I looked at the network you shared with me privately, and here's what I found:

On my machine, building the network as is takes 4 hours and 22 minutes. (You must have a faster machine!)  This is an "incremental rebuild" where it senses that it has dirty areas (places where it's been edited) and builds only the dirty areas.  Hypothetically, this should make the build quicker if there are only a few small areas in the network that have been changed.

If I instead force the network dataset to do a full rebuild (so it ignores dirty areas and builds the whole thing whether or not it's dirty), it actually takes less time.  For me it took about 2 hours and 5 minutes.

Unfortunately, I've seen this type of behavior before.  Sometimes it just takes forever to sift through a large number of small dirty areas and handle them individually, even longer than it takes to rebuild the network from scratch.  I will log a bug for our developers to investigate your particular use case to see if there's a way we can improve performance; however, this isn't something we can fix in the short term.  You should probably consider some kind of workaround.

One workaround is to make your automated process force a full rebuild.  This will still take a considerable time, but it does seem to be faster than your incremental rebuild speed.

Here's two ways to force a full rebuild:

Method 1:

1. Open the network properties and add a new restriction attribute.
2. Click OK to close the property page
3. Reopen the network properties and delete the restriction attribute.
4. Click OK to close the property page.  The network now thinks an attribute has been modified and that it requires a full rebuild.

Method 2:

1. Run the Create Template From Network Dataset tool to create an xml template with your network dataset's schema.
2. Delete the network dataset.
3. Run the Create Network Dataset From Template tool to recreate your network using the xml template file you just created.  The network is now completely fresh and requires a full rebuild.

 

If I've understood your original question correctly, though, it sounds like you're working in a production environment where edits are being made daily, and you want this build to run quickly.  I'm curious what edits are being made daily to your network.  When I looked at the network's dirty areas, I see a LOT of them.  Tons of little overlapping dirty areas all over the place.  It looked to me as if the entire network was effectively dirty, but probably all the small changes accumulated over time in a way that left a ton of these little overlapping dirty areas.   What kind of edits (automated or manual?) are being done to cause all these changes?  Perhaps you can consider streamlining this part of the process to avoid this large accumulation of edits.

Another thing you could consider: It looks like your network includes a bunch of small areas spread around the country that are not connected. If you are hosting a network analysis service, there's a way you can configure it to use separate network datasets and broker between them depending on the location of the user's analysis inputs.  So, if my inputs are in New York City, use the New York City network, but if they're in Los Angeles, use the Los Angeles network.  This way, edits could be kept contained to only a single network, and smaller networks build faster.  This is more complicated to set up than I can describe to you here on Esri Community, and if you want to go down this route, you should talk to Esri Support (or whoever your usual contact is - I believe I recognize the name of your company as a partner/start-up).

Hope this helps!