Using Free North American Detailed Streets for Network Analysis

[derikandreoli]

I'm not new to ArcGIS, but I am new to ArcGIS 10.

I've done service area analysis in the past, but this was done on a University computer maintained by IT.

I downloaded and installed the free trial software for both ArgGIS 10 and Network Analyst, and I downloaded the free "North American Detailed Streets for Network Analysis" layer package from here: http://www.arcgis.com/home/item.html?id=f38b87cc295541fb88513d1ed7cec9fd

I added the streets to a basemap, activated the network analyst extension, and added the network analyst toolbar. But all options are 'grayed out' on the 'network analyst' dropdown window.

I suspect that I need to build a network from the streets layer package.

Has anyone else gone down this path?

Any guidance is greatly appreciated.

-Derik

[derikandreoli]

Bueller... Bueller

[AlexeyTereshenkov]

Derik,

The data you've downloaded is a layer package (.lpk) file which can be unpacked. To unpack a layer package, either drag the .lpk file into ArcMap or right-click on the .lpk file and select Unpack. Alternatively, you can use the Extract Package tool and specify an output folder.

Unpack will always extract the package into your user profile under:

    XP - C:\Documents and Settings\%username%\My Documents\ArcGIS\Packages
    Vista and Windows 7 - C:\Users\%username%\Documents\ArcGIS\Packages

The unpacked data for this layer package is an SDC dataset which represents the streets for the North America (basically you can think of this SDC dataset as a geodatabase feature class stored in a special non-geodatabase compressed format). This implies that you would have to create a new ArcGIS network dataset based on the streets data. That is why all the Network Analyst buttons are greyed out - there is no valid network dataset layer added to the map document.

However, a network dataset cannot be created directly on the .sdc file. Thus, you'd need to export the SDC dataset to a shapefile or a feature class. Shapefile would not be a good alternative if you want to export the whole North America (shapefile size is limited to 2GB). You could try to export just a selection of some streets over a small town to see how it works for you. I've tried to export the whole dataset on my laptop (i.e., not the best server machine) and after some hours the size of the output geodatabase feature class was over 6GB.

The streets dataset contains fields which are necessary for creating a proper network dataset: oneway restrictions, administrative boundary field (state and county), elevation field (F_ZLEV and T_ZLEV), hierarchy, speed limits, shield and highway number (might be relevant for generating directions) and some others. There is no data regarding turn restrictions (since it is just a streets layer) and no signposts (which are often good when generating directions).

To conclude, the dataset is really good for those who want to get started with the Network Analyst and want to get hands at a topologically and attribute-wise correct streets dataset. Yet I wouldn't use it for production routing - for those purposes I'd suggest to take a look at Esri StreetMap Premium product (http://www.esri.com/data/streetmap/index.html) or Street Data Processing tools (http://resources.arcgis.com/gallery/file/geoprocessing/details?entryID=7DD58DB5-1422-2418-A0EC-0929C...) if you have access to the raw TomTom Multinet or Navteq Navstreets data.

Good luck!

[derikandreoli]

Alexy,

Thanks for the thorough response.

Through trial and error, it looks like the extract and build technique works, but is extremely limiting regarding size of extracted dataset.

I am working on a high spec desktop, and it keeps running out of memory even when I attempt to build a network out of Washington (as in the state, not the capitol city) after exporting the selection to shapefile format, then importing it as a feature class to a new feature dataset in a personal geodatabase.

I wonder if there is a better way of building the network? One which is not so memory intensive?

-Derik

[AlexeyTereshenkov]

Derik,

Do you experience memory lack during the export of SDC dataset to a shapefile? In this case, I'd suggest exporting to a feature class in a geodatabase directly. Shapefiles are not the best storage format performance-wise 🙂

I guess you might get slightly better performance and control over the export process if you will use the Feature class to Feature class tool (from the Conversion toolbox) rather than selecting features in the Data View in ArcMap and then exporting. Thus, you could save the resources because you do not need to draw features on the map. In the Feature class to Feature class tool dialog box, you might use the STATE field (I believe it's been there) in the Field Expression field to export only those streets that have this attribute value (Washington).

Please do not use personal geodatabase for storing your data! They have 2GB per database size limit and often go extremely slow after 500MB. Use file geodatabase instead.

Let the software do the export over the night, for instance, since it can take some hours for sure and you do not want to sit and wait for the software to finish. You might see that ArcCatalog or ArcMap can go to the "Non Responding" state during the heavy load (like exporting data or building a network). Don't worry about it - let it do its job. A good way to check that the software has not been hanged is to use the Task Manager to see if the memory use for the application is being changed. If it does, then you are good. If it does not, wait for some time just to make sure it does not indeed and then restart the software and try again.

Maybe you could talk to your system administrator to see if there are any better machines at your disposal? Do you have x64 bits operating system? I do and should say it makes a huge difference when processing data (not just in ArcGIS). Remember that WinXP x86 has 4GB RAM limit (128GB for x64); Windows 7 x86 has 4GB limit, too.

If nothing helps, I thought maybe you could try to do the following:
Build a network dataset in a feature dataset in a file geodatabase just for a minor part of the Washington state. Make sure it did compile and things look good. Then edit the source streets feature class by loading the streets left (you can use Simple Object Loader in ArcCatalog or Load Objects in ArcMap. Then rebuild the network dataset. Since it is just about "re-attaching" the edges, the re-building will go quicker and may not exhaust your resources as it did previously.

Let me know how it goes!