topic Re: LocalRouteTask geodatabase format, documentation/how? in Qt Maps SDK Questions

LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Mon, 14 Mar 2016 12:16:39 GMT

I am trying to set up offline streeet routing in ArcGisRuntime for Qt, Using LocalRouteTask.

However, I cannot find adequate documentation for how to obtain/configure an appropriate database.

The info I've found, just refers to two string parameters, with no further details. The main things I've been able to guess, is from examining/reverse engineering the demo "RuntimeSanFrancisco.geodatabase" files.

What I've learned so far:

- There are 4 "geodatabase" formats. (One for each format name would have been sufficient for me.)

Until that page is moved, a link here: https://developers.arcgis.com/qt/cpp/guide/supported-geodatabase-formats.htm

describing the 4 formats.
- the 'RuntimeSanFrancisco' is format 1 - .geodatabase-SQLite.

Using QGis, I can see the SanFrancisco offline-routing example contains

ALL the following data - I don't know how much of it is required for a street routing example.

I notice the N_1_* tables, and the 3 Streets* tables (relevant because the named network is called Streets_ND)

and the ND_27* tables, and Streets_ND_Junctions.

(however, I note that the geometry-type is different compared to the next picture I show.)

We had an ESRI partner prepare a 'geodatabase' data set for our client for offline routing, and they came up with

a ".gdb" folder, which I, based on the link above, guess to be a socalled "file geodatabase" type (it's called 'file', because it's stored in a directory, as opposed to the SQLite variant, which is stored in a file. In case you wonder.)

When I open that 'file geodatabase' in QGis, I get this:

Again we see names 'Streets', 'Streets_ND_Junctions', and 3 ND_XXX* tables.

However, trying to load this in our ArcGis Runtime application, just leads to crashes giving us no relevant info.

(executing the LocalRouteTask constructor gives a crash.)

The only documentation I find is this:

https developers.arcgis.com/qt/cpp/guide/find-a-route.htm

Find a route—ArcGIS Runtime SDK for Qt | ArcGIS for Developers

from which I don't seem to find any info about how to obtain/build routing geodatabases?

QUESTION: Anybody got any hints for how to figure this out / how one is supposed to figure this out?

The ESRI partners who built the 'geodatabase' are nice people, but it appears to me, they have access to the same lack of documentation as I find..? How does one figure out how to build a usable routing-network data set for ArcGisRuntime LocalRouteTasks, and is it possible to diagnose it with other things than mute crashes?

I spent some time googling this, but only found people asking similar questions in threads around the web, that never attracted any replies 😞

Detail question: I would like to know, if LocalRouteTask only works with the SQLite variant? From the linked 'supported' page, it suggests that "geodatabase file" would work too, but given the very little doc-details on LocalRouteTask itself, and my crashes, I'm wondering..? IE SQLite is the thing I've seen it work with..

Thank you for any help or advice you can offer on this.

URLs:

http developers.arcgis.com/qt/cpp/guide/supported-geodatabase-formats.htm

Re: LocalRouteTask geodatabase format, documentation/how?

LucasDanzinger — Mon, 14 Mar 2016 16:00:02 GMT

You need to create a sqlite based geodatabase with the file extension ".geodatabase". We refer to these as mobile geodatabases. You author these in ArcMap using the Create Runtime Content tool. Here is the help for the tool - Creating ArcGIS Runtime content—Help | ArcGIS for Desktop..

Your ArcGIS Desktop users should be able to add in the transportation network dataset into ArcMap, and use this tool to export it to a mobile geodatabase that contains the transportation network.

I will make sure to add a note that we need to better doc this.

-Luke

Re: LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Tue, 15 Mar 2016 11:54:39 GMT

Thank you for the useful answer - my ESRI partner and I have tried to follow this, and it certainly brings us closer.

However, I still get a crash when we try to load the .geodatabase.

I am unsure, if the .geodatabase is enough on its own, or if an accompanying folder is necessary, which was there in the ESRI test data: "RuntimeSanFrancisco.tn".

UPDATE: it would seem 'xxx.tn' subfolder may be required.. at least, if I hide that folder, in the ESRI demo example, I get a similar(?) crash when trying to load the geodatabase..?

[UPDATE2: the .tn folder was indeed critical, see next reply]

I went through the check-list on the webpage you've linked above (see below next screenshot.)

I include a screenshot of the contents of the built geodatabase, according to QGis:

It seems to include Streets, Streets_ND_Junctions, and RestrictedTurns.
I assume, but am not sure, that layers 20-21-22 would be the coordinate-system description.

The checklist, with my uninformed comments:

[[JG: ]] I was going through the info from http://desktop.arcgis.com/en/arcmap/10.3/map/working-with-arcmap/creating-arcgis-runtime-content.htm

to see if we are violating anything obvious. I've listed the relevant sections here below, with comments:

"(quoting) Network Data -"

"If your map contains a network dataset or a network analysis layer that references a network dataset, you can include the network dataset in your ArcGIS Runtime content as a transportation network for routing and driving directions. When including a network, all feature layers the network references become part of the ArcGIS Runtime content. This may include layers that are not part of the map. For example, if you only have a network layer in the map, your ArcGIS Runtime content includes the streets, junctions, and turns feature classes associated with the network.

For a network dataset to become a transportation network, there are a few restrictions to keep in mind:

The network dataset must be part of a geodatabase [JG: I assume this part is OK?] . This excludes networks that are in SDC or shapefile format.
The network dataset must be from ArcGIS 10.0 or later. [JG: I assume OK.. You can judge this.] If your network is from an earlier version, you can upgrade your geodatabase and your network.
The network dataset cannot have an unknown coordinate system. [JG: I can see layer 20,21 and 22 mention something like this, but don't know if that is the required part.]
The network dataset cannot use any VB or Python script evaluators. [JG: shouldn't apply here.]
The network dataset cannot use any custom COM evaluators. [JG: shouldn't apply here.]
A network dataset that uses live traffic [JG: shouldn't apply here.] will be ported to a transportation network, but the live traffic components are excluded since they are not supported.

style="margin-bottom:20.55pt;background:#FEFEFE"

Do not include network data—Exclude network datasets from the ArcGIS Runtime content.

[JG: I assume you chose the first of these two options?]

In short, it doesn't appear obvious to me, that we are violating any items on this list..

I'll return with more info, if we figure more out.

Re: LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Tue, 15 Mar 2016 12:46:05 GMT

(FYI - this is not a request for help at the moment, it is just a status update on situation so far.)

Hi Again. As mentioned, the '.tn' folder was indeed critical - my ESRI partner says they 'tried something different', and sent me a new data set, which now included a ".tn" sub-folder.

I am trying to get them to describe what 'something different' was..

[ UPDATE:

"Before analyse I have to select the network in the left pane and the analyze and share...."

]

Note: this only means I'm not crashing anymore, when trying to load data - I have no idea of behaviour/quality yet.

Main note for this is: Building the data set, they got the error

"...Actually I get and warning with the network now, saying that the network dataset has not been dissolved. .."

We are looking into what that means..

I will update here with info on how/if we got it working.

Re: LocalRouteTask geodatabase format, documentation/how?

LucasDanzinger — Tue, 15 Mar 2016 17:34:44 GMT

Sounds like you are on the right path. The "dissolved" warning is just a warning in ArcMap, and should not be related to this. You will need the geodatabase and .tn folder. They are both required, though you just point to the .geodatabase in your local route task. My suggestion would be to take the local routing sample, switch out the network dataset and basemap, and try and see if you can use it with your data.

Re: LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Fri, 08 Apr 2016 10:06:09 GMT

Update: Since march 15th, we have locally had a lot of back-and-forth on this issue (between me and my 'esri partner'), trying to get it to work. The good news is that it 'sort of works'. The bad news is the 'sort of' part..

We experience that 'benign'/obvious connections unexpectedly don't work. E.g. a major road intersection, that in reality allows traffic in all directions, is impossible to pass from north to south.
The routing error we get is is code -1, "an unexpected error occured".
What makes this so strange, is that the ESRI partner has no problems whatsoever, doing this "same" routing task in her source data (ie from/to same lat/lon as I get errors with.)
Note that the ESRI partner is not running off the SQLLite-geodatabase - she's running on the source data set, which she uses to build the sqllite-network.
We've so far spent fruitless weeks trying to resolve this, having an impatient client that was supposed to do production on our result, if it would work.
I'll try to include (some of) my data case here. I'm considering posting this issue as a new post instead, and closing this existing query off as resolved (whose subject was the proper format of LocalRouteTask data.)
However the queries are somewhat related, since it's the same goal - we were/are attempting the 'nobrainer of just setting up a local routing network :-)'..

Here the details:

(edit: I HAVE ATTACHED the sqllite Geodatabase that fails/on which we can't route between the following coordinates.)
I'm trying to route from -21,949970 / 64,079085 to -21,950415 with 64,078141

(supposedly WGS84 / decimal degrees lat lon.)
On this screenshot, that is from point 24 to point 25.
And the problem is, it won't move south across the west-east-going road.
As the basemap (and reality) suggests, this should very much be possible.

The esri partner says she is using "I am using the network dataset in Arc GIS desktop, and the network extension there".
And she can trace across here, just fine.

She sends me this screenshot, of how the street-graph actually looks:

(ignore the green (2) circle on the right. The trace goes to the leftmost southern green dot.)

What is visible from this screenshot is, that the intersection is a bit weird (the triangle),

but I think I've seen that structure arise in other similar cases, with roads that split/merge.)

I just received a mail from the esri partner, with some technical notes on how they build the data set.

I qoute them here:

We have street centerlines as a feature class in ArcGIS File Geodatabase. The feature class has attributes that describes things like speed limit, road class and route weight.

There is also another feature class that contains restricted turns.

Then we used ArcGIS 10.4 Network Analyst to create a Network dataset from these feature classes, through wizards and with directions from Esri.

The network dataset runs fine in our ArcGIS software (ArcMap) and routes everything correctly. It is a rather simple dataset based on few rules as we lack good information about lot of places but that should not cause errors in the routing.

After that we used ArcMap to create a run time geodatabase with the Network and sent it to you.

If you deem necessary, we can try to provide you with the 'working' source data too.

Thank you in advance for any light you can shed on this issue.

Re: LocalRouteTask geodatabase format, documentation/how?

LucasDanzinger — Fri, 08 Apr 2016 17:25:42 GMT

Hi Jakob,

I think you have a couple of potential issues here. First is that when I look at the streets dataset and where your coordinates are, I think the streets dataset is clipped to not include some of the streets you need to solve. I tested this by adding the streets layer from your geodatabase into the map, and I can see that the coordinates are not over a street.

My best guess is that the data is getting unintentionally clipped when created in ArcMap. The data will automatically clipped by the extent of the data frame. Sometimes, the "Analyze" window that pops up further shrinks the data frame and causes this issue, so perhaps your analyst can undock the window to make sure they are getting the correct extent to you.

The other potential issue is mixing spatial references. I don't know what spatial reference your map is in, but it may be good to try and have a basemap in 3057, since your network dataset is in this spatial reference. I also recommend projecting your input points to 3057.

The above gif shows the original points outside of the street extent. However, as I click inside of the street network, I am able to solve successfully.

I attached my code for you to try.

Hope this helps!

Luke

Re: LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Mon, 11 Apr 2016 10:32:52 GMT

Thank you yet again!

With your input, the esri partner was able to produce an updated (and much bigger) geodatabase.

Unfortunately, now it behaves very weirdly. It does not deliver any actual routes,

but now claims to solve all queries successfully?!

They (the source) have tried to tweak various settings, but all attempts so far give same behaviour.

Strange, that this time it doesn't give the '-1 unexpected error' when attempting to trace, but instead claims success.

The source claims they are building with "same" settings as they built the 'original error set'

(that DOES trace, but has problems.)

All 'part' traces respond like this:

Route km:0; geom:1/2/1, ie 1 path, 2 points, 1 seg. (apparently no curve is displayed.)

I believe all we do happens in wkid 3057 - both basemap and network-graph.

I have attached the new network here.

We are continuing to look into what could be wrong this time, but I'd appreciate and love it, if you can offer any insight into why this dataset wouldn't work..

Cheers, Jakob.

Re: LocalRouteTask geodatabase format, documentation/how?

LucasDanzinger — Mon, 11 Apr 2016 16:47:54 GMT

It works for me with your new geodatabase and the code I posted earlier. Did you try that code? However, the network dataset still doesn't cover the area for the coordinates you provided. Below you can see that the original points you gave are not within the network dataset extent. However, using the code I provided, I can click on the map (near roads) and get routes back.

Re: LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Tue, 12 Apr 2016 07:44:23 GMT

Once again thank you for your very helpful answer!

Once I saw the clipping in your screenshot, I re-tried a trace in new data set, further north.

And it worked! So it appears to me, the clipping issue is now my/our primary remaining issue.

Regarding your QML code, thank you for that. I've now read it. Tracing-wise, it appears I do the same as you.

However, the insight/ephipany you are giving me here, is that your QML code

is also able to DISPLAY how the street-network from the Sqlite geodatabase looks (correct me if I'm wrong..)

Assuming this is true, you've now given me 'the missing piece', solving my main problem of trying to "diagnose something invisible".
I'll assume/hope GeodatabaseFeatureTable+FeatureLayer+Geodatabase can be combined in a similar way in C++/Qt.

Otherwise, time for me to get better acquainted with QML..

I'll return to the ESRI partner/analyst, with your 'new clipped screenshot', and let them know it still gets clipped somehow.
Ironically, the area that is missing from the street network, is where they themselves are located 🙂

If we figure out what goes wrong with the routing-network, I'll put the answer back here.

Thank you so far.

Re: LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Tue, 12 Apr 2016 08:01:03 GMT

A question from the analyst/ESRI partner:
"Does it matter whether it is arcmap 10.3 or 10.4. ?" (that they use to build the geodatabase).

My own instinct says 'no', but given the situation, I thought it might be better to ask..?

Re: LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Tue, 12 Apr 2016 14:03:11 GMT

It works now :-).

The esri partner reports this about what caused it:

"...

When I run this Arcruntime version there is a popup window that opens below the dataframe and cuts the dataframe extent, from below

.."

So presumably that is the UI aspect to be aware of, when building/exporting such a network graph.

Again, Thank you for all your assistance in getting this to work!

Re: LocalRouteTask geodatabase format, documentation/how?

LucasDanzinger — Tue, 12 Apr 2016 16:54:00 GMT

You are welcome. Everything you did here in QML can be done with C++ as well.

-Luke

Re: LocalRouteTask geodatabase format, documentation/how?

JakobGaardsted — Sat, 11 Dec 2021 11:06:28 GMT

Just an update: I ported your QML example (for visualizing the network layer) in C++, and thought I'd add that snippet here,

to aid others working with similar stuff (in particular, it was something I wish I had worked out for myself..)

(excuse the weird formatting, I tried to repair it as best I could)

// First, find us a geodatabase:

QString gdbPath = "straeto_network.geodatabase";

ER::Geodatabase* geodatabase = new ER::Geodatabase(gdbPath);

// Then, make a featureTable:

ER::GeodatabaseFeatureTable* gdft = new ER::GeodatabaseFeatureTable;
gdft->setGeodatabaseWithLayerName(geodatabase, "Streets");

// Now make a layer out of that table:
ER::FeatureLayer* featureLayer = new ER::FeatureLayer(gdft);

// Hmm, we should probably set some symbol stuff?
// Finally, add that layer to the map.
m_map->addLayer(featureLayer);