Data Loading in GeoDataBase with Geom Network *ON*?

[SathiaG]

Hi.

We are working on an Electrical Utility Project (ArcFM Configured DB). In production, we are using Complex Data Loader from ArcMap to load the new data from survey in a particular version with the GeoMetricNetwork ON.

Problems arise with the time consumption & man hours spent for the activity. It take lotsa time & its merely a Semi-Manual activity. The DB needs to be converted to ESRI objects from ArcFM objects to avoid the autoupdaters from firing, which could save us sometime. But still, the downtime is not enough sometimes to load thousands of records in 50+ feature classes.

Is there any other way or utility to automate the loading of the data with the Geom Network ON?..

S/W config as follows:
ArcSDE 9.3 - SP1, SP2
Oracle 10g R2 10.2.0.1
ArcGIS 9.3.1 - SP1, SP2
ArcFM 9.3.1.5725

Cheerz !
-Sathia

[VinceAngelo]

Oracle 10.2.0.1 (Enterprise/Standard) is not supported with any release of ArcSDE -- It is
known to have both performance and reliability issues.  Oracle 10g XE (Express) is supported
(only available as 10.2.0.1), but not for production use.

Try patching to 10.2.0.3 (with the most recent CPU patchset), and see if this doesn't help.
If using Direct Connect, and your server is 10.2.0.3 or 10.2.0.4, then try patching the Clients
to match.

- V

[SathiaG]

Hi V

Yes, we use direct connection everywhere. We tried with 10.2.0.3 once in the staging server, it gave the same kinda performace. So the updation/patching in the Prod server is postponed since we are planning for a 11g upgrade sometime soon. The production is still in 10.2.0.1.

Is there any other way to load the data?

[CraigGillgrass]

The object loader in ArcMap is the recommended method of loading data into a geometric network after it has been created.  Can you clarify how many features you loading and how many feature classes are in the geometric network?

[SathiaG]

Hi Craig,

Sorry for the late reply !

We have 53 feature Classes and 32 of them in the Geom Network.

[CraigGillgrass]

While there is no limitation to the number of feature classes you can have in a geometric network, every feature class you have in the network comes with a cost.

Each time a feature is created, the geometric network has to search through each class and determine if there are any coincident features.  So, the more feature classes you have in the network, the more time it takes to search for any coincident features.  The object loader is essentially a batch loader that automates the loading of features; but it still has to create them one at a time.

I'd suggest that you look at ways of decreasing the number of feature classes in your network by combining several feature classes into 1 and using subtypes.  You can also look at eliminating any feature classes that aren't being used at this time, if you've added empty feature classes to the network thinking you'll use them at some point in the future, remove them and only add them when needed.

you best option is probably to delete the geometric network, get all your data loaded and then recreate it.  It will no doubt be much faster.

You never responded with how many features you are loading into the network. Can you respond with that?

[SathiaG]

Thanks for the prompt explanation.

We already decreased the number of feature classes in the Geom Network. 32 is the minimum we could have. Merging them would only increase the load on a single feature class.

Deleting the Geom Network will only increase the time much more cos the junctions need to be configured for ArcFM. Running ElectricTraceWeights for all the junctions all over again & tracing them will take much time than loading the data comparitively.

The total number of features need to be loaded on an average of 70k to 90k in a window of 3 hours.

[CraigGillgrass]

32 is a lot of feature classes to have a in a geometric network, it's probably the most number of classes that I've ever seen.  Here's why I'm raising this as an potential issue; as I mentioned, each time you create a new feature, the geodatabase opens a cursor on each class in the network.  If we can handle 5 cursors in 1 second (as an example); you would be looking at over 6 seconds to determine whether there are any features at that location.  This is a cost you are paying every time you create a new feature.  So, I would strongly suggest that you try and re-assess the number of feature classes in your network. 

If you are adamant that you cannot either decrease the number of feature classes in the network or drop the network in order to load the features; then consider using a Map Cache.  If you're loading data into discrete areas of you network, with a reasonable number of existing features (reasonable being a couple thousand) you could try to build a Map Cache over the area you're loading the features into.  This could speed up a performance a bit, but you will need to prototype this.

[SathiaG]

Hi Craig

After much consideration & w.ref.to your recommendation, we have decreased the number of feature classes in the Geom Network to 21.

Thanks for the idea about the map cache. We will try it once after carefully analysing it. Hopefully Performance wont be an issue here after. 🙂

We tried a dumb idea at our end. Data loading through SQL queries. I mean, we put our dataset in a dummy instance and extracted insert queries from it. And then ran the queries in our DB. But that did not work well. The features got inserted but those features did not participate/ come inside in the geom network (absense of junctions in bw).

Is complex data loader the only savior here?