UPDM 2019 Edition

[#1]

This is the official release of the 2019 Edition of the Utility and Pipeline Data Model (UPDM).

 It is designed to support data management for the natural gas and hazardous liquids industries.  Supporting the diverse needs of these industries means supporting multiple implementation patterns.  Specifically a network topology with the Utility Network implementation pattern, and a linear referencing implementation pattern with the ArcGIS Pipeline Referencing solution..

NOTE: April 3,2020:  Updated the Asset Package to remove the folder "ap_workspace", and to remove some outdated schematic design templates.  With the ArcGIS Pro 2.5 release of Utility Network Package Tools, this is no longer needed, and is generated errors in some loading instances.

If you have questions, please post them to geonet, so everyone can see and share the information.

Thank you

Tom DeWitte

Esri Technical Lead – Natural Gas Industry

[NathanEasley]

Hi Tazura,

UPDM 2019 has the feature classes and tables to work with Pipeline Referencing.  Once you've downloaded the model, you would need to configure the LRS, networks, and events with Pipeline Referencing then load your data.  I would suggest looking at this help topic (Create the LRS, networks, and events—ArcGIS Pipeline Referencing for ArcGIS Pro User Help | Document... ) that provides a high level overview of the configuration and loading process for Pipeline Referencing.

Nathan

ArcGIS Pipeline Referencing team

[Siti_Rohana_TazuraBinti_Ali]

Thank you Nathan for your fast response. 

For configuring LRS, networks and Events with Pipeline Referencing , will that be using user's data or not? Or is that something we just create inside the updm geodatabase and can configure independently without existing data?

At the moment, user is wondering, what is the requirement that their data must have in order for them to eventually be able to load into updm model?

Appreciate your response.

Tazura 

[NathanEasley]

Hi Tazura,

I'd suggest taking a look at the LRS data model (LRS data model—ArcGIS Pipeline Referencing for ArcGIS Pro User Help | Documentation ) to understand what fields are expected to be populated to get your data into Pipeline Referencing.

UPDM already has the Pipeline Referencing feature classes/tables.  You would need to use the Create LRS/Network/Event from Existing Dataset geoprocessing tools to create the LRS and associate those feature classes/tables with it.  Once the configuration is complete, you could load your data into the networks/events using the Append Routes and Append Events geoprocessing tools.

Nathan

ArcGIS Pipeline Referencing team

[Siti_Rohana_TazuraBinti_Ali]

Hi Nathan, 

Thank you for your response.

The Create LRS/Network/Event from Existing Dataset geoprocessing tools are the tool that we used to associate existing data into LRS, network and Event. 

The Append Routes and Append Events geoprocessing tools are the tool to load our existing data into network line and Events. 

We were able to Create LRS, Network and Event dataset as required but currently pending on append Routes since the user does not have routes feature class. We have tried to creates routes using Creates route tool and then tried append routes with network line created from Create network tool but the tools seems couldn't map the RouteID, RouteName, fromDate and ToDate field (screenshot : 2020-05-04_1703 )

Nevertheless, I am still wondering how these all  dataset can be eventually load into UPDM model? May I know what is the steps/ tool that we can use? User is expecting to load their data into UPDM model for ArcGIS Pipeline referencing. 

Really appreciate your response on this. 

Thank you. 

Tazura

[NathanEasley]

Hi Tazura,

I would take a look at the topics I mentioned in my earlier responses (Create the LRS, networks, and events—ArcGIS Pipeline Referencing for ArcGIS Pro User Help | Document... and LRS data model—ArcGIS Pipeline Referencing for ArcGIS Pro User Help | Documentation ).  They cover the high level configuration and loading process.  Since you already have UPDM, the feature classes and tables necessary are already in place.  You simply need to associate them with an LRS using the "From Existing" geoprocessing tools.

In order to load data using Append Routes, you would need to have the route data ready to load.  If you don't, then you would need to create it in a separate feature class to be appended using the Append Routes tool.  For the screenshot you showed above, it appears as though the PipelineCreateRoutes2 layer doesn't have the field of type/length that match those required in the network_petron layer you're trying to append into.  You might want to verify their type and length matches.  If you're not able to get the tool to work as expected when they do match, I would suggest opening a support case with Esri Technical support to investigate the issue further.

Thanks,

Nathan

ArcGIS Pipeline Referencing team

[JoeHershman]

Tom.

Looking at Tees and Reducers, neither have a domain assigned to diameter.  Both do to the diameter2 field.  What is the reasoning here?

Thanks

-Joe (and Scott)

[TomDeWitte]

Hello Joe and Scott,

Thank you for identifying this inconsistency in the data model.  The PipelineJunction featureclass should have the coded value domain: Pipeline_Fitting_Diameter assigned to the "Diameter" field for the subtypes "Tee" and "Reducer".  I will add this to the UPDM 2020 change log.

Please continue posting any other suggestions to improve the data model

Thanks

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[JoeHershman]

Tom DeWitte  ..Thanks for the quick reply.  We thought was likely an oversight, but wanted to confirm we had not missed something

[RobRader1]

Nathan Easley‌ and Tom DeWitte‌ the data model configuration for APR within a UN coordinated model has a great set of information.

The data model documentation for Centerline using the PipelineLine is extremely straight forward and great to see included in Implementation Patterns document in the download. If this could be expanded on in the next model release, that would be great!!

For further clarification: 

- If we can register Utility Network feature classes, what about the structures feature classes added to the LRS configuration as well?

- In 2016, valves were standalone feature classes. Now the only valves in the data model appear to be in the PiplineDevice feature class which is part of the core Utility Network.  Is the intention from ESRI's modeling team that moving forward this feature class is also to be registered with LRS ?  (specifically running the gp tool : "Create LRS Event from Existing Dataset") 

Resources looked over prior to posting here, so that everyone else also has those links:

- There is documentation here (Manage Pipeline Referencing and a Utility Network together—ArcGIS Pro | Documentation ) for the editing patterns supported when using both and how a specific UN feature class can be updated with : Update Measures From LRS (Location Referencing)—ArcGIS Pro | Documentation 

- The registration tool in 10.8.1 with Pro 2.6 only works with lines:  Configure Utility Network Feature Class (Location Referencing)—ArcGIS Pro | Documentation 

- Data Model supplied documents:

1. Data Dictionary (p.g. 718 is APR geometry info) 

2. Implementation Patterns (again it has a note at the bottom for the combined scenario that is Great!)

3. Change Log

[MarkCederholm]

Are there any known issues with Pro 2.5.2 / Enterprise 10.7.1?  Although the containment associations are there, toggling "Display Content" doesn't work (everything is always visible), and device-to-junction connectivity associations aren't tracing when I update subnetworks.  I had no such problems with UPDM 2018 at 2.3 / 10.6.1.  I see UPDM 2020 is out, but I'm not ready yet to go to Pro 2.6 / Enterprise 10.8.1.  I found some nasty bugs in Pro 2.6.

[TomDeWitte]

Hi Rob,

Glad to hear you are getting value out of UPDM and the related documentation.  I talked to Nathan about your questions and here are our responses:

Question 1: If we can register Utility Network feature classes, what about the structure feature classes added to the LRS configuration as well?

Esri Response: You can treat the StructureJunction and StructureLine feature classes similar to the Device, Junction and Line Utility Network feature classes. The LRS measures can be populated and updated using the Update Measures from LRS GP tool.

Question 2:  In 2016, valves were standalone feature classes. Now the only valves in the data model appear to be in the PiplineDevice feature class which is part of the core Utility Network.  Is the intention from ESRI's modeling team that moving forward this feature class is also to be registered with LRS ?  (specifically running the gp tool : "Create LRS Event from Existing Dataset") 

Esri Response: The intention is to have all of the assets of the pipe system participate in the Utility Network and be able to maintain routes and measures on those same assets. At the current capabilities this means using the "Update Measures from LRS" GP tool to populate and update the valve's LRS information.

Tom DeWitte

Esri Technical Lead  - Natural Gas Industry

[TomDeWitte]

Hi Mark,

I have not run into the display and tracing issues you are describing with Pro 2.5.2/ Enterprise 10.7.1.  If you have not already, please contact Esri Tech Support to get assistance with resolving these issues.  Tech Support is our best mechanism for getting information about bugs and their associated use cases to the development team for resolution.

Thanks

Tom DeWitte

Esri Technical Lead - Natural Gas Industry 

[MikeMillerGIS]

Ensure you have your display filters set correctly.  The Show/Hide is enabling and disabling them.  

[MarkCederholm]

Drat.  Good call.  I had all that configured, and I must have reverted to an earlier version of the project.  Now I need to do it again.  I'll get with tech support about the tracing issue.

[MarkCederholm]

I finally spotted the problem with the tracing issue.  I wasn't populating the Lifecycle Status of the connecting junction, so it acted as a condition barrier.  It's working now.

[TomDeWitte]

Hi Mark,

Thank you for sharing what you learned about your data and its impact on the behavior of the tracing tools.  The information you share will help others who will run into similar issues.

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[JiaLiu]

Hi Tom, I heard UPDM2020 is out. Is it available to the public yet? If so, where can we get it? Thanks!

[XanderBakker]

Hi jialiu ,

As far as I understand you can now use the deployment add-in to deploy the "Gas and Pipeline Enterprise Data Management" solution that includes the UPDM 2020 Edition : Gas and Pipeline Enterprise Data Management | ArcGIS Solutions for Gas 

Date	Version	Description
August 2020	1.0	ArcGIS Pro 2.6, UPDM 2020, and Utility Network Release 4.

[TJ_Houle]

Hi Xander Bakker‌, that is true. However, the previous releases had a good deal of associated stand-alone documentation that was quite helpful in implementation discussions. I'm hoping there is a plan from Esri to provide something similar for UPDM 2020. Any update on that would be great!

[MarkCederholm]

It turns out that Pro 2.6.x has core issues with file GDBs, and probably you should stick to Pro 2.5.2 and avoid UPDM 2020 for now.

[MarissaMijal]

What issues did you run into?  I created a UPDM 2020 file gdb with UN and APR and was able to load data into the file gdb...Its the SQl Server version I am encountering issues with.  I used ArcGIS pro 2.6.1 for all of it.

[MarkCederholm]

According to Esri tech support, there's a widespread problem with arcpy and geoprocessing tools at 2.6.x when applied to file GDBs.  Specifically, crashes.  I've run into the problem in my UPDM migration scripts, and now I've encountered the problem in other scripts as well.

[MarissaMijal]

Thanks, I will be on the lookout for that!  I have not encountered issues when loading new data into an empty UPDM 2020 file gdb, using the Location Referencing tools and ArcPy.

[TJ_Houle]

Hi Mark- 

Would you be able to specify why you are concerned about the UPDM 2020? Is it because you've had issues with your loading scripts or a fundamental error in the model itself?

[MarkCederholm]

I take that back: I can load the solution into 2.5.2.  I was just assuming I wouldn't be able to do that.

[MarissaMijal]

Has anyone encountered issues with the Append Routes tool using UPDM 2020 database on SQL Server?  I have ArcGIS enterprise 10.8.1, and ArcPro 2.6.1.  My UPDM 2020 has both a Utility Network and LRS, and PipelineLine is configured as the centerline.  The AppendRoutes tool requires attribute mapping for EngineeringNetwork, and also loads the centerline geometry into PipelineLine.  In order for data to load into PipelineLine, you must have an AssetType.  AppendRoutes does not accept AssetType as an attribute.  ArcPro 2.6.1 has an excellent data validation system, so there is now no way I can load valid centerlines using the Append Routes tool....I have been working with Esri tech support on this error for several months now..

• Does anyone have a work around? 
• When will AppendRoutes be updated to correct this?

AppendRoutes works perfectly in a file geodatabase...

Thanks,

[JelmerAkkerman]

Hi Marissa,

yes I ran into the exact same issue. Please find a link to a question on this issue I put on the ArcGIS pipeline referencing on the ESRI Community forum where you can also find a workaround:

Append Routes geoprocessing tool failing in combined setup for UN & APR on an Enterprise GDB 

Kind regards, Jelmer.

[MarissaMijal]

Thank you Jelmer!  We will use the append tool and the workaround.  I appreciate your quick response

Next...When will Esri modify the Append Routes tool...?

[NathanEasley]

Hi Marissa,

Glad to hear you'll be able to use the workaround to get data loaded for your deployment of Pipeline Referencing and the Utility Network.  The Pipeline Referencing team plans to make quite a few enhancements to the Append Routes and Remove Overlapping Centerline geoprocessing tools to better support loading data into a combined APR-UN deployment in the Pro release that will be during the first half of next year.  We plan to enhance these two tools and also make some additional enhancements for APR editing tools and others throughout the software based on feedback from users and partners that have started to utilize the Pipeline Referencing and the Utility Network together.

Thanks,

Nathan

ArcGIS Pipeline Referencing team

[MarissaMijal]

Thanks Nathan,

It is difficult to recommend UPDM to our clients when the current Location Referencing tools are not working.  For the updated tools, can you please:

• test them on UPDM 2020 in both a SQL Server and an Oracle environment
• add more specific error messages (most of the time I get 99999 call tech support)
• If AppendRoutes is supposed to load the centerline geometry into PipelineLine, and we also load PipeSegment data into PipelineLine, clarify the logic on how we differentiate centerline geometry PipelineLine data from pipe segment PipelineLine data.  Currently, the centerline geometry PipelineLine data looks like it is pipe data incorrectly loaded... 

Thanks,

Marissa

[aaizMe]

Can I migrate geometric network data to UPDM(2019) without linear referencing?

I have two databases storing pipeline data. One is spatial and the other is non-spatial. The spatial has one geometric network and other simple feature classes. I need to perform migration from those databases to one centralized database, UPDM,2019. This is very simple data as it does not have any utility network or any linear referencing. I have seen the data template of UPDM. It has a utility network containing center-lines, calibration points etc. that are used for pipeline linear referencing.

My question is, can I transform my data to updm without linear referencing? Because, I have seen some feature classes in UPDM, they contain route_id.  

The data is huge, so if LRS is needed, how would I able to transform the data? 

What is best practice in such sceneries?

Can someone give me a guide line to implement this?

[Ainur_Rychshanova]

Hi Tom,

Could you please explain me that the purpose of splitting at each isolating device, pressure changing device, and system delineation in the documentation? Is it necessary only for the correct operation of the subsystem Utility Network, or it is also necessary for correct work with Pipeline Referencing?

Thank you in advance 

[TomDeWitte]

Hi Ainur,

Your understanding of the reasoning for the splitting the pipe segments at each isolating device, pressure changing device, and system delineation device is correct.  To support the correct and accurate creation of the subnetworks (system, pressure, isolation, cathodic protection) the pipe must start/end at the devices which delineate these subnetworks. 

Additionally, for accurate trace results for gas industry critical traces such as isolation traces, the pipe segments need to start/end at the emergency valves.  Having the trace stop at the isolation valves insures an accurate listing of the meters (and their customers) impacted by the closing of those valves.

Tom DeWitte

Technical Lead - Natural Gas, District Heating and Cooling Industries

Esri, Inc

[Ainur_Rychshanova]

Hi Tom,

 I just have some more questions about UPDM. Could you please explain them to me?

1. We are creating a GIS for an oil transportation company based on the ArcGIS platform using UPDM. The trunk-line (magistral) pipeline contains the main pipeline, reserve pipelines, bends (branch), loopings, jumper piping (the pipeline that allows disconnecting individual sections if necessary (accident, repair, etc.)). Are loopings, bends, jumpers objects related to the PipelineLine UPDM class?
2. We need to determine the sections of a division of the pipeline into constituent parts according to UPDM classification. We do not plan to allocate isolation subsystems, cathodic protection, etc. The pressure in our pipeline is almost the same everywhere. By what criteria can a more detailed division of the pipeline into constituent parts be made?
3. The correct functioning of the LRS, as well as Pipeline Referencing, is very important for us. How will the choice of one or another pipeline splitting option affect the work of LRS and Pipeline Referencing?

Thank you so much for your help

Best regards,

Ainur

[Anonymous User]

Is Utility Network feature dataset is UPDM 2019 not editable? I am trying to add features in Pipeline feature class, but it says "Warning: Could not edit the data in the folder or database you selected".

How can I add features if its not editable?

[DianeMorin]

Hi @TomDeWitte ,

We are building out an LRS network using the UPDM 2019 data model and trying to make the P_GasLeak feature class an event. The feature class has fields that indicate it is intended to be used as an event layer (event id, to/from date, measures), however when using the "Create an LRS Event from an existing dataset" Location Referencing tool, we get an error message that says the input feature (P_GasLeak) is not in the same workspace as the network feature class (P_ContinuousNetwork). This is because P_GasLeak resides in the root of the UPDM db whereas P_ContinuousNetwork resides in a dataset. What would be the recommended way around this issue?

Thanks,
Diane Morin
Critigen

[TomDeWitte]

Hi Diane,

The ArcGIS Pipeline Referencing solution has a requirement that all featureclasses which are to participate in the LRS must be within the same feature dataset as the LRS topology & routes (ContinuousNetwork).

In UPDM 2019 the P_GasLeak featureclass is stored at the root level of the geodatabase as you noted, which is not compatible with the APR geodatabase requirements.  In UPDM 2020 this was corrected and the P_GasLeak featureclass was moved to be within the PipeSystem feature dataset.

Recommend you move the P_GasLeak featureclass to be within the PipeSystem feature dataset.

Hope this helps

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[Ainur_Rychshanova]

Hi Tom,

We are still trying to build our UPDM system. I have a question about using PipelineLine feature class as Centerline feature class for our LRS data model.

In our LRS data model we have 2 LRS Networks which have some identical geometry. We have tried to get rid of it by using the "Remove Overlapping Centerlines" tool. But we got the error 130166: The centerline feature class cannot be M-aware. So, can we just import our PipelineLine feature class to the same geodatabase as feature class without M-values or there is some other way to solve this problem? Maybe the UPDM model has already had some solution for our problem. Our main goal is not to duplicate our geometry when we are trying to load 2 LRS Networks.

Thank you in advance for your help

[TomDeWitte]

Hi Ainur,

Great to hear that you are continuing to make progress in implementing UPDM with the Utility Network and Linear Referencing (APR) configuration.

-Yes, the UPDM + UN + APR configuration does support multiple LRS networks.

-The combining of the two networks onto the singular PipelineLine based centerline configuration as you have found out is not a simple 1 tool solution. The consolidation of data you are attempting is complicated and will require multiple tools and possibly some python scripting.  

Tom DeWitte

Esri Technical Lead - Natural Gas

[JohnHu]

In UPDM 2019,  is there defined codes or values for the LIFECYCLESTATUS to use ?  Or should we have to define our own values.  Please note that this attribute appears in multiple UPDM feature layers such as Compressor Station:

lifecyclestatus (Short Integer)—The text value indicating the life cycle state of the compressor station.

Please help.  Thanks.

[Dianaran]

APR in UPDM & APDM streamlines referencing with fields like "engrouteid" in the unified P_PipelineMarker layer. The new data model eliminates the necessity for separate "Location" layers, simplifying and automating referencing processes for improved efficiency.

[evanroman]

How does APR in UPDM & APDM streamline referencing, specifically by eliminating separate "Location" layers for enhanced efficiency?

[Dianaran]

APR in UPDM & APDM streamlines referencing by integrating location data, eliminating separate layers. This integration enhances efficiency by providing a centralized system for accessing and managing location-related information.