UPDM 2019 Edition

[#1]

This is the official release of the 2019 Edition of the Utility and Pipeline Data Model (UPDM).

 It is designed to support data management for the natural gas and hazardous liquids industries.  Supporting the diverse needs of these industries means supporting multiple implementation patterns.  Specifically a network topology with the Utility Network implementation pattern, and a linear referencing implementation pattern with the ArcGIS Pipeline Referencing solution..

NOTE: April 3,2020:  Updated the Asset Package to remove the folder "ap_workspace", and to remove some outdated schematic design templates.  With the ArcGIS Pro 2.5 release of Utility Network Package Tools, this is no longer needed, and is generated errors in some loading instances.

If you have questions, please post them to geonet, so everyone can see and share the information.

Thank you

Tom DeWitte

Esri Technical Lead – Natural Gas Industry

[mrasiukevicius]

What is the best way to upgrade UPDM2018 to UPDM2019 version? Apply Asset Package fails with message "This Utility Network was created with an older release of the software". Previous version UN was created using Pro 2.3.

Also AssetPackage Domain network name has changed from "Gas" to "Pipeline".

[PaulLeBlanc1]

If you want to use Apply Asset Package at Pro 2.4, you'll need to create a new UN or upgrade your existing UN.

[TomDeWitte]

Following up on your question about the Domain change.  The name was deliberately changed from Gas to Pipeline to better represent that UPDM is a data model for both Gas and Hazardous Liquid pipe systems.

[mrasiukevicius]

Seems that create new UN is the only way. Tried GeoDB upgrade to latest version, UN Dataset upgrade to ver.3. But then Apply Asset Package fails with messages:

Running script AssetPackageToUtilityNetwork...
ArcGIS Pro 2.4.19948
untools 2.4.0
Executing from ArcGIS Pro, 5 map(s), activeMap = False
Parallel processing enabled: 9
Asset Package schema version: 2.4
 The asset package has the following errors:
                             Item                           Field   OID Value                                                 Message    Level
0   P_LineValveInspection__ATTACH  p_linevalveinspection_globalid  None  None      This field is too long for some database platforms  warning
1                P_PipelineMarker                       condition  None  None  This field name is reserved on some database platforms  warning
2               StructureJunction                         assetid  None    64                             Target field length is '32'    error
3               StructureBoundary                         assetid  None    64                             Target field length is '32'    error
4                   StructureLine                         assetid  None    64                             Target field length is '32'    error
5                  PipelineDevice                         assetid  None    64                             Target field length is '32'    error
6                PipelineAssembly                         assetid  None    64                             Target field length is '32'    error
7                PipelineAssembly                            name  None    64                             Target field length is '20'    error
8                PipelineAssembly             locationdescription  None   100                            Target field length is '255'    error
9                PipelineJunction                         assetid  None    64                             Target field length is '32'    error
10                   PipelineLine                         assetid  None    64                             Target field length is '32'    error
 Failed to execute (AssetPackageToUtilityNetwork).

[LeeDavis1]

Do you anticipate this new model to be the standard for some time? Or will you all continue to make yearly updates? Just trying to decide if I should upgrade from 2018 version in the next few months. I've had to extend the model quite a bit for pipeline midstream work.

[TomDeWitte]

Mindaugas,

Thank you for pointing out these Asset package issues. I have modified the Asset package and documentation to account for the following changes:

1) P_PipelineMarker - Changed the name of the field "condition" to "markercondition" to not conflict with the PostgreSQL reserved word constraint.

2) Updated the change log and Data Dictionary to correctly reflect the change in field length of "assetid" from 32 to 64.

3) Removed the table "ValveInspection" as it was a duplicate of "P_ValveInspection"

Please let me know of any additonal issues or concerns

thanks

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[TomDeWitte]

HI Lee,

The Utility and Pipeline Data Model is planned to continue to have annual updates.  This is important to keep the data model up to date with changes to ArcGIS capabilities and industry best practices.

Since you extended the data model to meet your needs for pipeline midstream, would really like to know what changes you felt were needed to meet your needs.  Would you be willing to share those changes that you feel are applicable to the entire industry, and should be in the default data model.

thanks

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[deleted-user-FrOR7qrGn6Op]

I was curious about the same thing as Lee Davis, and am a little disheartened to learn that this will be a yearly occurrence. I'm only now becoming comfortable with navigating the 2018 version and our company has has also extended the data model. This amount of frequent and drastic change makes maintenance difficult with things like third party applications, Event Editor, automated data population scripts, layer files, and map documents just to name a few. Having tools that edit/migrate the database are a nice feature, but that's only scratching the surface of the impact from this kind of change. Since we're only using the linear referencing for transmission, and are not ready for the Utility Network, we've decided to only make necessary modifications that came with this version. One last thing I'll mention is that three versions in and the word engineering is still spelled incorrectly in places... which pretty much sums up my confidence in this as a product.

[TomDeWitte]

Hi Jeanetta,

Appreciate the honest feedback on our efforts with the Utility and Pipeline Data Model (UPDM).   We update annually to keep our promise to the pipe community.  The promise was to keep the data model current with changes in Esri technology, changes in industry organization and changes in regulatory compliance. That promise was made over ten years ago when Esri took back responsibility of the gas industry data model. We continue to make every effort to keep that promise.   

I completely understand the frustration of trying to keep up with the changes.  My recommendation for you is to keep doing what you are doing.  By that I mean, review the changes (ie. read the change log), and determine if implementing those changes will help your organization.

As for the spelling errors.  I take full responsibility for those.  I am an engineer and as everyone knows that means my spelling is horrible.  We will work to improve our review processes to do a better job of catching my spelling errors before you see the finished product.

Thank you again for your honest feedback, and please continue to provide feedback.  It is honest feedback like yours, that helps us to continue to improve our products.

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[TheodoreRakel]

I notice that there's 1150 features in the feature class PipelineLine in the asset package.  There's 1017 features in PipelineJunction.  Is that intentional?  

[TomDeWitte]

Hi Theodore,

Yes is was intentional for there to be features in the core asset featureclasses (PipelineLine, PipelineDevice, PipelineJunction, PipelineAssembly, StructureJunction) and ServiceTerritory.  This is a sample dataset to help understand what a pipe system in UPDM looks like.  

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[BarendNackaerts]

Hello,

Is there a reason why the assetgroup "service line" isn't named "service pipe" like the other assetgroups?

Although I think it has a reference in the fitting assembly.

Thx,

Barend

[TomDeWitte]

Hi Barend,

You are correct that using the term "Service Line" instead of "Service Pipe" is an inconsistency in the naming conventions for the PipelineLine Asset Groups.  I will mark it down for review and likely correction in the UPDM 2020 Edition.

Thank you for pointing this out

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[LeeDavis1]

Tom, 

I'd be happy to setup a go to meeting and let you all tour my GIS to get a feel for how and why I have made certain model extensions/ additions.

Email me at lee@wwm-llc.com if you all are interested.

Thanks,

Lee

[JoeHershman]

Tom,

Does this support fittings with differing inlet and outlet diameter as per ASTM standard?

Thanks

-Joe

[TomDeWitte]

Hi Joe,

The quick answer is "Yes".

With UPDM 2019 you will see new attributes on the PipelineJunction and PipelineDevice to support fittings and devices with differing inlet and outlet diameters and wallthickness.  This is documented in the Change log doc which is a part of the download.  Based on our internal testing, UPDM 2019 is now fully in sync with the data management needs of the ASTM F2897 barcode standard.

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[JelmerAkkerman]

Hi Tom,

Thanks for making this edition available. I've downloaded and successfully created a new (10.7) enterprise geodatabase with a Utility Network following the steps described in the Implementation Patterns document for UPDM 2019. Since I would like to implement UPDM using Linear Referencing Data Management, I'm trying to delete the Utility Network topology through the Catalog view in ArcGIS Pro (2.4.0) as instructed in the Implementation Patterns document. That however gives me an error indicating that the object 'May be locked by another application'. There are no users connected to/locks on the geodatabase. Any idea what is causing this error and how can it be avoided?

Thanks,

Jelmer Akkerman.

[TomDeWitte]

Hi Jelmer,

I asked the Esri Utility Network team about this, and it seems to be a bug that we inadvertently added to ArcGIS Pro 2.4. In the interim until the bug is fixed it was suggested that you try using the ArcGIS Catalog view to copy and paste the data model from the file Geodatabase based Asset Package to your Enterprise Geodatabase.  When using the copy/paste, please ignore the tables which are prefixed with A_, B_, and  C_  .

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[José_JoaquínMuñoz_Villarreal]

Hi Tom:

The question we have is how UPDM manages the offline layers to reference them to the network line. In the old APDM model there were several layers that had in their name the ending "Location". These layers were used to represent the information of the layers that are outside the line and reference them on the line. In the new UPDM data model I understand by the description of the APR fields that the reference on the line will be in the same layer, that is, before there were Marker and MarkerLocation layers and now there is only the P_PipelineMarker layer. Within this layer are the fields "engrouteid", "engroutename", "engfromm", "engtorouteid", "engtoroutename", "engtom", "continrouteid", "continroutename", "continfromm", "contintom", "fromrefmethod", "fromrefoffset", "fromreflocation", "torefmethod", "torefoffset", "toreflocation" and "locationerror". The values of these fields are generated and administered automatically by APR (that's what Esri's documentation says), how does APR manage these fields? If so, don't we need the "Location" layers in the new data model?

Thanks.

Jose Joaquin Muñoz Villarreal

[TomDeWitte]

Hi Jose,

It sounds like there are two key questions you are asking.

Question 1: How does UPDM and APR handle location, and how is it different from an APDM based linear referencing solution?

Question 2: How does APR create and maintain route and measure information within UPDM?

If these are not the questions you are looking to understand please reply to this forum and let me know.

Question 1 Response:  In APDM event layers were managed as geodatabase tables which contained route and measure information as attributes.  To display the location of these events on a map requires special software such as ArcMap which was able to dynamically generate geometries from the route and measures on-the-fly and display them on a map. This was an inefficient process that did not perform well as the number of records grew.  In UPDM and ArcGIS Pipeline Referencing (APR) we addressed this issue by turning all event layers (geodatabase tables) into featureclasses.  This eliminated the need for special processing to display event layers on a map, greatly increasing the display performance, and improving scalability for displaying larger data sets. 

Question 2 Response: The route and measure information in a APR + UPDM environment is automatically created and maintained by the ArcGIS Pipeline Referencing tools. The following fields:"engrouteid", "engroutename", "engfromm", "engtorouteid", "engtoroutename", "engtom", "continrouteid", "continroutename", "continfromm", "contintom", "fromrefmethod", "fromrefoffset", "fromreflocation", "torefmethod", "torefoffset", "toreflocation" and "locationerror" are part of the APR Information schema and are included in UPDM. These fields once APR is installed and configured are automatically managed by the APR  and not the end user.

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[José_JoaquínMuñoz_Villarreal]

Thanks Tom.
Those were indeed the issues. Thank you for the answers.

Regards

[PabloSantos]

Hi Tom,

I have a question referents the feature class P_Anomaly (Integrity) and the feature class P_ILISurveyReadings (Inline Inspection). Both features classes have the same description and we do not know which is the different between them apart from the value's domain from the type fields.

Could you explain to me which are the functional differents between both features classes in UPDM model? In APDM we have only the feature class Anomaly.

We are analizing the UPDM model to make the migration from APDM to UPDM and we do not know on which feature class are we going to migrate the data.

Thanks Tom,

Pablo Santos Merayo

[TomDeWitte]

Hi Pablo,

You are correct the Data Dictionary description for P_Anomaly and P_ILISurveyReadings are a little vague and could use improvement.  I will mark that down as a task for the UPDM 2020 data dictionary.  In the meantime here is an additional description of these two featureclasses and their purpose.

P_ILISurveyReadings:  The purpose of this featureclass is to store the raw data collected from an Inline pipe inspection.  I hear most pipeline organizations describe this as the data collected by a smart pig run through a portion of the pipe system.

P_Anomaly:  This is a filtering of the data collected by smart pig runs, Direct Assessment, Close Interval Surveys, and other pipe inspections.  It identifies the locations where a degradation of the pipe system has been identified by the previously mentioned pipe inspections. These degradation's include correction, material defects, and geometric distortions.

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[PabloSantos]

Thanks Tom.

Its like we thought. Thank you for the answer.

Regards,

Pablo Santos Merayo

[PabloSantos]

Hi Tom,

I have created the UPDM schema in ArcGis Pro and when I want to draw the Centerline elements I get the next message: "Layer is marked not editable. You control application level editability in the List by Editing view of Contents pane.", and in the Contents pane the message is: "This table is registered as versioned with a different model than connected version. Edits cannot be made".

The geodatabase has only one version created and the default. I am the owner of the version and I can edit all the Features Class of the gdb, ver5sioned and no versioned, except the APR Feature Classes. I could not find the reason of it and I need to create the centerlines elements. 

I am attaching a capture of the ArcGis Pro meesages.

If you can guide me I will be very grateful with you.

Thanks Tom,

Pablo Santos Merayo

[TomDeWitte]

Hi Pablo,

I have experienced this error message before.  It is due to a mixture of Traditional versioning and the newer Branch versioning. Here is what I suggest you do to verify if what you are experiencing is the same as I have experienced.

Step 1: In ArcGIS Pro, go to the Catalog panel.

Step 2: In the Databases portion of your Catalog panel, select the Database connection you are referencing to edit.

Step 3: Right-click on this connection and select "Geodatabase Connection Properties"

Step 4: At the top of the dialog window, check to see what type of versioning your connection is set for (Traditional or Branch).

Now comes the decision on what type of versioning is correct.

-If you are editing through the feature service, then you must have your geodatabase connection configured for Branch versioning.

-If you are editing through the database connection, then you can use Traditional versioning.

The tool for upgrading your geodatabase connection to branch versioning is "Update Geodatabase Connection Properties to Branch".  It is located "Workspace" toolset in the "Data Management" toolbox.

I hope this helps

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[PabloSantos]

Hi Tom, 

Thanks for your prompt answer. I had alredy tried to it creating a connection database connected to the version that I have created, however, it does not work. I used the "Update Geodatabase Connection Properties to Branch" tool now and I tried to edit again but the same message is appearing in the Contents pane.

So, it is not the solution to resolve my problem, any more idea? I really need to create the Centerline elements to continue with the migration to UPDM model.

I am attaching some captures of the ArcGis Pro messages.

 

Thanks Tom,

Pablo Santos Merayo

[PabloSantos]

Hi Tom,

Finally I could create the RLS Network and I am working on it. However, I found another problem, I draw some Centerlines and I could created some routes in the network, but I have two centerlines which are crossing and I cannot crate two routes from these centerlines.

This is the situation, in the image you can see both centerlines:

I have created the first route and when I try to create the second one, the image message error appears, my question is, how can I create two differnet routes which are crossign each other?

Both pipelines have different depth, so it should not be a problem.

I wish you can help me with this doubt.

Thanks Tom and regards,

Pablo Santos Merayo

[NathanEasley]

Hi Pablo,

Do either of your centerlines self-intersect?  Are you trying to add the second route onto a line that includes the first route?  Are you selecting one or both centerlines when trying to create the second route?  If you can give me a little more information on the edit being made, we can determine why the edit isn't going through.

Thanks,

Nathan

Esri Pipeline Referencing team

[PabloSantos]

Hi Nathan,

Do either of your centerlines self-intersect? No, Any centerline does self-intersect. 

Are you trying to add the second route onto a line that includes the first route? Yes, the case that I am working on, consist in two different routes which are crossing each other like the image I published on the first text. I have tried to split both centerlines in the crossing and as a result I get four centerlines, then I tried to create the route by selecting two centerlines as you are suggesting to me, but the error message was the same.

Thanks Nathan,

Pablo Santos Merayo

[NathanEasley]

Hi Pablo,

Would it be possible for you to share a copy of your data and the steps you took to create the error?  If so, please let me know and I'll give you a location to upload the data.

Thanks,

Nathan

[PabloSantos]

Hi Nathan,

Yeah, I could send to you the data and I will indicate where it is the problem occurs. Please give to me a secure way to send to you the data.

Thanks Nathan,

Pablo Santos Merayo

[BerndWermes]

Hi Tom,

Great work on the UPDM model. We're following the implementation pattern #3 for UPDM 2019 v5 (utility network with location referencing). Now, when it comes to the point to setup the ALRS the documentation says the PipelineLine feature class MUST be configured as the Centerline layer for location referencing. Here we have got two questions:

1. Which of the attributes of the PipelineLine feature class should become the Centerline ID to be used in APR (see screenshot)?

2. The PipelineLine feature class contains attributes such as: route ids, route names, engineering and continuous measures, which are usually used in APR managed line event feature classes. These names are not common for the Centerline feature class when setting up APR without the utility network. In a stand-alone APR environment, measures are available within the continuous and engineering network feature class, not in the Centerline feature class. I'm wondering if the PipelineLine feature class is the right feature class to use as a Centerline feature class as we know it from the APR context. Please can you elaborate?

Thanks in advance,

Thomas Schuster

[TomDeWitte]

Good morning,

Thank you for the complement about UPDM. Glad to hear you are finding it useful in your efforts to manage pipe systems.  Here are my responses to your questions.

1. Which of the attributes of the PipelineLine feature class should become the Centerline ID to be used in APR (see screenshot)?

 TD Response:  I would recommend that you add an attribute named: CenterlineID with a data type of GUID.  This was inadvertently left out of UPDM 2019 and is on the list to be added to UPDM 2020.

2. The PipelineLine feature class contains attributes such as: route ids, route names, engineering and continuous measures, which are usually used in APR managed line event feature classes. These names are not common for the Centerline feature class when setting up APR without the utility network. In a stand-alone APR environment, measures are available within the continuous and engineering network feature class, not in the Centerline feature class. I'm wondering if the PipelineLine feature class is the right feature class to use as a Centerline feature class as we know it from the APR context. Please can you elaborate?

TD Response: With the upcoming January 2020 release of ArcGIS 10.8, we will be enhancing APR to allow the PipelineLine feature class to be both the centerline and a linear event layer.  This is inline with one of our main integration goals which was: No duplication of data.  Having the pipes layer be the source of geometry for the routes is key to also streamlining the data editing process.

Please continue to ask questions through this forum, so the information can be shared with everyone.

thanks

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[BerndWermes]

Wonderful, thanks for the quick update!

[MarissaMijal]

Hi,

I'm looking for best practices on moving a UPDM database from one Oracle Server to another.  This UPDM contains both Utility Network and LRS.

Thanks!

[TomDeWitte]

Hi Marissa,

Moving an Enterprise Geodatabase with UPDM, Utility Network properties and ArcGIS Pipeline Referencing properties from one Oracle instance to another Oracle instance is a challenge.  Here are a couple of ideas:

1) Use the Oracle backup and restore tools to move the entire Oracle instance (Geodatabase tables, featureclasses, UN properties, APR properties, versioning properties) from one server environment to another.

2) Use the "Export Asset Package" tool provided with the "Utility Network Package Tools" georprocessing toolset.  You can download these additional tools from Esri's solution web site at the following URL: Utility Network Package Tools | ArcGIS Solutions   The Export Asset Package will export the full set of Utility Network properties, and all data stored in those Utility Network specific featureclasses. It should also grab the other data in the PipeSystem feature dataset such as the APR event featureclasses.  What I am not sure of is whether it will also grab all of the APR configuration tables.  Since the export is actually a file GDB, you can also use the ArcGIS Pro copy/paste tools to copy over any other tables or featureclasses that were not copied by the Export Asset Package tool.  To import into the other Oracle instance use the "Apply Asset Package" tool.  an additonal warning about this tool.  If you have a populated multiple version environment.  This tool will only capture the data associated with the single version specified in your database connection properties.

Option #2 is an option I have used several times for moving by UPDM with UN enterprise geodatabases.  Hopefully it meets your needs.

Hope this helps

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

tdewitte@esri.com

[JavierPablos1]

Good morning,

Is there an estimated date in which UPDM 2020 will be released?

Thanks a lot in advance.

Cheers

[TomDeWitte]

Hi Javier,

The list of fixes and enhancements for UPDM 2020 are being finalized now.  No firm date for release has been set yet.  Release goal is similar to last year which is to have UPDM released by the July 1st.

Tom DeWitte

Esri Technical Lead - Natural Gas Industry

[MarissaMijal]

Thank you Tom,

I appreciate the heads up on what to look for using Option 2.

Marissa

[JavierPablos1]

Thank you Tom for the quick answer. Cheers.

[NurkuisaRametov1]

Thank you for all!

Who can share .lyrx file for UPDM?

Regards

[PaulaLopes2]

Hello,

In order to work with the UPDM and the Utility Network for a gas network, which data model should we chose as the base one? The UPDM? I've remarked UPDM has the 7 feature classes similar to UN ones, although not all AssetGroups of UN seem to be present. And what about all UN domains, are they also present in the UPDM model?

I've been reading the documentation that come with UPDM and in ArcGIS Pro help, but is not completely clear.

Does anyone experience in doing this?

Thanks,

Paula

[MarissaMijal]

Hi Paula,

At the top of this geonet page is the UPDM 2019 zip file by Tom Dewitte (UPDM 2019 Edition ).This zipped file contains the UPDM asset package (file geodatabase) with the Gas utility network included.  The asset package includes the domains and everything you need to get started creating an enterprise UPDM with the utility network.  If you want to add more asset groups, you can extend this file geodatabase, but keep in mind you have to add additional utility network rules when you add new asset groups.

Cheers,

Marissa

[XanderBakker]

Thanks Tom DeWitte‌ for uploading the updated model!

[PaulaLopes2]

Hi Marissa,

Although late (I was working on other projects), thank you for the answers.

Now I understand.

Cheers,

Paula

[SamiSnunu]

Hi Tom DeWitte:

You mentioned the UPDM 2019 Edition is designed to support data management for the natural gas and hazardous liquids industries..

However, with the help of the Electric Asset Package v3.3, we're looking for the UPDM 2019 documentation specific to Electric domain network, would that be available?

Thanks!

Sami

[JakeJacobs]

Tom DeWitte I was just seeing the electric data dictionary on the "experience" part of the solutions site (Experience).  Is the gas data dictionary heading there too? I couldn't find it.  I only found it here: Gas 

[TomDeWitte]

Hi Jake,

It is good to hear from you.  Hope all is well in Spokane.

The answer to your question is "yes".  As part of this year's annual update to UPDM, we are working to get UPDM 2020 posted on the Esri solutions page site, and to create a digital version of the UPDM data dictionary.

Tom DeWitte

 Esri Technical Lead - Natural Gas Industry

[Siti_Rohana_TazuraBinti_Ali]

Hi, 

User gave me some points and lines data, and wondered, how they can load their existing data to this updm 2019 model? The purpose is to apply for ArcGIS Pipeline Referencing . 

Their steps were:
1. download the data model.
2. extract and make folder connection to arcgis pro.
3. now figuring out how to use their own data in this model

Any input would be really appreciate. Thanks!

Tazura