Utility Network - Subnetworks: Move "Manage IsDirty" option to validation function

[JürgenBiendara]

When configuring subnetwork definitions, several options must be specified, including the "Manage isDirty" option. This option determines whether subnetworks should be marked as edited during validation if elements of the subnetwork have been edited.

Would it be possible or useful to include this option as a dynamic option in the "Validate" function instead of or in addition to permanently anchoring it in the core configuration of the Utility Network?

[RobertKrisher]

@JürgenBiendara can you provide a bit more context for why you want this? Is it that you want to set a tier to be managedirty=true and occasionally ignore a validate? Or do you want to have a tier be managedirty=false and choose to only mark the network as dirty at a time-and-place of your choosing?

[JürgenBiendara]

Hi @RobertKrisher ,
Thank you very much for your feedback. I would be happy to try and explain my thought process in more detail.
I was wondering why you want to permanently (!) set for a tier whether the subnetworks should be marked as "edited" (=dirty) or as "not edited" during validation.
If I understand it correctly, the marker that a subnetwork is dirty means that you should start the "Update Subnetworks" tool to update the subnetworks. For me, this process belongs in the overall context of data up-to-dateness and thus validation (during which the topology is written). I would therefore link the "Manage IsDirty" option with the validation of the edited data.
You could leave it up to the user, with a dynamic option, whether they want to take the subnetworks into account during validation (i.e. mark them as dirty) or not. Thinking a little further, you could then also offer the updating of the subnetworks as a direct follow-up action to the validation.
I hope I'm not talking complete nonsense here. 😊
Best regards,
Jürgen

[RobertKrisher]

@JürgenBiendara we have a detailed article explaining the purpose of this option here Subnetwork Deep Dive: Status. I'll try to summarize it here. Managing Dirty means that when data is validated, the system will identify which specific subnetworks were modified by the edit. This is used to assist in quality assurance, so an editor knows specifically which subnetworks need updating. If you disable this setting, then the system will no longer perform this analysis and every subnetwork will always be considered dirty. Running update subnetwork is always a separate operation that must be run separately by a user (or process), the state of clean/dirty is just a way of indicating which subnetworks have been modified.

So, if you care about precision with your quality assurance and just focusing on the subnetworks that have changed, you want to manage the status of your subnetworks. If you don't have any subnetwork-specific quality assurance and are ok with just running update subnetwork on all your subnetworks on a periodic basis, then you can disable managing the status of subnetworks.

[JürgenBiendara]

@RobertKrisher 

Many thanks for the information and the reference to the very interesting article. I have understood the procedure from a technological point of view, but unfortunately not yet fully from a workflow perspective.
The fact that the "Manage IsDirty" option is anchored in the tier definition means that there is no flexibility - the tier is always handled according to this configuration unless the tier definition is changed. However, this means that only the administrator of the utility network can determine whether the subnetwork status should be evaluated during validation.
If, on the other hand, this option were offered as an option in the validation function, the user could decide for himself whether or not to have the status of the subnetworks checked during validation.
Best regards, Jürgen

[RobertKrisher]

I'm still waiting for you to answer my initial question around what your use case is for this. From your discussion, it sounds like you want the state to be normally managed but want to give users the ability to opt out of this check. But why?

If a user opts out of marking subnetworks as dirty, then they are destabilizing the system by allowing a subnetwork to appear clean when it actually needs to be updated. This means that the subnetwork name and "is connected" field can no longer be relied on to be accurate. Any integrations, interfaces, or tools that rely on that information or look at the status/last modified date of subnetworks will also become unreliable or out of sync with the GIS.

Given all these problems, what is the driver behind this? Why are you intentionally wanting subnetworks to ignore certain edits and have their information become out-of-date (with no indication that it is inaccurate)?

[JürgenBiendara]

@RobertKrisher 

I don't want the subnetwork information to be out of date - why would anyone want that? On the contrary, I would like the subnetwork information to be kept up to date by the system as automatically as possible. That's why I'm surprised that this option even exists.
I don't have a specific use case in mind with my suggestion. But now that this option exists, I wonder why it is fixed for each tier in the configuration and cannot be used dynamically. Ultimately, with this option it is possible to make the subnetwork information of an entire tier out-of-date, also with no indication that it is inaccurate. Or am I wrong?

[RobertKrisher]

This is outlined in the article I linked to, so you may want to do a closer reading to make sure you were following all the points. When you set manage isdirty=false, the subnetwork will always have the status of dirty. For customers with large water and gas systems, it does not make sense to track status because any time any edit occurs in the network the entire system becomes dirty. By setting manage isdirty=false it makes their validates a little faster (because it doesn't have to trace the entire system) and they just run a script to update their system subnetworks several times a day.

The default behavior of the system is to track the status of subnetworks, we added the option to not manage the status for customers using hierarchical networks that had large tiers like the system tier. We would never want to you have a tier configured to manage status and then allow a user to opt-out during validate because this would invalidate the state of the entire system.

[JürgenBiendara]

@RobertKrisher 

Hi Robert,
Thank you for the clarification. It was not obvious to me that the "Manage IsDirty" option was a special case or exception that was not originally intended in the concept of the Utility Network. With this in mind, this suggestion can be closed or deleted.
Best regards,
Jürgen

[RemiMyers]

As per discussion below.