Standardizing Data Storage for a Civil Engineering Firm

Question

Hey everyone,

The company I work for is recently getting more and more into ArcGIS but very few of us have training beyond ESRI's online classes. As we get more into GIS I have found that we have massive discrepancies in the data that is collected and how it is formatted. We are currently working to create web maps for some utilities companies but going through the data we have, it seems as though everyone has all sorts of different field names and attributes for many of the same things (like a Waterline being labeled "pipe" and "waterline" by different people with the attributes for Diameter as "DIA" and "Diameter"). I know tools like join and relate can work but any fields are not the same data type and it makes it very difficult.

This all becomes more difficult when add-ons such as InfoWater get involved that produces dozens of fields with unique names and varying data types. What are some of the best tools, add-ons standards of practice that could be applied to help standardize our data?  If there are none, what are some of the best ways you all have gone about doing this?

Thank you.

ThomasHamill · Answer

@DavisBuckley, I can share with you that I've similarly run into the same challenges throughout my time (past ten years) doing work for clients and colleagues at engineering firms when working with all type of data, especially utility network data.  There are some good solutions from Esri, like the Local Government Model (utilities geodatabase schema) and others.  At one point the LGM seemed to be growing in its adoption by municipal utilities, but I've found that the design often still had to be adapted and changed based on client needs and other requirements.  You might overtime come to refine your own special set of standardized geodatabase schemas complete with the effective domain names and descriptions that make it through the various trials of application and reliability.

The unfortunate reality is that data will almost always require some amount of development, processing, and management in order to get it into a condition that is more organized, useful, and robust than the condition it was in when received.

I recall hearing a saying once like "when one sees a map, you're seeing a product in which 5% of the effort/time was spent creating the map/cartographic representation (at the very end).  The other 95% of the effort/time was spent on data development."

DanaNolan · Answer

You may want to look into the SDSFIE federal standards and the Esri Utility models (I don't know what these are called now). SDSFIE has utility schemas and some standard domains.