Address Management: Best workflow for converting existing address point data into solution schema?

367
2
Jump to solution
09-13-2022 11:12 AM
avonmoos
Occasional Contributor

Any ideas on the best workflow for this?

0 Kudos
1 Solution

Accepted Solutions
Joshua-Young
Occasional Contributor III

One way is using commercial extract, transform, load software to convert and load your data into the address management schema. Another option is to use Python to set up an extract, transform, and load process yourself. Create a copy of your data, add new fields for anything that needs to be formatted (i.e. string to integer or vice versa), calculate values into the new fields, and then use the append tool with field mapping. I know that is easier said than done, but it works without having to buy additional software. I guess you could use Model Builder as well, but I prefer the control Python offers.

Either way I would start with a small sample to get the process down and then apply the final process to the entire dataset.

"Not all those who wander are lost" ~ Tolkien

View solution in original post

2 Replies
Joshua-Young
Occasional Contributor III

One way is using commercial extract, transform, load software to convert and load your data into the address management schema. Another option is to use Python to set up an extract, transform, and load process yourself. Create a copy of your data, add new fields for anything that needs to be formatted (i.e. string to integer or vice versa), calculate values into the new fields, and then use the append tool with field mapping. I know that is easier said than done, but it works without having to buy additional software. I guess you could use Model Builder as well, but I prefer the control Python offers.

Either way I would start with a small sample to get the process down and then apply the final process to the entire dataset.

"Not all those who wander are lost" ~ Tolkien
jcarlson
MVP Esteemed Contributor

I would agree with @Joshua-Young , Python is really great.

You could create a Jupyter Notebook first to interactively test and build up the process before offloading it to a standalone script of some kind. I would highly recommend looking into using pandas for manipulating and reshaping your data.

We have a layer of address points that we maintain for a proprietary CAD software, but we also need to submit our address points in a NG911-compliant schema for a statewide initiative, so we developed a script that handles the entire transformation.

- Josh Carlson
Kendall County GIS