Any ideas on the best workflow for this?
Solved! Go to Solution.
One way is using commercial extract, transform, load software to convert and load your data into the address management schema. Another option is to use Python to set up an extract, transform, and load process yourself. Create a copy of your data, add new fields for anything that needs to be formatted (i.e. string to integer or vice versa), calculate values into the new fields, and then use the append tool with field mapping. I know that is easier said than done, but it works without having to buy additional software. I guess you could use Model Builder as well, but I prefer the control Python offers.
Either way I would start with a small sample to get the process down and then apply the final process to the entire dataset.
One way is using commercial extract, transform, load software to convert and load your data into the address management schema. Another option is to use Python to set up an extract, transform, and load process yourself. Create a copy of your data, add new fields for anything that needs to be formatted (i.e. string to integer or vice versa), calculate values into the new fields, and then use the append tool with field mapping. I know that is easier said than done, but it works without having to buy additional software. I guess you could use Model Builder as well, but I prefer the control Python offers.
Either way I would start with a small sample to get the process down and then apply the final process to the entire dataset.
I would agree with @Joshua-Young , Python is really great.
You could create a Jupyter Notebook first to interactively test and build up the process before offloading it to a standalone script of some kind. I would highly recommend looking into using pandas for manipulating and reshaping your data.
We have a layer of address points that we maintain for a proprietary CAD software, but we also need to submit our address points in a NG911-compliant schema for a statewide initiative, so we developed a script that handles the entire transformation.