Best approach at continually updating data

[AdrianWelsh]

Hi Everyone,

I am hoping to generate some ideas on how I should best handle a weekly update of some GIS data, particularly the attributes.

This seems simple enough but I wanted to see what other’s thought. I am working on a project where we have some parcels and each one has a status (with like 9 different options). Each week, we update this status with a new status, but only when it needs updating. The parcels have unique identifiers.

We get this info in the form of an Excel sheet (with the unique parcel number and the new status, etc.). So far, I have been manually updating each parcel’s status with the new status since this is the quick and dirty way, but I want to find a more automated solution.

Based on this vague information, what are your thoughts on the most efficient way of handling this?

Thanks!

(to shorten this post, I’ll put some thoughts in a reply below)

[AdrianWelsh]

I had thought of making a Python toolbox through ArcMap or ArcGIS Pro that would list each parcel, show the current status, and then give a drop-down list for updating the status. But, it seems that the toolboxes do not necessarily behave this way. This might call for a different kind of program.

I had thought of having some kind of interface that lists each parcel and allows the user to update the status with a drop-down, and use the UpdateCursor (arcpy) to update each row, etc. But, wasn’t sure if this would work the best way. My thoughts were leaning towards utilizing the tkinter module to make a form…

The data is in a file geodatabase but that doesn’t really make or break anything. The data will also be displayed in ArcGIS Online. I am not opposed to using any kind of web-based programming to handle these updates either.

Anyways, I would love ideas and suggestions

[RebeccaStrauch__GISP]

Adrian, if it is already in and Excel spreadsheet with a common field (parcel ID), why not just related the two files and calculate the field.  If there isn't a reason to view this manually before the update, the should work faster.

might not hurt to create a backup of the attribute table before the update, in case there are any blips in the process.  Depending on where the data is stored, this might just be a version or an archive.

Geodatabase archiving—ArcGIS Help | ArcGIS for Desktop

[AdrianWelsh]

Rebecca, thanks for the reply. That is a great idea and would work well if the data was always clean and nice. I guess I could 'force' the people providing the data to make it match my domains. Definitely good idea about the archiving.

I had the thought of allowing the project manager access to some kind of updating tool so that he could be the one putting in the updates (with a simple drop down list) when he got the updates (but these types of people really like to delegate, haha). The other thing is, it's not always in a spreadsheet. Sometimes it's an email list.

But, I like the idea of making this a spreadsheet type of action to where I can force the updating people to give the correct information, and then relate, then calculate the field.

[TedKowal]

Adrian  Just to add a little more to Rebecca's comments...  I have a similar situation with updating maintenance needs on bridges.  Our situation, we have many different consultants inspecting bridges all with different software systems and such ... but all had an excel spreadsheet in common.  So I learned and applied a little vba programming to excel to "Standardize" and validate the excel form to match our domains and attributes... this being a little down and dirty has work out very well for us.

[DarrenWiens2]

This sounds like something accomplished with a join (on parcel number) then field calculate status.

[AdrianWelsh]

Darren, this just seems too simple. I want to make an app that will be flashy and do everything I want without any issues!!

Though, I like your approach. I might have to start making people give me spreadsheets in a standard format instead of the [junk] that I always get, haha.

[DarrenWiens2]

For your own sanity, I'd keep it simple as possible - it is so painful trying to anticipate every little thing the user can do to break your system. If possible, you should only give them the option to enter data correctly. I agree you should request a standardized format. You may also consider using a database that can take advantage of data validation upon entry (like an FGDB domain, although which DB you choose will depend on your setup).

[AdrianWelsh]

I like this idea. Sanity is good. Users are good at breaking systems, unfortunately. I will have to implement something like this. I appreciate y'all's help!

[WesMiller · Accepted solution]

Adrian,

You could absolutely over complicate if you wanted.

1. Get your domain
2. Validate your excel against your domain
3. Fix errors
4. Join the tables
5. Query for changes originField <> updatedField
6. Calculate originField to match updatedField

#import module
import arcpy


#get the feature class we want the domains for
fc = arcpy.GetParameterAsText(0)


desc = arcpy.Describe(fc)
desc1 = arcpy.Describe(desc.path)


#print desc1.datasetType
if desc1.dataType == 'FeatureDataset':
    fcworkspace = desc1.path
else:
    fcworkspace = desc.path




#List the domains
domains = arcpy.da.ListDomains(fcworkspace)


#List the fields 
fields = arcpy.ListFields(fc)


domainList = []


#Loop through the fields, determine if they have a domain and add
#those with domains to the dictionary and assign it a default value
for field in fields:
    if field.domain:
        domainList.append(field.domain)


#Create a dictionary to hold the data
domaindict = {}


#Loop through domains and grab the coded values for feature class
for domain in domains:
    if domain.domainType =='CodedValue':
        if domain.name in domainList:
            domaindict[domain.name]=domain.codedValues
textline = ""
for key,value in domaindict.iteritems():
    textline = textline + key + '\n'
    for k,v in value.iteritems():
        textline = textline + '\t' + str(k) + '\t:\t' +str(v) + '\n'
arcpy.AddMessage(textline)