Select to view content in your preferred language

Unique ID's & routine export and compilation of survey123 data

360
7
08-14-2024 02:15 PM
WindsorConnecticut
Occasional Contributor

I'm currently caught in a wormhole trying to figure out how to go about my project:

I'm creating a model in ArcGIS Pro 3.3.0 to generate a Master Address Table (MAT) for my municipality. We're using Survey123 Web Browser to collect subaddresses within our existing parcels (units, apartments, etc.). The Survey123 data will be downloaded, cleaned and added to a master subaddress shapefile, to later be added to another table of all addresses (MAT). I just did a lot of reading on UID's and found that Global ID's can repeat and are not necessarily 'unique'. 

I need help determining how to (1) create a process that will allow the new entries to be added to the master subaddresses, without overwriting the data that has already been cleaned and entered. I currently have the Append Tool in my model, (2) but it will not function until I have established a UID.

Potential Solutions(?) :

1. Play with settings in append tool to ensure the clean data is not overwritten or use a different tool if this isn't possible. 

2. Determine how to auto-generate UID's in Survey123 Web Browser, that will not repeat, especially if part of the solution to issue #1 is to delete the previous entries from the survey after they've been exported. The ideal situation would be to have a new UID every entry, even if one was deleted or scrapped to avoid error in the data. 

Each subaddress will be assigned another UID after they're compiled, both old and new but before combining with the parcel addresses (separate sequence format), so I don't need the UID to serve any other purpose other than being unique to allow the data to be appended to the existing/previously entered data table. I'm suspicious of the web browsers capabilities, and fearful of having to create this in Survey123 Connect. 

If anyone has any thoughts/suggestions/questions, all are welcome and very much appreciated! I keep reading and reading and my brain is going in circles... Thank you.

0 Kudos
7 Replies
ZacharyHart
Honored Contributor

Could you cite the source suggesting where Global IDs are not unique?

0 Kudos
WindsorConnecticut
Occasional Contributor

I read it in one of these blogs, perhaps that's incorrect? I hope so!! If a survey is cleared of data, is there any chance a GID may be repeated?

0 Kudos
ZacharyHart
Honored Contributor

The chance of a collision for UUIDs is so low it is hard to comprehend (especially within one database/dataset).

0 Kudos
WindsorConnecticut
Occasional Contributor

I also see in the file's properties that GID is not unique. I ran the "Add Index" tool to try to change this but it did not work. 

WindsorConnecticut_0-1723736451516.png

 

0 Kudos
ZacharyHart
Honored Contributor

this post may be helpful for you in planning out your process. This also covers what you are seeing above. Also, to my knowledge, having the index being set to unique enforces there not being duplicates, but even if the index on a Global ID field is non-unique, the Global ID values are expected to be unique because of the GUID generation process.

I mean, you can create your own UUID's all day long w/o the attribute index and (see previous citations about uniqueness) not worry.

 

def CalcGUID():
  import uuid
  return '{' + str(uuid.uuid4()).upper() + '}'

 


You could always capture your GlobalID's in an alternate UUID/GUID field if you need to actually keep them. [If you read the various supporting documentation via the thread I provided you, you will see the details about when/how GlobalID's can be preserved].

0 Kudos
abureaux
MVP Frequent Contributor

I'm with @ZacharyHart. GID should be unique as well.

0 Kudos
abureaux
MVP Frequent Contributor

uuid is a field in S123:

abureaux_0-1723747970333.png

 

In the table, OID is unique. Same with RowID (at least in the parent table). GID should also be unique.

There is a blog about UIDs in S123. Give it a read: Creating 'Ticket Numbers' in Survey123 for ArcGIS

If you don't want OID or the uuid field I mentioned above, you will likely need to manually generate them.

EDIT: I should keep my thought's together. Added GID to this post in addition to my post below...

0 Kudos