I have recently observed descripencies in time between what is shown in the attribute table in arcmap and what is obtained from reading the date field in python. In particular approximately 1 out of 10 time elements differ by one second. This difference prevents use of the datetime field for identification of duplicates.Flow: JSON serialized datetime (1) -> python datetime obj (2) -> written to feature class as obj (3) -> read from feature class (type = date) as obj(4).In the process the time element is consistent from step 1 to 3, however, the read time in (4) may be one second less than observed in the attribute table.Code to match and/or convert JSON to ESRI 10.0(python) datetime (and others):
def MatchDataType(ESRI_Cursor_Object, JSON_Data_Structure):
DType = ESRI_Cursor_Object.type
DName = ESRI_Cursor_Object.name
JData = JSON_Data_Structure[DName]
if DType == 'SmallInteger':
return int(JData)
if DType == 'Integer':
return int(JData)
if DType == 'SHORT':
return int(JData)
if DType == 'LONG':
if long(Jdata) < 2147483647: # Had to add this due to GeoPro 16 digit Integer range greater than ESRI accomodates with LONG
return long(JData)
else:
msgr('Conversion to long above ESRI limit for long integer')
return
if DType == 'Single':
return int(JData)
if DType == 'Double':
return long(JData)
if DType == 'String':
return str(JData)
if DType == 'Date':
if JData.find('Date('): # Java JSON date is expressed as milliseconds since IBM inception Date(XXXXXXXXXXXXX-YYYY) X = milliseconds Y=offset (positive or negative?)
# eg. /Date(1348860772000-0400)\ Note that if you do a find on '/Date(' it can fail! Some of my tests were successful others not.
try: # poor man's error checker in case the date stamp is not correct. In this case return nothing
index = int(JData.find('Date('))
JData_milliseconds = int(JData[index + 5:index + 5 + 13]) # rip out the milliseconds in the proper placement starting from the location of finding the match
JData_hourshift = int(JData[index + 5 + 13:index + 5 + 13 + 5])
dt = datetime.datetime.fromtimestamp(round(JData_milliseconds/1000)) # need to get rid of the tailing bits of real info due to conversion to ESRI date that round this information differently.
dtn = dt - datetime.timedelta(hours=round(JData_hourshift/100)) # not sure if I should be subtracting/adding this hour shift or what exactly this means. Only want the hour not minutes.
return dtn # may need to only convert the -04 to integer, need to verify dates against the GeoPro webpage.
except:
return
returnThe code writing the datetime element uses a regular insert cursor and converts the JSON data (Gmsg) as a python dictionary list into a matching field in the feature class based upon the feature class field type (predefined). # Create the ESRI point geometry
inPoint = arcpy.Point(SEKI_Location['Longitude'],SEKI_Location['Latitude'])
newIncident = rowInserter.newRow()
# Populate Shape Attributes
newIncident.SHAPE = inPoint
# Populate matching fields.
for field in fieldList: # loop through the Feature class attribute names
if field.name in Gmsg.keys():
print field.name
newIncident.setValue(field.name, MatchDataType(field, Gmsg))
# add attribute stuff here
rowInserter.insertRow(newIncident)
# newIncident.setValue(descriptionField, inDescription)
It can then be read lader and matched, however the times may be off by one second.
This is observed as a discripency in the print statement. Approximately one out of 10 times will be off by one seconddef IsDuplicateEntry(GeoPro_fc, ComparisonList,FieldList, GeoProMessageObject):
# SCursor=SearchCursor.reset() unsupported # reset the search cursor in the feature class for checking the next entry
Duplicates = 0
SCursor = arcpy.SearchCursor(GeoPro_fc)
try:
for row in SCursor: # Iterate through all the rows to find a match if there is one
Duplicates = 0
for ComparisonName in ComparisonList:
if ComparisonName == "EventDate":
print row.getValue(ComparisonName), ' --- ', MatchDataType(GetESRIFieldObject(ComparisonName, FieldList), GeoProMessageObject)
if not(row.getValue(ComparisonName) == MatchDataType(GetESRIFieldObject(ComparisonName, FieldList), GeoProMessageObject)):
continue # if there is any non match we don't need to keep checking
else:
Duplicates = Duplicates + 1
if Duplicates == len(ComparisonList): # if all the comparison fields have duplicates then this is a duplicate record and pass a True message
#del row
#del SCursor
return True # This record is a duplicate - no need to keep searching through the feature class.
#del row
#del Scursor
return False
except:
#del row
#del SCursor
return # If there is an unusual error just pass the duplicate found flag.
