|
POST
|
Still no Halo (should be Red) or alignment (should be middle on the vertical access) Made the halo 20 and nothing changed.
... View more
12-01-2017
01:43 PM
|
0
|
4
|
2536
|
|
POST
|
I am using API 3.22 Trying this...It works but HaloColor, HaloSize, Vertical Alignment and Horizontal Alignment dont seem to be working.. Thoughts? var labelClass = new LabelClass({
labelExpressionInfo: {"value": "{STREET_NAME}"},
symbol: {
"type": "esriTS",
"color": [225,225,0],
"font": {
"family": "Arial",
"haloColor": "#FF0000",
"haloSize": "3.25px",
"size": "8px",
"verticalAlignment": "middle",
"horizontalAlignment": "center"
}
}
});
... View more
12-01-2017
01:26 PM
|
0
|
10
|
3134
|
|
POST
|
I ran the BuildRasterAttributeTable on the Extract I did of the original landcover...this extract is now just the 5 counties I am having an issue with It ran successful
... View more
10-18-2017
10:11 AM
|
0
|
2
|
2256
|
|
POST
|
What do you mean be isolating them so they are the only ones... What I did was extract the overall LandCover raster to the boundary of the 5 counties causing issues. Then tried to tabulate area on each county individually. I simply selected the Single County and then ran the TabulateArea on the Extracted 5 County LandCover Raster.
... View more
10-18-2017
09:40 AM
|
0
|
4
|
2256
|
|
POST
|
I have a landcover raster dataset. I have a county dataset. (171 counties) I am trying to run tabulate area on each county to pull statistics. It works for ALL but 5 counties...166 counties work fine, 5 fail Error 010419: Unable to build the attribute table of .... as required by the operation Failed to Execute Tabulate Area I tried to repair geometry on the Counties I tried to Build Raster Attribute Table I build Pyramids I then reran the TabulateArea and get same error....why are only 5 failing and all others process fine? Thoughts?
... View more
10-18-2017
06:44 AM
|
0
|
6
|
2538
|
|
POST
|
OK I am trying to figure this out...something seems to have changed but I did not change ANY code. I had the first example below this since the beginning... I added "None" for the username and it appears to be running again Have to do some more checks to make sure things are working properly. def getToken(username=, password=None, portal_URL = 'https://www.arcgis.com'):
'''Gets a token from ArcGIS Online/Portal with the given username/password''' def getToken(username=None, password=None, portal_URL = 'https://www.arcgis.com'):
'''Gets a token from ArcGIS Online/Portal with the given username/password'''
... View more
09-14-2017
10:19 AM
|
0
|
0
|
1188
|
|
POST
|
I have been using the below Python script for quite sometime now. It basically grabs a feature class, attachments etc in our ArcGIS online account and pulls it down to our SDE db. For some reason it just stopped working...I am wondering if something changed in AGO that would have caused this? I dont know if parameters changed or something else.... I know its super long but hoping someone can shed some light on this. #-------------------------------------------------------------------------------
# Name: SyncSurvey
# Purpose: Initiate
#
# Author: jame6423
#
# Created: 15/08/2016
# Copyright: (c) jame6423 2016
# Licence: <your licence>
#-------------------------------------------------------------------------------
import os, tempfile, shutil
import json, re
import uuid
import datetime, time, pytz
import urllib, urllib2
import sys
import arcpy
print "Start"
arcpy.AddMessage("Start")
print "Starting"
def getToken(username=, password=None, portal_URL = 'https://www.arcgis.com'):
'''Gets a token from ArcGIS Online/Portal with the given username/password'''
arcpy.AddMessage('\t-Getting Token')
print "\t-Getting Token"
if password == None:
password = getpass.getpass()
parameters = urllib.urlencode({
'username': username,
'password': password,
'client': 'referer',
'referer': portal_URL,
'expiration': 60,
'f': 'json'
})
tokenURL = '{0}/sharing/rest/generateToken?'.format(portal_URL)
response = urllib.urlopen(tokenURL, parameters).read()
token = json.loads(response)['token']
return token
def getServiceDefinition(token, survey_URL):
'''Gets the JSON representation of the service definition.'''
response = urllib.urlopen("{0}?f=json&token={1}".format(survey_URL, token)).read()
serviceInfo = json.loads(response)
return serviceInfo
def getUTCTimestamp(timezone):
'''Returns a UTC timestamp'''
now = datetime.datetime.now()
timeZone = pytz.timezone(timezone)
localNow = timeZone.localize(now)
utcNow = localNow.astimezone(pytz.utc)
return utcNow
def createTimestampText(datetimeObj):
'''Format a datetime for insertion using Calculate Fields'''
outText = ""
timeStringFormat = "%Y-%m-%d %H:%M:%S"
outText = datetimeObj.strftime(timeStringFormat)
return outText
def getSurveyTables(workspace, prefix=''):
'''Return a list of the tables participating in the survey'''
originalWorkspace = arcpy.env.workspace
arcpy.env.workspace = workspace
#This is used in 2 contexts:
#Downloaded GDB - tables have no prefix
#Enterprise GDB - prefix is added to table name
#The full table name (i.e. GDB.SCHEMA.NAME) is returned, so prefix is in the middle
wildcard = '*{0}*'.format(prefix) if prefix != '' else '*'
#List the Feature Classes & Tables
#Tables also returns Attachment tables
featureClasses = arcpy.ListFeatureClasses(wildcard)
tables = arcpy.ListTables(wildcard)
#Loop through the tables, checking for:
#1) Is this an attachment table?
#2) Does the prefix actually match the prefix term exactly?
allTables = featureClasses
allTables.extend(tables)
outTables = []
for t in allTables:
tableName = t.split('.')[-1]
nameParts = tableName.split('_')
if '__ATTACH' not in t:
if nameParts[0] == prefix or prefix == '':
outTables.append(t)
arcpy.env.workspace = originalWorkspace
return outTables
def getLastSynchronizationTime(workspace, tableList):
'''Looks at the existing records in the SDE and returns the latest synchronization time'''
arcpy.AddMessage('\t-Checking Last Sychronization')
print "\t-Checking Last Sychronization"
arcpy.env.workspace = workspace
statTables = []
#Dummy value to compare time
lastSync = datetime.datetime.fromtimestamp(0)
for table in tableList:
#Skip if empty table (i.e., no rows)
arcpy.AddMessage('\t\t-Checking sync on {0}'.format(table))
print "\t\t-Checking sync on {0}".format(table)
#Just use the last part of the table name
tableName = table.split(".")[-1]
rowCheck = arcpy.GetCount_management(tableName)
rowCount = int(rowCheck.getOutput(0))
if rowCount > 0:
statTable = arcpy.Statistics_analysis(tableName, r'in_memory\stat_{0}'.format(tableName), "SYS_TRANSFER_DATE MAX")
statTables.append(statTable)
for s in statTables:
with arcpy.da.SearchCursor(s, ['MAX_sys_transfer_date']) as rows:
for row in rows:
thisDate = row[0]
if thisDate > lastSync:
lastSync = thisDate
#If we get no results (i.e., no tables) return None
if lastSync == datetime.datetime.fromtimestamp(0):
return None
else:
arcpy.AddMessage('\t\t-Last Synchornized on {0}'.format(createTimestampText(lastSync)))
print "\t\t-Last Synchornized on {0}'.format(createTimestampText(lastSync))"
return lastSync
def getReplica(token, serviceURL, serviceInfo, now, outDir=None, outDB="outSurvey.geodatabase", lastSync=None):
'''Downloads the full replica and then process client-side'''
# See http://resources.arcgis.com/en/help/arcgis-rest-api/#/Create_Replica/02r3000000rp000000/
arcpy.AddMessage('\t-Getting Replica')
print "\t-Getting Replica"
createReplicaURL = '{0}/createReplica/?f=json&token={1}'.format(serviceURL, token)
replicaParameters = {
"geometry": "-180,-90,180,90",
"geometryType": "esriGeometryEnvelope",
"inSR":4326,
"transportType":"esriTransportTypeUrl",
"returnAttachments":True,
"returnAttachmentsDatabyURL":False,
"async":True,
"syncModel":"none",
"dataFormat":"sqlite",
}
if serviceInfo["syncCapabilities"]["supportsAttachmentsSyncDirection"] == True:
replicaParameters["attachmentsSyncDirection"] = "bidirectional"
layerList = [str(l["id"]) for l in serviceInfo["layers"]]
tableList = [str(t["id"]) for t in serviceInfo["tables"]]
layerList.extend(tableList)
replicaParameters["layers"] = ", ".join(layerList)
layerQueries = {}
createReplReq = urllib2.urlopen(createReplicaURL, urllib.urlencode(replicaParameters))
#This is asynchronous, so we get a jobId to check periodically for completion
thisJob = json.loads(createReplReq.read())
if not "statusUrl" in thisJob:
raise Exception("invalid job: {0}".format(thisJob))
jobUrl = thisJob["statusUrl"]
resultUrl = ""
#Check for a max 1000 times (10000 sec = 2hr 46 min)
sanityCounter = 1000
while resultUrl == "":
checkReq = urllib2.urlopen("{0}?f=json&token={1}".format(jobUrl, token))
statusText = checkReq.read()
arcpy.AddMessage(statusText)
status = json.loads(statusText)
if "resultUrl" in status.keys():
resultUrl = status["resultUrl"]
if sanityCounter < 0:
raise Exception('took too long to make replica')
if status["status"] == "Failed" or status["status"] == "CompletedWithErrors":
raise Exception('Create Replica Issues: {0}'.format(status["status"]))
arcpy.AddMessage('\t\t-Check {0}: {1}'.format(str(1001-sanityCounter), status["status"]))
sanityCounter = sanityCounter - 1
time.sleep(10)
#Download the sqlite .geodatabase file
resultReq = urllib2.urlopen("{0}?token={1}".format(resultUrl, token))
if outDir == None:
outDir = tempfile.mkdtemp()
outFile = os.path.join(outDir, outDB)
with open(outFile, 'wb') as output:
output.write(resultReq.read())
del(output)
#transfer from sqlite to GDB
surveyGDB = os.path.join(outDir, 'outSurvey.gdb')
arcpy.CopyRuntimeGdbToFileGdb_conversion(outFile, surveyGDB)
del(outFile)
return surveyGDB
def filterRecords(surveyGDB, now, lastSync):
'''Filter the records to those that need to be updated'''
#Note - This excludes new entries that are *after* the timestamp
# Depending on how active the survey is, there may have been new submissions
# after the start of the script
# We put in a max time to ensure consistency in operation from run to run and
# table to table
arcpy.AddMessage('\t-Filtering records to new set')
print "\t-Filtering records to new set"
arcpy.env.workspace = surveyGDB
nowText = createTimestampText(now)
tableList = getSurveyTables(surveyGDB)
dateField = arcpy.AddFieldDelimiters(surveyGDB, "CreationDate")
excludeStatement = "CreationDate > date '{1}'".format(dateField, nowText)
if lastSync != None:
lastSyncText = createTimestampText(lastSync)
excludeStatement = excludeStatement + " OR CreationDate <= date '{0}'".format(lastSyncText)
arcpy.AddMessage('\t\t-{0}'.format(excludeStatement))
print "\t\t-{0}'.format(excludeStatement)"
i = 0
for table in tableList:
i = i + 1
thisName = 'filterView{0}'.format(str(i))
dsc = arcpy.Describe(table)
if dsc.datatype == u'FeatureClass' or dsc.datatype == u'FeatureLayer':
arcpy.AddMessage('\t\t{0}'.format('featureclass'))
print "\t\t{0}'.format('featureclass')"
arcpy.MakeFeatureLayer_management(table, thisName, excludeStatement)
arcpy.DeleteFeatures_management(thisName)
else:
arcpy.MakeTableView_management(table, thisName, excludeStatement)
arcpy.DeleteRows_management(thisName)
arcpy.Delete_management(thisName)
def addTimeStamp(surveyGDB, timestamp):
'''Disables editor tracking, adds and populates the timestamp field'''
arcpy.AddMessage('\t-Adding Syncronization Time')
print "\t-Adding Syncronization Time"
arcpy.env.workspace = surveyGDB
tableList = getSurveyTables(surveyGDB)
for table in tableList:
#Disable Editor Tracking
arcpy.DisableEditorTracking_management(table)
#Add a synchronization field
arcpy.AddField_management(table, 'SYS_TRANSFER_DATE', 'DATE')
#Set it to the timestamp
with arcpy.da.Editor(surveyGDB) as edit:
for table in tableList:
with arcpy.da.UpdateCursor(table, ['SYS_TRANSFER_DATE']) as rows:
for row in rows:
rows.updateRow([timestamp])
del(edit)
return
def addKeyFields(workspace):
'''To enable transfer of attachments with repeats, we need an additional GUID field to serve as a lookup'''
arcpy.env.workspace = workspace
dscW = arcpy.Describe(workspace)
tableList = []
relClasses = [c.name for c in dscW.children if c.datatype == u'RelationshipClass']
for child in relClasses:
dscRC = arcpy.Describe(child)
if dscRC.isAttachmentRelationship:
originTable = dscRC.originClassNames[0]
originFieldNames = [f.name for f in arcpy.ListFields(originTable)]
if 'rowid' not in originFieldNames:
arcpy.AddField_management(originTable, 'rowid', 'GUID')
with arcpy.da.Editor(workspace) as edit:
with arcpy.da.UpdateCursor(originTable, ['rowid']) as urows:
for urow in urows:
urow[0] = '{' + str(uuid.uuid4()) + '}'
urows.updateRow(urow)
del(edit)
def createTables(surveyGDB, outWorkspace, prefix):
'''Creates the doamins, tables and relationships of the survey in the target workspace'''
arcpy.AddMessage('\t-Creating Tables')
print "\t-Creating Tables"
arcpy.env.workspace = surveyGDB
allTables = getSurveyTables(surveyGDB)
dscW = arcpy.Describe(arcpy.env.workspace)
#migrate the domains
arcpy.AddMessage('\t\t-Creating Domains')
print "\t\t-Creating Domains"
for domainName in dscW.domains:
if domainName[0:3] == 'cvd':
arcpy.AddMessage('\t\t\t-'.format(domainName))
print "\t\t\t-'.format(domainName)"
tempTable = 'in_memory\{0}'.format(domainName)
domainTable = arcpy.DomainToTable_management(surveyGDB, domainName, tempTable,'CODE', 'DESC')
newDomain = arcpy.TableToDomain_management(tempTable, 'CODE', 'DESC', outWorkspace, domainName, update_option='REPLACE')
arcpy.Delete_management(tempTable)
arcpy.AddMessage("\t\t-Creating Feature Classes & Tables")
print "\t\t-Creating Feature Classes & Tables"
for table in allTables:
dsc = arcpy.Describe(table)
newTableName = "{0}_{1}".format(prefix, table)
templateTable = template=os.path.join(surveyGDB, table)
if dsc.datatype == u'FeatureClass':
newTable = arcpy.CreateFeatureclass_management(outWorkspace, newTableName, "POINT", template=templateTable, spatial_reference=dsc.spatialReference)
else:
newTable = arcpy.CreateTable_management(outWorkspace, newTableName, template=templateTable)
arcpy.AddMessage("\t\t\t-Created {0}".format(newTableName))
print "\t\t\t-Created {0}".format(newTableName)
#Attach domains to fields
tableFields = arcpy.ListFields(table)
for field in tableFields:
if field.domain != '':
arcpy.AssignDomainToField_management(newTable, field.name, field.domain)
if dscW.workspaceType == "RemoteDatabase":
arcpy.RegisterAsVersioned_management(newTable)
arcpy.AddMessage('\t\t-Creating Relationships')
print "\t\t-Creating Relationships"
#Reconnect Relationship classes, checking for attachments
CARDINALITIES = {
'OneToOne': "ONE_TO_ONE",
'OneToMany': "ONE_TO_MANY",
'ManyToMany': "MANY_TO_MANY"
}
for child in [(c.name, c.datatype) for c in dscW.children if c.datatype == u'RelationshipClass']:
dscRC = arcpy.Describe(child[0])
RCOriginTable = dscRC.originClassNames[0]
RCDestTable = dscRC.destinationClassNames[0]
newOriginTable = "{0}_{1}".format(prefix, RCOriginTable)
newOriginPath = os.path.join(outWorkspace, newOriginTable)
if dscRC.isAttachmentRelationship:
#Simple case - attachments have a dedicated tool
arcpy.EnableAttachments_management(newOriginPath)
else:
newDestTable = "{0}_{1}".format(prefix, RCDestTable)
newDestPath = os.path.join(outWorkspace, newDestTable)
newRC = os.path.join(outWorkspace, "{0}_{1}".format(prefix, child[0]))
relationshipType = "COMPOSITE" if dscRC.isComposite else "SIMPLE"
fwd_label = dscRC.forwardPathLabel if dscRC.forwardPathLabel != '' else 'Repeat'
bck_label = dscRC.backwardPathLabel if dscRC.backwardPathLabel != '' else 'Main Form'
msg_dir = dscRC.notification.upper()
cardinality = CARDINALITIES[dscRC.cardinality]
attributed = "ATTRIBUTED" if dscRC.isAttributed else "NONE"
originclassKeys = dscRC.originClassKeys
originclassKeys_dict = {}
for key in originclassKeys:
originclassKeys_dict[key[1]] = key[0]
originPrimaryKey = originclassKeys_dict[u'OriginPrimary']
originForiegnKey = originclassKeys_dict[u'OriginForeign']
arcpy.CreateRelationshipClass_management(newOriginPath, newDestPath, newRC, relationshipType, fwd_label, bck_label, msg_dir, cardinality, attributed, originPrimaryKey, originForiegnKey)
#Regular Relation
def getTablesWithAttachments(workspace, prefix):
'''Lists the tables that have attachments, so that we can seperately process the attachments during migration'''
arcpy.AddMessage('\t-Finding Attachments')
print "\t-Finding Attachments"
originalWorkspace = arcpy.env.workspace
arcpy.env.workspace = workspace
dscW = arcpy.Describe(workspace)
tableList = []
relClasses = [c.name for c in dscW.children if c.datatype == u'RelationshipClass']
for child in relClasses:
dbNameParts = child.split(".")
childParts = dbNameParts[-1].split("_")
if childParts[0] == prefix:
dscRC = arcpy.Describe(child)
if dscRC.isAttachmentRelationship:
originTable = dscRC.originClassNames[0]
originParts = originTable.split(".")
tableList.append(originParts[-1])
arcpy.env.workspace = originalWorkspace
return tableList
def createFieldMap(originTable, originFieldNames, destinationFieldNames):
'''Matches up fields between tables, even if some minor alteration (capitalization, underscores) occured during creation'''
arcpy.AddMessage('\t\t-Field Map')
print "\t\t-Field Map"
fieldMappings = arcpy.FieldMappings()
for field in originFieldNames:
if field != 'SHAPE':
thisFieldMap = arcpy.FieldMap()
thisFieldMap.addInputField(originTable, field)
if field in destinationFieldNames:
#Easy case- it came over w/o issue
outField = thisFieldMap.outputField
outField.name = field
thisFieldMap.outputField = outField
#arcpy.AddMessage("\t".join([field, field]))
else:
#Use regular expression to search case insensitve and added _ to names
candidates = [x for i, x in enumerate(destinationFieldNames) if re.search('{0}\W*'.format(field), x, re.IGNORECASE)]
if len(candidates) == 1:
outField = thisFieldMap.outputField
outField.name = candidates[0]
thisFieldMap.outputField = outField
fieldMappings.addFieldMap(thisFieldMap)
return fieldMappings
def appendTables(surveyGDB, workspace, prefix):
'''Append the records from the survey into the destination database'''
arcpy.AddMessage('\t-Adding records')
print "\t-Adding records"
arcpy.env.workspace = surveyGDB
tableList = getSurveyTables(surveyGDB)
attachmentList = getTablesWithAttachments(workspace, prefix)
for table in tableList:
#Normalize table fields to get schemas in alignmnet- enable all editing, make nonrequired
fields = arcpy.ListFields(table)
for field in fields:
if not field.editable:
field.editable = True
if field.required:
field.required = False
destinationName = "{0}_{1}".format(prefix, table)
destinationFC = os.path.join(workspace, destinationName)
arcpy.AddMessage('\t\t-{0} > {1}'.format(table, destinationName))
print "\t\t-{0} > {1}".format(table, destinationName)
#First, append the table
#Match up the fields
originFieldNames = [f.name for f in arcpy.ListFields(table)]
destFieldNames = [f.name for f in arcpy.ListFields(destinationFC)]
fieldMap = createFieldMap(table, originFieldNames, destFieldNames)
arcpy.AddMessage('\t\t-{0}'.format(table))
print "\t\t-{0}".format(table)
arcpy.Append_management(table, destinationFC, 'NO_TEST', fieldMap)
if destinationName in attachmentList:
appendAttachments(table, destinationFC)
def appendAttachments(inFC, outFC, keyField='rowid', valueField = 'globalid'):
arcpy.AddMessage('\t\t\t-Adding attachments')
print "\t\t\t-Adding attachments"
# 1) scan through both GlobalID and rowID of the old and new features and build a conversion dictionary
GUIDFields = [keyField, valueField]
inDict = {}
outDict = {}
lookup = {}
inAttachTable = "{0}__ATTACH".format(inFC)
outAttachTable = "{0}__ATTACH".format(outFC)
with arcpy.da.SearchCursor(inFC, GUIDFields) as inputSearch:
for row in inputSearch:
inDict[row[0]] = row[1]
with arcpy.da.SearchCursor(outFC, GUIDFields) as outputSearch:
for row in outputSearch:
outDict[row[0]] = row[1]
for key, inValue in inDict.iteritems():
if key not in outDict.keys():
raise Exception('missing key: {0}'.format(key))
lookup[inValue] = outDict[key]
# 2) Copy the attachment table to an in-memory layer
tempTableName = r'in_memory\AttachTemp'
tempTable = arcpy.CopyRows_management(inAttachTable, tempTableName)
# 3) update the attachment table with new GlobalIDs
with arcpy.da.UpdateCursor(tempTable, ['REL_GLOBALID']) as uRows:
for uRow in uRows:
uRow[0] = lookup[uRow[0]]
uRows.updateRow(uRow)
# 4) Append to destination attachment table
arcpy.Append_management(tempTable, outAttachTable, 'NO_TEST')
arcpy.Delete_management(tempTable)
def FAIL(sectionText, err):
#arcpy.AddMessage ('======================')
#arcpy.AddMessage ('FAIL: {0}'.format(sectionText))
#arcpy.AddMessage ('exception:')
#arcpy.AddMessage (err)
#arcpy.AddMessage (err.args)
#arcpy.AddMessage (sys.exc_info()[0])
#arcpy.AddMessage (sys.exc_info()[2].tb_lineno)
#arcpy.AddMessage ('----------------------')
#arcpy.AddMessage ('arcpy messages:')
#arcpy.AddMessage (arcpy.GetMessages(1))
#arcpy.AddMessage (arcpy.GetMessages(2))
#arcpy.AddMessage ('======================')
print "======================"
print "FAIL: {0}".format(sectionText)
print "exception:"
print "err"
print "err.args"
print "sys.exc_info()[0]"
print "sys.exc_info()[2].tb_lineno"
print "----------------------"
print "arcpy messages:"
print "arcpy.GetMessages(1)"
print "arcpy.GetMessages(2)"
print "======================"
return
def cleanup(ops, sdeConnection, prefix, now):
if 'append' in ops.keys():
arcpy.env.workspace = sdeConnection
nowTS = createTimestampText(now)
i = 0
for table in ops['append']:
i = i + 1
thisName = 'layerOrView{0}'.format(str(i))
whereStatment = "sys_transfer_date = timestamp'{0}'".format(nowTS)
dsc = arcpy.Describe(table)
selection = None
if dsc.datatype == u'FeatureClass':
arcpy.MakeFeatureLayer_management(table, thisName, whereStatment)
arcpy.DeleteFeatures_management(thisName)
else:
arcpy.MakeTableView_management(table, thisName, whereStatment)
arcpy.DeleteRows_management(thisName)
arcpy.Delete_management(view)
if 'createTables' in ops.keys():
arcpy.env.workspace = sdeConnection
tableList = getSurveyTables(sdeConnection, prefix)
for table in tableList:
arcpy.Delete_management(table)
# if 'tempdir' in ops.keys():
# shutil.rmtree(ops['tempdir'])
def ConfigSectionMap(cfg, section):
dict1 = {}
options = cfg.options(section)
for option in options:
try:
dict1[option] = cfg.get(section, option)
if dict1[option] == -1:
DebugPrint("skip: %s" % option)
except:
print("exception on %s!" % option)
dict1[option] = None
return dict1
def test(section):
import ConfigParser
os.chdir(r'C:\Users\jame6423\Documents\Projects\SDEmigration')
cfg = ConfigParser.ConfigParser()
cfg.read('test.ini')
## Polio Test
testConfig = ConfigSectionMap(cfg, section)
testConfig['sde_conn'] = os.path.join(os.path.abspath(os.curdir), testConfig['sde_conn'])
process(testConfig['sde_conn'], testConfig['prefix'], testConfig['service_url'], testConfig['username'], testConfig['password'], testConfig['timezone'])
def process(sdeConnection, prefix, featureServiceUrl, timezone, portalUrl=None, username=None, password=None):
'''Operations:
1) Query Feature Service endpoint for table names & IDs
2) Check for existing tables
3) If existing tables, get last synchronization time
4) CreateReplica a FGDB
5) Download the FGDB
6) If new, create the tables
7) Append
'''
now = getUTCTimestamp(timezone)
cleanupOperations = {}
section = 'Beginning'
try:
section = 'Logging in to Survey'
tokenTest = arcpy.GetSigninToken()
token = None
if tokenTest == None:
token = getToken(username, password, portalUrl)
else:
token = tokenTest['token']
serviceInfo = getServiceDefinition(token, featureServiceUrl)
if 'Sync' not in serviceInfo['capabilities']:
arcpy.AddError('Sync Capabilities not enabled')
raise Exception('Sync Capabilities not enabled')
section = 'Checking Existing Data'
existingTables = getSurveyTables(sdeConnection, prefix)
lastSync = None
if len(existingTables) > 0:
lastSync = getLastSynchronizationTime(sdeConnection, existingTables)
section = 'Downloading Survey'
tempdir = tempfile.mkdtemp()
#cleanupOperations['tempdir'] = tempdir
surveyGDB = getReplica(token, featureServiceUrl, serviceInfo, now, outDir=tempdir, lastSync=lastSync)
section = 'Preprocess Surveys for transfer'
filterRecords(surveyGDB, now, lastSync)
addTimeStamp(surveyGDB, now)
addKeyFields(surveyGDB)
if len(existingTables) == 0:
section = 'Making Tables'
createTables(surveyGDB, sdeConnection, prefix)
cleanupOperations['createTables'] = True
section = 'Updating Tables'
cleanupOperations['append'] = existingTables
appendTables(surveyGDB, sdeConnection, prefix)
cleanupOperations.pop('append', None)
cleanupOperations.pop('createTables', None)
except Exception as e:
FAIL(section, e)
finally:
#clean up
cleanup(cleanupOperations, sdeConnection, prefix, now)
return
def main():
sde_conn = r"C:\\Users\\xxxx\\AppData\\Roaming\\ESRI\\Desktop10.4\\ArcCatalog\\[email protected]"
prefix = r"BRs"
featureService = r"https://services2.arcgis.com//xxxxxx//arcgis//rest//services//xxxxx//xxxxx//"
timezone = r"US/Eastern"
portal = r"https://www.arcgis.com"
username = r"username"
password = r"password"
process(sde_conn, prefix, featureService, timezone, portal, username, password)
arcpy.SetParameterAsText(7, sde_conn)
#pass
if __name__ == '__main__':
main()
... View more
09-13-2017
07:49 AM
|
0
|
2
|
1538
|
|
POST
|
1. I have a raster Landcover dataset 2. I have a vector County Boundary Layer I am looking to get a table that will breakdown each county Then for each county give me the total acres of each Landcover type Forest Wetlands etc County1 56.65 123.32 County2 3432.67 4343.44 Is there a tool to do this?
... View more
09-13-2017
06:39 AM
|
0
|
1
|
1082
|
|
POST
|
THINK I GOT IT....one minute for testing I had to fix the apostrophe if string.find(varQuery, "'") <> -1:
print 'fixing apostrophe in', varQuery, 'to be queryable.'
fn_name_qry = string.replace(varQuery, "'", "''")
print fn_name_qry
else:
fn_name_qry = varQuery #Set variables for the FC above
var_SITENAME,var_WATERBODY,var_ACCESSAREA,var_BODYOFWATE = 'SITENAME' ,'WATERBODY','ACCESSAREA','BODYOFWATE'
var_TYPE,var_NO_OFRAMPS,var_COUNTY,var_REGION = 'TYPE','NO_OFRAMPS','COUNTY','REGION'
var_Lat,var_Long = 'Lat','Long'
QueryBoatRamps = [var_SITENAME,var_COUNTY,var_REGION,var_WATERBODY,var_BODYOFWATE,var_ACCESSAREA,var_TYPE,var_NO_OFRAMPS,var_Lat,var_Long]
BoatRampList=[]
# Build a query that will grab the records from the FC where the two SiteName fields are the same from the FC (BoatRampsWGS) and the LIST (myBoatRampList)
print ""
for i, val in enumerate(mynewMatchedlist):
varQuery = val
print varQuery
if string.find(varQuery, "'") <> -1:
print 'fixing apostrophe in', varQuery, 'to be queryable.'
fn_name_qry = string.replace(varQuery, "'", "''")
print fn_name_qry
else:
fn_name_qry = varQuery
#RampQry="""SITENAME = '{}'""".format(varQuery)
RampQry="""SITENAME = '{}'""".format(fn_name_qry)
#print RampQry
with arcpy.da.SearchCursor(myLayer, QueryBoatRamps, RampQry) as cursor:
for row in cursor:
varBoatRampsList = str('{0},{1},{2},{3},{4},{5},{6},{7},{8},{9}'.format(row[0],row[1],row[2],row[3],row[4],row[5],row[6],row[7],row[8],row[9]))
BoatRampList.append(varBoatRampsList)
del cursor
print BoatRampList
... View more
05-05-2017
09:59 AM
|
0
|
0
|
1374
|
|
POST
|
I am trying this but getting an error... ERROR: Traceback (most recent call last): File "E:\ArcGISProjects\BoatRampFacilities\PythonScripts\PythonSync\PythonScripts\~6_30days ago.py", line 170, in <module> for row in cursor: RuntimeError: Underlying DBMS error [[Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near 's'.] [DGIF_TEST.DBO.BoatRampsWGS] LIST: CALLED "matched" ['Airfield Pond', 'American Legion', 'Anthony Ford #4', 'Aylett', 'Blackwater', 'Blackwater Bridge', "Carey's", "Carter's Wharf", 'Coopers', 'Deep Bottom'] I think I am getting the error because the value has an apostrophe???? Carey's #Set variables for the FC above
var_SITENAME,var_WATERBODY,var_ACCESSAREA,var_BODYOFWATE = 'SITENAME' ,'WATERBODY','ACCESSAREA','BODYOFWATE'
var_TYPE,var_NO_OFRAMPS,var_COUNTY,var_REGION = 'TYPE','NO_OFRAMPS','COUNTY','REGION'
var_Lat,var_Long = 'Lat','Long'
QueryBoatRamps = [var_SITENAME,var_COUNTY,var_REGION,var_WATERBODY,var_BODYOFWATE,var_ACCESSAREA,var_TYPE,var_NO_OFRAMPS,var_Lat,var_Long]
BoatRampList=[]
# Build a query that will grab the records from the FC where the two SiteName fields are the same from the FC (BoatRampsWGS) and the LIST (myBoatRampList)
for i, val in enumerate(matched):
varQuery = val
print varQuery
RampQry="""SITENAME = '{}'""".format(varQuery)
#print RampQry
with arcpy.da.SearchCursor(myLayer, QueryBoatRamps, RampQry) as cursor:
for row in cursor:
varBoatRampsList = str('{0},{1},{2},{3},{4},{5},{6},{7},{8},{9}'.format(row[0],row[1],row[2],row[3],row[4],row[5],row[6],row[7],row[8],row[9]))
BoatRampList.append(varBoatRampsList)
del cursor
print BoatRampList
... View more
05-05-2017
09:44 AM
|
0
|
2
|
1374
|
|
POST
|
I can hard code a sitename and get a return...just don't know how to get it to read the LIST and give me a record for each value in the LIST. How do I replace my LIST with the variable "varQuery" varQuery = "Airfield Pond"
RampQry="""SITENAME = '{}'""".format(varQuery)
print RampQry
# write these records to a new list
BoatRampList=[]
with arcpy.da.SearchCursor(myLayer, QueryBoatRamps, RampQry) as cursor:
for row in cursor:
varBoatRampsList = str('{0},{1},{2},{3},{4},{5},{6},{7},{8},{9}'.format(row[0],row[1],row[2],row[3],row[4],row[5],row[6],row[7],row[8],row[9]))
BoatRampList.append(varBoatRampsList)
del cursor
print BoatRampList Result: ['Airfield Pond,Sussex,1,Airfield Pond,AIRFIELD POND,AIRFIELD POND,R,1.0,32.90782387,-80.02724715']
... View more
05-05-2017
09:23 AM
|
0
|
3
|
1374
|
|
POST
|
I have a LIST containing SiteNames (this is a LIST i created named "mynewlist") I have a FC that has a SiteName field I want to create a LIST / ARRAY of the data that is in the FC, where the List (mynewlist) and FC (Field "SiteName") are the same. For instance... I have 5 Sitenames in the LIST (# 1 above) I have 200 Sitenames in the FC (#2 above) I want a resulting LIST that is comprised of fields 1-9 from the FC where the LIST values are the same as the FC SiteName field. Think I have everything in order just need help with the actual query??? I don't even know if I can create a Query from a LIST and a FC....do I need to convert the list to something else first? arcpy.env.workspace = "C:\\Users\\xxxx\\AppData\\Roaming\ESRI\\Desktop10.4\\ArcCatalog\\[email protected]"
myFCLayer = "BoatRampsWGS"
#Set variables for the FC above
var_SITENAME,var_WATERBODY,var_ACCESSAREA,var_BODYOFWATE = 'SITENAME' ,'WATERBODY','ACCESSAREA','BODYOFWATE'
var_TYPE,var_NO_OFRAMPS,var_COUNTY,var_REGION = 'TYPE','NO_OFRAMPS','COUNTY','REGION'
var_Lat,var_Long = 'Lat','Long'
QueryBoatRamps = [var_SITENAME,var_COUNTY,var_REGION,var_WATERBODY,var_BODYOFWATE,var_ACCESSAREA,var_TYPE,var_NO_OFRAMPS,var_Lat,var_Long]
# Build a query that will grab the records from the FC where the two SiteName fields are the same from the FC (BoatRampsWGS) and the LIST (myBoatRampList)
RampQry= #Grab all the records from FC (BoatRampsWGS) where th4e Sitename is the same as the sitename in the LIST (myBoatRampList)
# write these records to a new list
BoatRampList=[]
with arcpy.da.SearchCursor(myFCLayer, QueryBoatRamps, RampQry) as cursor:
for row in cursor:
varBoatRampsList = str('{0},{1},{2},{3},{4},{5},{6},{7},{8},{9}'.format(row[0],row[1],row[2],row[3],row[4],row[5],row[6],row[7],row[8],row[9]))
BoatRampList.append(varBoatRampsList)
... View more
05-05-2017
09:13 AM
|
0
|
4
|
1555
|
|
POST
|
OK this is where I am and think I have what I need , albeit very messy...maybe someone has a more streamlined way to accomplish this. Background: I have a FC that has ALL boat Ramps in our state It will have records/boat rampsin the state I have a FC of inspections of these boat ramps this FC will not have all the boat ramps in it as not all have been inspected this FC will have duplicates boat ramp entries as its an inspection FC over time I am writing ALL the Boat Ramps to a list I am writing ALL the Inspections in the last 30 days to a list I then need to find the Unmatched between the Boat Ramps and the Unique Inspections This will give me a result of the ALL Boat Ramps that were not Inspected in the last 30 days. where the MATCHED will be the Boat Ramps that were inspected in the last 30 days where the UNMATCHED are the Boat Ramps that were NOT inspected in the last 30 days. I think this works in it current form but am sure this can be done more efficiently!!!!! Anyone have any input or thoughts on how to streamline this? ALL Boat Ramps: I create a list of ALL the Boat Ramps from the FC # Get list of all Boat Ramps
myLayer = "BoatRampsWGS"
myField = "SITENAME"
myBoatRampList = [row[0] for row in arcpy.da.SearchCursor(myLayer, myField)] Inspections: Query Inspection FC to get all the inspections in the last 30 days # Loop through List of Boat Ramps and return those that dont have inspection within 30 days
Inspected = []
dateQry="""CreationDate > '{}'""".format(month_ago)
count = 0
with arcpy.da.SearchCursor(fc, QueryFields2, dateQry) as cursor:
for row in cursor:
#print(row)
count = count + 1
#CreatevalDate = (str(row[0]),str(row[1]))
CreatevalDate = (str(row[0]))
Inspected.append(CreatevalDate)
del cursor
# List of boat ramps inspected in last 30 days
print Inspected
Convert to SET and get unmatched and Matched Boat Ramps #Convert ALL Boat Ramps to a set to get the unique values
mysetRampList = set(myBoatRampList)
#Convert Inspections from last 30 days to a set to get the unique values
mysetInspected = set(Inspected)
# Get the Matched and Unmatched values using a SET
matched = mysetRampList.intersection(mysetInspected)
unmatched = mysetRampList.symmetric_difference(mysetInspected)
# MATCHED
# convert back to list sort and print
mynewMatchedlist = list(matched)
mynewMatchedlist.sort();
print mynewMatchedlist
# UNMATCHED
# convert back to list sort and print
mynewUnmatchedlist = list(unmatched)
mynewUnmatchedlist .sort();
print mynewUnmatchedlist
... View more
05-05-2017
08:23 AM
|
0
|
0
|
739
|
|
POST
|
Line 18 is commented out right now. I don't know why I am having a hard time with this. I don't see where in the query it compares the last 30 day Boat Ramp Inspections against ALL the available Boat Ramps to determine which ones were not visited.
... View more
05-05-2017
07:53 AM
|
0
|
2
|
739
|
|
POST
|
OK...I am now getting a return....I added a count to the FOR loop and am getting 64 records. These are the 64 records that have been added in the last 30 days. There are duplicate Boat Ramp names as some have been visited multiple times. So maybe 50 or so unique Boat Ramps were entered in the last 30 days. What I need now is the opposite. I need the Ramps that were NOT visited in the last 30 days. I have the ramps that were visited in the last 30 days (this query we just got working) And I have a list of ALL the ramps (LIST "myBoatRampList" in the code above) Can I do this in one query? Or: Do I need anew query to give me a list of the Ramps that differ from the Query results LIST (number 1 above) and the ALL ramps LIST (number 2 above) QueryFields2 = [var_ramp_1,var_CreationDate]
# Get Date 30 days ago
import datetime as DT
today = DT.date.today()
month_ago = today - DT.timedelta(days=30)
print month_ago
# Get list of all Boat Ramps
myLayer = "BoatRampsWGS"
myField = "SITENAME"
myBoatRampList = [row[0] for row in arcpy.da.SearchCursor(myLayer, myField)]
print myBoatRampList
# Loop through List of Boat Ramps and return those that dont have inspection within 30 days
Inspected = []
#dateQry="""CreationDate > date{} AND CreationDate < date{}""".format(month_ago, today)
dateQry="""CreationDate > '{}'""".format(month_ago)
print dateQry
count = 0
with arcpy.da.SearchCursor(fc, QueryFields2, dateQry) as cursor:
for row in cursor:
print(row)
count = count + 1
CreatevalDate = (str(row[0]),str(row[1]))
Inspected.append(CreatevalDate)
del cursor
print count
print Inspected
... View more
05-05-2017
07:13 AM
|
0
|
4
|
2086
|
| Title | Kudos | Posted |
|---|---|---|
| 1 | 09-20-2018 11:09 AM | |
| 1 | 09-10-2018 06:26 AM | |
| 1 | 09-15-2022 11:02 AM | |
| 1 | 05-21-2021 07:35 AM | |
| 1 | 08-09-2022 12:39 PM |
| Online Status |
Offline
|
| Date Last Visited |
09-19-2022
09:23 PM
|