Latest Contributions by MarkWingfield

‎02-08-2017

Trick one to explain but I will give it a go with simplified code. I originally had a python script that had some global variables defined at the top, some functions and then the main code in a "try" statement. eg PythonScript.py import arcpy, os, sys arcpy.env.workspace = arcpy.GetParameterAsText(0) LogAllEvents = arcpy.GetParameterAsText(1) globalVariable1 = "Global1" globalVariable2 = "Global2" sdeConn = arcpy.ArcSDESQLExecute(arcpy.env.workspace) def somefunc(Global1): print Global1 def someotherfunc(Global2) global Global1 print Global2 Global1 = Global2 try: print "Starting Process" somefunc(Global1) someotherfunc(Global2) #now carry out some logic using sdeconn I now have realized I need a method of calling this python script to perform the main functionality, so have altered this as follows: PythonScript.py import arcpy, os, sys def somefunc(Global1): print Global1 def someotherfunc(Global2) global Global1 print Global2 Global1 = Global2 def main(arcpy.env.workspace, LogAllEvents ) print "Starting Process" globalVariable1 = "Global1" globalVariable2 = "Global2" sdeConn = arcpy.ArcSDESQLExecute(arcpy.env.workspace) somefunc(Global1) someotherfunc(Global2) #now carry out some logic using sdeconn if __name__ == "__main__": main(dummy,dummy) I will always call the main() from another python script so please ignore the details of the "if __name__" statement above. My call from the other script will be: OtherScript.py import arcpy, os, sys, PythonScript arcpy.env.workspace = arcpy.GetParameterAsText(0) LogAllEvents = "Y" PythonScript.main(arcpy.env.workspace, LogAllEvents) As you can see, in PythonScript, I moved all global variables into the function main(). I had to do this as it was a function that I would be calling directly from another script. Could I have left these at the top of the script and accessed in main() by defining them as global? I understand these are accessible when it is the default body of code being called but was not sure calling a function from another script. I have simplified my scripts above. In real life there are dozens of globals and I need to get my head around this before I start shifting variables wholesale into main().

‎11-30-2016

... I am using ArcGIS 10.2

‎11-30-2016

We have a system that currently has a mosaic dataset in SQL Server 2008 R2 that is accessing a folder of the full UK TIF files for 10K Rasters. These TIF files are now outdated and so we have been asked to replace the current TIF raster files with a full UK copy of OS Vector MAP TIF files. It appears that the filenames and details are identical, it is just that the actual maps in the rasters are more up-to-date. Everything else appears to be the same format. I will be required to quote for the work and this assumes I understand what is the simplest method. Currently, I just have instructions for creating the mosaic dataset in SQL (using ArcMAP) from scratch. My question is, if I simply replace entirely the existing folders with the new folders of data in Windows and, assuming these are identical in structure, down to folder and file names, will the mosaic dataset simply pick up these new rasters without any other work or do I need to carry out some extra actions to ensure the mosaic dataset uses the new files?

‎09-23-2016

I have been tasked with getting spatial information off an ArcView SQL Server database (sde) onto an ArcMap database. This will involve creating a shape file on the ArcView tool. It is a couple of years since I used ArcView and I recall that you have to enter a connection string, that I believe consists of 4 elements and has at least the database server name and the text "sde" in it somewhere. Can anyone recall this and the format required for this text? Also, if there are any instructions regarding general use of ArcView (so I can remind myself of anything I forget), I would be grateful to know where to find this.

‎10-29-2015

I have worked with such languages as Python when dealing with ESRI geometries. However, I have now been asked to amend some C# to add a group of simple line geometries (straight lines) to a map, sourced from a soatial SQL table. We only seem to have ArcGIS Client available and I don't know where to start. I will try and explain the issue as carefully as I can. We have a SQL Server spatial table called DP_Accesses that, in simple terms, has a unique key and a geometry of LineString e.g. APID - char(18) Shape - geometry of LineString This is itself a huge table with about 30 million rows I have managed to build a list of APID values in C# that I want to use to select rows from this table and add to my map. However, please note that this could be a list of anything from 1 to about 7000 APID values, so in itself it is not a small list As I am at a point where I have completed this list of APID values, how do I create the layer/spatial collection that can be added to the map, using the ARCGIS Client functionality in C#? I am on .Net Framework 4 and using ARCGIS 10.2 Client. Cheers Mark

‎09-16-2014

I have an Arcpy script that is having issues with escape backslashes in a folder name when passed in from a C# Process command and this does not happen in command prompt. I run both on the same server so would expect both to work the same when opassed in the same data. The code that is behaving differently is below (apolgies in advance as I cannot find the command to embed source code in a nice way): message = "[INFO] GGF - Output_dir initially is " + output_dir + " at " + str(datetime.datetime.now()) #print message arcpy.AddMessage(message) win32evtlogutil.ReportEvent(appName,0,0,win32evtlog.EVENTLOG_INFORMATION_TYPE,[message]) ## workaround to deal with C# putting in extra \ in folder names output_dir_frm = output_dir.replace("\\\\", "\\"); message = "[INFO] GGF - Output_dir formatted is " + output_dir_frm + " at " + str(datetime.datetime.now()) #print message arcpy.AddMessage(message) win32evtlogutil.ReportEvent(appName,0,0,win32evtlog.EVENTLOG_INFORMATION_TYPE,[message]) As you can see, the statement in question is in bold. When I call from command prompt, the 3rd argument is the one I am trying to fix: "D:\\TFS\\Pegasus_MWingfield\GenerateGeorouteFilesNetwork.py" "C:\\Users\\mark.r.wingfield\\AppData\\Roaming\\ESRI\\Desktop10.2\\ArcCatalog\ \DWOP.SDE" 3903 "\\\\RMGAPPODC047V\\PegasusTier1DevData$\\alstalba\\Universal\\" This is simply to mimic what is happening in the C# call. In C#, I am using the 32 bit version of Python.exe (the 64 bit doesn't work at all) and the 3rd argument is as follows: "\\\\RMGAPPODC047V\\PegasusTier1DevData$\\alstalba\\Universal\\" Looks the same so far. When running the command prompt version, the message values created are as follows: [INFO] GGF - Output_dir initially is \\\\RMGAPPODC047V\\PegasusTier1DevData$\\alstalba\\Universal\ at 2014-09-16 14:40:32.242000 [INFO] GGF - Output_dir formatted is \\RMGAPPODC047V\PegasusTier1DevData$\alstalba\Universal\ at 2014-09-16 14:40:32.244000 This is fine as it correctly removes escape values in every occurrence. When running from a C# Process command: [INFO] GGF - Output_dir initially is \\\\RMGAPPODC047V\\PegasusTier1DevData$\\alstalba\\Universal\" at 2014-09-16 16:03:11.289000 [INFO] GGF - Output_dir formatted is \\RMGAPPODC047V\\PegasusTier1DevData$\\alstalba\\Universal\" at 2014-09-16 16:03:11.290000 So, using exactly the same python script (I have no other!!!), this time it seems to treat the "\\\\" as physical backslashes and so only replaces that first set of 4 backslashes. This then means the folder format is invalid and my process fails when trying to use this folder name. It is cleard from the output that the string to be formtted is identical in each case but the result of the arcpy replace is different. Is there any means of printing some information from within the script that will show what version of python.exe is being run as this seems to be the only explanation I can think of. Once again, there is only one version of my script and I can guarantee that this is the version called from the C# (indeed, when it was failing and i needed to identfiy the issue, I put the command in the one script, along with the before and after message, and reran from both environments, thus proving it is the one script.

‎08-19-2014

I have a process that creates a polyline shape one at a time. For each polyline, depending on associated keys, I need to update the shape on either table A matching on the keys or table B matching on the keys. ie I have the polyline if key 1 and key 2 are populated update the shape on a row on table A matching on key1, key2, plus 2 other keys (key3 and key4) else update the shape on a row on table B matching on key3 and key4 The issue is I cannot work out whether it is best to get layers of table A and B up front (many thousand of rows) and somehow filter before updating or should I get these one at a time. Also, what would the actual shape update command be as I have not yet written code to update a shape on an existing SQL server table row? There may be many polyline shapes to process. I realise this is vague but any ideas about how I should do this would be appreciated. It needs to be as efficient as possible. Maybe someone has already done something similar and has code examples.

‎06-16-2014

I have had issues whereby a SelectLayerByAttribute CLEAR_SELECTION and NEW_SELECTION is producing "The value cannot be a feature class" messages. I googled this and found that I should make my class a layer either by using MakeFeatureLayer or MakeTableView for the layer to be selected. I tried both these but still get the same message. Any help would be appreciated. My code is below: import arcpy, sys, math, os, datetime, traceback import arcpy.mapping as map from arcpy import da, env XYPointsLayer = layerMemory + os.sep + "XYPointsLayer" XYPointsLayerIntersect = layerMemory + os.sep + "XYPointsLayerIntersect" #arcpy.MakeXYEventLayer_management(WorkLayerName, viewName + ".GRID_X", viewName + ".GRID_Y", XYPointsLayer) arcpy.MakeXYEventLayer_management(fullviewName, "GRID_X", "GRID_Y", XYPointsLayer) #### Note looks like using the view directly may work instead of the MakeQueryTable option arcpy.MakeTableView_management(XYPointsLayer, XYPointsLayerIntersect) #arcpy.MakeFeatureLayer_management(XYPointsLayer, XYPointsLayerIntersect) viewName = "DWOP.DBO.VWO_MDPAG_DPA_WORK" fullviewName = arcpy.env.workspace + os.sep + viewName arcpy.SelectLayerByAttribute_management(XYPointsLayerIntersect,"CLEAR_SELECTION") arcpy.SelectLayerByAttribute_management(XYPointsLayerIntersect,"NEW_SELECTION","TIMESTAMP0 = '20140613 08:59:58.063' and FK_ROADLINK_TOID = 'PEGASUS005108964' and AP_ID_ARRAY1 = '141863003'") The SelectLayerByAttribute is within an "for" statement in the original code so i am wondering if this may be an issue? Also, the "NEW_SELECTION" criteria are in a variable in the original code but I have traced this and cut and pasted these in just to show the criteria used. However, it fails on the CLEAR_SELECTION first. The issue seems to be the basic interpretation of what XYPointsLayerIntersect actually is. Any ideas?

‎06-16-2014

Found a way around my issue that no longer requires this so please ignore

‎06-14-2014

I am creating a 200m buffered rectangle around a selected feature layer of points. Sometimes tere are many points, in which case I appear to create a correct rectangle. However, if there is only one point, the buffer does not appear to create a rectangle. I think this may be because the polygon array I create has the same co-ordinate for lowerleft/right etc so the initial shape the buffer is based on may not be a valid rectangle. My code is below: AddressPointsCopy = AddressPointsLayerName + str(rowcount) arcpy.CopyFeatures_management(AddressPointsLayerName, AddressPointsCopy) # A good idea to limit the number of RoadLinks to a sensible area around the address points # Will stick to 200 metres just to ensure we don't miss anything desc = arcpy.Describe(AddressPointsCopy).extent array = arcpy.Array() # Create the bounding box array.add(desc.lowerLeft) array.add(desc.lowerRight) array.add(desc.upperRight) array.add(desc.upperLeft) # ensure the polygon is closed array.add(desc.lowerLeft) # Create the polygon object message = "ARRAY " + str(array[0]) + " " + str(array[1]) + " " +str(array[2]) + " " +str(array[3]) + " " + str(array[4]) print message polygon = arcpy.Polygon(array) array.removeAll() # buffer extentPoly = layerMemory + os.sep + "RoadlinkBuffer" + str(rowcount) arcpy.Buffer_analysis(polygon, extentPoly, "200 meters", "OUTSIDE_ONLY", "FLAT", "NONE") How can I tailor this to deal with a single point and create a rectangle buffered by 200m in this case?

‎05-30-2014

Thanks everybody. The data dictionary method was superfast and has definitely solved a very thorny problem. I did use makequerytable first to get the data into a layer i could interrogate in testing and then used this to populate the data dictionary. All works very well Cheers

‎05-29-2014

Thanks all for your help so far. I am going with the data dictionary option and am currently working through this. The issue I do have is with debugging, in that, when running my code, it suggests that the data dictionary (I have called this rdlkDict) has been created successfully (based on using a MakeQueryTable step that calls a view returning SQL data to be matched). The data in the MakeQueryTable layer looks to be fully populated when I open it in ArcMap. However, the match does not appear to have updated anything so i need to check contents of the data dictionary. My code is below: message = "[INFO] Create table of general GGF data started at " + str(datetime.datetime.now()) print message viewName = "DWOP.DBO.VWO_GenerateGeorouteFilesNonSpatial" fullViewName = arcpy.env.workspace + os.sep + viewName tableLayerName = "GGF_Data" + OfficeID fieldList = [[viewName + ".OBJECTID"],[viewName + ".TOID"],[viewName + ".OFFICE_ID"],[viewName + ".SEG_ID"],[viewName + ".STL_NAME"],[viewName + ".STR_NAME"],[viewName + ".ST_CLASS"],[viewName + ".ORIG_LVL"],[viewName + ".DEST_LVL"],[viewName + ".ONEWAY"],[viewName + ".SPEED1"],[viewName + ".SPEED2"],[viewName + ".SPEED3"],[viewName + ".SPEED4"],[viewName + ".TRN_MODE"],[viewName + ".BTH_MODE"],[viewName + ".RESTR_ORI"],[viewName + ".RESTR_DEST"]] whereClause = "OFFICE_ID = " + OfficeID arcpy.MakeQueryTable_management(fullViewName,tableLayerName,"USE_KEY_FIELDS",viewName + ".OBJECTID",fieldList,whereClause) message = "[INFO] Create Data Dictionary of general GGF data started at " + str(datetime.datetime.now()) print message mySqlFieldsToReturnList = ["OBJECTID",viewName + ".TOID",viewName + ".OFFICE_ID",viewName + ".SEG_ID",viewName + ".STL_NAME",viewName + ".STR_NAME",viewName + ".ST_CLASS",viewName + ".ORIG_LVL",viewName + ".DEST_LVL",viewName + ".ONEWAY",viewName + ".SPEED1",viewName + ".SPEED2",viewName + ".SPEED3",viewName + ".SPEED4",viewName + ".TRN_MODE",viewName + ".BTH_MODE",viewName + ".RESTR_ORI",viewName + ".RESTR_DEST"] rdlkDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(tableLayerName, mySqlFieldsToReturnList)} #Would like to check the rdlkDict values here message = "[INFO] Looping through selected RoadNetwork started at " + str(datetime.datetime.now()) print message updateFieldsList = ["OBJECTID","SEG_ID","STL_NAME","STR_NAME","ST_CLASS", "ORIG_LVL","DEST_LVL","ONEWAY","SPEED1","SPEED2","SPEED3","SPEED4","TRN_MODE","BTH_MODE","RESTR_ORI","RESTR_DEST"] with arcpy.da.UpdateCursor(FinalNetworkLayerName, updateFieldsList) as updateRows: for updateRow in updateRows: ObjectIDVal = updateRow[0] currentRow = currentRow + 1 if currentRow%100 == 0: message = "[INFO] Processed " + str(currentRow) + " of " + str(RoadNetworkCount) + " rows at " + str(datetime.datetime.now()) print message if ObjectIDVal in rdlkDict: updateRow[1] = rdlkDict[ObjectIDVal][3] updateRow[2] = rdlkDict[ObjectIDVal][4] updateRow[3] = rdlkDict[ObjectIDVal][5] updateRow[4] = rdlkDict[ObjectIDVal][6] updateRow[5] = rdlkDict[ObjectIDVal][7] updateRow[6] = rdlkDict[ObjectIDVal][8] updateRow[7] = rdlkDict[ObjectIDVal][9] updateRow[8] = rdlkDict[ObjectIDVal][10] updateRow[9] = rdlkDict[ObjectIDVal][11] updateRow[10] = rdlkDict[ObjectIDVal][12] updateRow[11] = rdlkDict[ObjectIDVal][13] updateRow[12] = rdlkDict[ObjectIDVal][14] updateRow[13] = rdlkDict[ObjectIDVal][15] updateRow[14] = rdlkDict[ObjectIDVal][16] updateRow[15] = rdlkDict[ObjectIDVal][17] updateRows.updateRow(updateRow) How can I interrogate values in rdlkDict and display each row of data so I can check whether the data dictionary is correctly populated?

‎05-27-2014

Given that there was mention of using a list of objectIDs from the FinalNetworkLayerName feature layer, it would be possible for the function to be part of a SQL Select statement and the objectids passed into this as a Where clause. And then I guess this can be called to create a data dictionary but still unsure about the code. If this was the solution (and given that I would need to deal with chunks of data as I don't believe SQL Server 2008 can deal with over 60000 entries in a single WHERE.....IN (....)... statement), how would I code: 1. Create a data Dictionary for the first chunk of objectIDs from FinalNetworkLayerName, passing these to the WHERE clause of a SQL statement 2. Update the same Data Dictionary for x chunks of objectIDs, also passing to a WHERE clause, until the end of FinalNetworkLayerName 3. Then update FinalNetworkLayerName by matching to the objectIDs in the Data Dictionary Cheers Mark

‎05-27-2014

I think I need to build the data dictionary first based on the list of rows already in the feature layer FinalNetworkLayerName as this restricts the number of row to be processed to between 3000 and 64000, whereas, in total, there are millions of potential rows to be processed. This still looks to me like I need to build the data dictionary row by row, which seems to me will probably result in the same amount of time taken if that time is taken up by mostly the SQL call istelf. However, if it is the update of the feature layer that is the majority of the 2 seconds taken, then there may be a saving, assuming that the building of the data dictionary is quick. That would mean attempting to build the data dictionary row by row using the list of all objectIDs from FinalNetworkLayerName. So is there efficient code for this process? Cheers Mark Wingfield

‎05-23-2014

I think I get the giste of what you are all saying but I am brand new to Pythons and am still struggling. Given the code I had before and the fact that a slightly altered version of the SQL function (hopefully to fit in with a data dictionary solution) returns one row of data in the following format with the OBJECTID for the current row as an input: SEG_ID int, STL_NAME varchar(100), STR_NAME varchar(100), ST_CLASS char(1), ORIG_LVL int, DEST_LVL int, ONEWAY int, SPEED1 float, SPEED2 float, SPEED3 float, SPEED4 float, TRN_MODE char(8), BTH_MODE char(8), RESTR_ORI varchar(2000), RESTR_DEST varchar(2000), TOID varchar(16), OBJECTID id My current code using da.UpdateCursor instead of the standard UpdateCursor is: fields = ("SEG_ID", "STL_NAME", "STR_NAME", "ST_CLASS", "ORIG_LVL", "DEST_LVL", "ONEWAY", "SPEED1", "SPEED2", "SPEED3", "SPEED4", "TRN_MODE", "BTH_MODE", "RESTR_ORI", "RESTR_DEST", "TOID", "OBJECTID") cursor = arcpy.da.UpdateCursor(FinalNetworkLayerName, fields) for featurerow in cursor: CurrentObjectID = featurerow[17] # Establishing connection to MSSQL database sql = "SELECT * FROM [DWOP].[dbo].[ufunc_ReturnGGFDataObjectID] (" + CurrentObjectId + "," + OfficeIDParm # Call function sde_return = sdeConn2.execute(sql) if isinstance(sde_return, list): for row in sde_return: featurerow[0] = row[0] featurerow[1] = row[1] featurerow[2] = row[2] featurerow[3] = row[3] featurerow[4] = row[4] featurerow[5] = row[5] featurerow[6] = row[6] featurerow[7] = row[7] featurerow[8] = row[8] featurerow[9] = row[9] featurerow[10] = row[10] featurerow[11] = row[11] featurerow[12] = row[12] featurerow[13] = row[13] featurerow[14] = row[14] cursor.updateRow(featurerow) del cursor, featurerow I am looking at a data dictionary definition of : rdlkDict = dict([(r.OBJECTID, (r.SEG_ID, r.STL_NAME, r.STR_NAME, r.ST_CLASS, r.ORIG_LVL, r.DEST_LVL, r.ONEWAY, r.SPEED1, r.SPEED2, r.SPEED3, r.SPEED4, r.TRN_MODE, r.BTH_MODE, r.RESTR_ORI, r.RESTR_DEST, r.TOID)) for r in arcpy.SearchCursor(FinalNetworkLayerName)]) How would I populate this from the SQl call and then how do I link this to the FinalNetworkLayerName feature class? I cannot get my head around how the loops work and relate to each other to update 15 columns from the SQL call via the data dictionary. Sorry, I am completely new to Python. Cheers

Online Status	Offline
Date Last Visited	‎11-11-2020 02:23 AM

My Ideas

Latest Contributions by MarkWingfield

Can I access global variables in called python?

Re: Mosaic dataset TIF file replacement

Mosaic dataset TIF file replacement

What connection string do you use in old ArcView

C# Arcgis client - how to create spatial set to add to Map objects

Arcpy script behaves differently in command prompt and C#

Most efficient way of updating shape on SQL server table

SelectLayerByAttribute issue

Re: Creating buffered rectangle for single point

Creating buffered rectangle for single point

Re: Updating a feature layer with a global sql update

Re: Updating a feature layer with a global sql update

Re: Updating a feature layer with a global sql update

Re: Updating a feature layer with a global sql update

Re: Updating a feature layer with a global sql update

Updating a feature layer with a global sql update