topic Re: Where can I go to find someone to review a lengthy python script in Python Questions

Where can I go to find someone to review a lengthy python script

BrendaJameson — Thu, 11 Dec 2014 04:34:21 GMT

I would like to rewrite a script that is executed in ArcMap as a standalone script. I was told it cannot be done as some of the commands are only available in ArcMap. I am fairly new to both languages and am at a loss as to where to go from here. I was hoping there was a place or person who does this kind of thing for a very nominal fee (I am on a very tight budget-my own personal expense, not a company's). I would not need it to be rewritten but pointers would help.I would like to send them the script and have them assess the effort and possibly give me a few pointers to get me going in the right direction.I am sorry if this is not the right forum. I just don't know where else to get the help.

To facilitate this as a discussion, perhaps the old python/GIS gurus could let us newbies know of the different forums out there and the purposes of each. Maybe there is a place where seasoned programmers are available to offer this kind of assistance.

Thanks in advance.

Re: Where can I go to find someone to review a lengthy python script

OwenEarley — Thu, 11 Dec 2014 04:53:49 GMT

I would like to rewrite a script that is executed in ArcMap as a standalone script. I was told it cannot be done as some of the commands are only available in ArcMap.

Do you intend on running the script as a standalone script on a machine that has ArcGIS installed? - if so this is possible as you can access ArcGIS functions via the arcpy package.

If you want to run the script outside of ArcGIS then there are likely issues - most notably ESRI licensing restrictions.

Re: Where can I go to find someone to review a lengthy python script

NeilAyres — Thu, 11 Dec 2014 06:07:57 GMT

"I was told it cannot be done as some of the commands are only available in ArcMap."

Just about everything is available through python / arcpy. So who is telling you this? Sure if you want to write customized interfaces, web stuff etc, then you need .Net, ArcObjects and all that.

Tell us what you want to do, what progress you have made so far etc. There is lots of arcpy / python snippets, help out there.

I am sure that the python fundies here or at GIS-SE will help with particular questions.

Re: Where can I go to find someone to review a lengthy python script

DanPatterson_Retired — Thu, 11 Dec 2014 07:53:54 GMT

What version of ArcMap are you using? This is quite important.

Lengthy is relative... how many lines of code are you talking about.

Are you using any non-standard python libraries? (simply copy and past any "import blah blah" lines here)

There is always someone around that is bored and/or in full procrastinating mode so ...

Re: Where can I go to find someone to review a lengthy python script

BrendaJameson — Thu, 11 Dec 2014 18:48:23 GMT

The script is very lengthy, over 700 lines of code and performs several analyses using several modules including arcpy, zipfile, win32com.client, glob, shutil, os.path, and OrderedDict from collections . It will be run on computers that have ArcGIS 10.2.1 advanced licenses.

I pared it down to just the commands I use and a few of the commands in the more complex functions (see below). Right now the script runs in immediate mode but I would like to be able to run it as a stand-alone script. My main concerns are the commands that manipulate the TOC, legend properties, and those exporting these as pdf files.Note: Incomplete code. This is just a sampling of the commands that may not run as stand-alone.

#Arcpy Commmands that are used in script

arcpy.Buffer_analysis

arcpy.CopyRows_management

arcpy.CreateFileGDB_management

arcpy.da.SearchCursor

append(row[0])

arcpy.DeleteRows_management

arcpy.Delete_managementarcpy.Delete_management

arcpy.Describe(CompanySite).shapeType

arcpy.Dissolve_management

arcpy.Frequency_analysis

arcpy.GetParameterAsText(0)

arcpy.ListFields

arcpy.MakeFeatureLayer_management

arcpy.MakeTableView_management

arcpy.mapping.ListLayers(mxd, "Company Locations")[0]

Lyr.visible = True

arcpy.mapping.ListLayoutElements

scaleBar.elementPositionX

arcpy.Polygon(arcpy.Array([df.extent.lowerLeft, df.extent.lowerRight, df.extent.upperRight, df.extent.upperLeft]), df.spatialReference)

arcpy.RefreshActiveView()

arcpy.SelectLayerByLocation_management

arcpy.Sort_management

arcpy.Statistics_analysis

arcpy.TableToDBASE_conversion

arcpy.TabulateIntersection_analysis

arcpy.env.overwriteOutput

arcpy.mapping.ExportToPDF

arcpy.mapping.Layer

arcpy.mapping.ListLayers

arcpy.mapping.RemoveLayer(df, lyr)

int(arcpy.GetCount_management)

# Setting up group layer and adding shapefiles to it. Setting order, renaming shapefiles, and visibility in

# TOC also happens below to export series of pdf maps

arcpy.mapping.Layer(CompanySite)

CompanySite_lyr.name = "Company Location"

arcpy.mapping.UpdateLayer(df,CompanySite_lyr, sourceLayer)

groupLayer = arcpy.mapping.Layer

groupLayer.name = "Company Locations"

arcpy.mapping.AddLayer

arcpy.mapping.AddLayerToGroup

df.extent

df.scale

title = arcpy.mapping.ListLayoutElements(mxd, "TEXT_ELEMENT", "MAPTITLE")[0]

title.text = CompanySite + "\n" + mapFile

mxd = arcpy.mapping.MapDocument("CURRENT")

df = arcpy.mapping.ListDataFrames(mxd, "Layers")[0]

# Other non arcpy commands used in script

os.path.splitext

while os.path.exists

shutil.rmtree

os.makedirs

# this is run in a function after buffer analysis results

run_excel_macros

# opens dbf output in excel and formats excel files to import into word document.

xlApp = win32com.client.DispatchEx('Excel.Application')

xlsPath = os.path.expanduser

wb = xlApp.Workbooks.Open(Filename=xlsPath)

wb.Application.DisplayAlerts = False

xlApp.Run

wb.Close

xlApp.Quit()

xlApp = None

zipfile.ZipFile("D:\\Templates\\wordTemplate.docx")

report = zipfile.ZipFile(myDir + "\\Report.docx", "a")

# This section takes the variables derived from analyses and modified text based on results and

# replaces it with variables imbedded in Word template: Commands used to do this are as follows:

with open(template.extract("word/document.xml", myDir + "\\")) as tempXmlFile:

tempXml = tempXmlFile.read()

for key in replaceText.keys():

tempXml = tempXml.replace(str(key), str(replaceText.get(key)))

with open(myDir + "\\temp.xml", "w+") as tempXmlFile:

tempXmlFile.write(tempXml)

for file in template.filelist:

if not file.filename == "word/document.xml":

report.writestr(file.filename, template.read(file))

report.write(rmyDir + "\\temp.xml", "word/document.xml")

# Runs word macro to format Word document and import excel tables

word = win32com.client.Dispatch('Word.Application')

doc = word.Documents.Open(myDir + "\\Report.docx")

word.Run("formattingMacros")

Any and all guidance greatly appreciated.

Re: Where can I go to find someone to review a lengthy python script

DanPatterson_Retired — Thu, 11 Dec 2014 19:50:49 GMT

Brenda
It would be better to list the functions in chronological order to see where the break where the arcpy.mapping begins and ends and all the excel word zippy thing fits in.
Many of the functions like buffering don't need to have Arcmap open at all and as long as you maintain a list of the things created you can call upon them for future work.

When it comes to arcpy.mapping you have to be careful since you are probably working with the Current mxd and the Current dataframe etc etc. This can largely be circumvented by creating an mxd on the fly by having a template project residing in the same or easily locatable folder relative to the location of the running script (remember sys.argv[0] is the name of the running script from which you can parse off the path and append an existing mxd name to it ). Your template may be constructed anyway you want to facilitate and minimize the code. For instance the whole block of functionality after this line...

# TOC also happens below to export series of pdf maps

could be minimized if a template mxd was created and all you would have to do is add data to it.

If you must run the whole thing at once, then you may have to put in time delays (time.sleep) to ensure that outputs are created before they become inputs to other events. In short...it seems like you have a handle on things. You can get really fancy and break up your script into several scripts which are then imported into a main script and information is passed back and forth between them as needed. For instance all the excel word zip stuff I would have as a separate import module, in that way I can test it separately and completely ignore it if I want.

As a practical example...I wrote a Crosstabulation/Kappa Statistics toolbox which has a main script called:

CrossTabulateStatistics.py - called by the *.tbx

in there I use system and other imports

#Import the basic modules
import os, sys, string

import arcpy
import crosstab as CT

crosstab is a script that I wrote to compartmentalize many functions. It has an import section as well

import math

import Distributions as d

import copy

Distributions is another script I wrote that cobbled together various distribution functions over the web.

so when the main script runs from the toolbox, all the previous modules (aka scripts) are used.

I also produced another script for the toolbox called PrepareData.py which does all the data preparation ahead of time. I could have incorporated everything into one script to run as a standalone script but I would have had to rename the whole thing Magic-O-rama

Re: Where can I go to find someone to review a lengthy python script

BrendaJameson — Mon, 15 Dec 2014 18:27:06 GMT

I do have a template which I rename and run the script from. I realize that the geoprocessing commands such as buffering can be run as a stand alone but am under the impression that some of the actions performed with arcpy.mapping cannot (such as manipulating the layer properties in the TOC to export as pdf files, moving elements around in layout based upon map extents, turning on and off layer visibilities, etc.)

I do not have the authority to post the script online but I am think I can send to one or people to review and let me know the basic scope of work required to make this a stand alone (or even a more 'tight' script) which is why in my original post I asked if there was some place I could go (or someone who was willing to help) to get this assessment done. Unfortunately the cost of getting the assessment done is on me and I am a lowly, underpaid employee (as I am sure most of us are these days.)

Re: Where can I go to find someone to review a lengthy python script

DanPatterson_Retired — Mon, 15 Dec 2014 19:19:36 GMT

Brenda... Sorry I know of no such place ... hope someone can pick it up as project ... I have to finish term marking and prep work for courses in January... best of luck

Re: Where can I go to find someone to review a lengthy python script

XanderBakker — Tue, 16 Dec 2014 03:21:10 GMT

The only thing I see from the listed command that won't work as a stand-alone script is the line:

mxd = arcpy.mapping.MapDocument("CURRENT")

The CURRENT keyword is not valid in a standalone script. Replacing this with a path to the MXD you want to you should not be a problem.

If someone is going to revise the script, you should provide this person the data used in the process, the MXD, the Excel and Word templates including the macro's, etc.

Re: Where can I go to find someone to review a lengthy python script

XanderBakker — Tue, 16 Dec 2014 12:11:01 GMT

I have a burning question... why do you want to convert the script to a stand alone script? What are you trying to gain from this? Is it a something you want to execute as a scheduled task?

Re: Where can I go to find someone to review a lengthy python script

BrendaJameson — Thu, 18 Dec 2014 23:03:24 GMT

Not sure I have a good answer for that other than it was requested and I thought it would be a good learning experience. I do like the idea of compartmentalizing sections of the script and calling them as needed. It would make testing the script faster when the code in question is towards the end of the process. Dan Patterson gave me some great suggestions. Now I just have to hunt down some sample scripts to help guide me along.

Re: Where can I go to find someone to review a lengthy python script

XanderBakker — Thu, 18 Dec 2014 23:48:05 GMT

Maybe this post can help you in some way: Some Python Snippets

If you have more specific problems you run into, just post them, and there will be loads of people eager to help you (for free),.. that's what the community is all about.

Kind regards, Xander