Latest Contributions by RobertStevens

‎09-02-2015

Over the last several weeks I have made several postings to Geonet about specific issues. I think all but one remain unresolved, though not through any fault of those responders who have tried to help with the experience they have (thank you all). My overall assessment of this software (I am talking 10.3.1 Basic, but previous 10.1+ versions exhibit these problems), is that the product is a trying experience for anyone not blessed with the patience of Job. 1. It is geologically slow. I have a the latest MacBook Pro, 16G solid state memory, run Windows under Parallels. I allocate 8G to windows, and ARcGIS can use only 4G. Windows7 boots into this hard/software environment within 5 seconds. My not exceptionally complex maps take about a minute, and frequently much longer to display on ArcGIS invocation. 2. The maps are endless redrawing, with attendant delay, for no apparent reason. and, contrariwise, I often see "redrawing Cancelled" also for no apparent reason. 3. Two maps of mine became corrupted for no obvious reason -- they had been displayed, and so reports generated about them with python scripts, but no saving and/or programmatic change had been requested. 4. The software seems to become slower the longer it remains up. 5. Basemap layers don't display correctly after extended uptime (see details in another post). 6. The python interface has at least one serious bug -- python dies or hangs. (see details in another post) Python scripts take an absolute eternity to run. 7. The program sometimes hangs, becoming completely unresponsive, and has to be restarted. 8. Certain spatial joins fail completely (see details in another post) 9. Every time (and I do mean every 10.0=>10.1,10.1=>10.2, 10.2=>10.3.1) I upgrade this software, I run into some problem or other, whether benign or not is difficult to say. Most recently, with 10.3.1, I get a cryptic message about "Tom Sawyer Software" being unable to register a DLL. That last one seems to be benign, and was investigated by ESRI with no answer being forthcoming. 10. Many of the python based tools have interfaces which are not well designed. For instance, the spatial join tool allows one to discard certain feature from a target layer in the join, but these have to be deleted one by one, which is unnecessarily time consuming. Some of the tools display existing layers in a dropdown list, but a bug with the BA analyst means that the list scrolls to the lowest layer in the map, when it is far more likely that the layer of interest is atvthe top of the list (reported and confirmed as a bug more than one year ago; no fix in sight) 11. And one more thing that I forgot in my list above (added on edit). Why is the software always asked me whether I want to save a map that has not changed? Just displayed; no changes of any kind made. But still asks me on exit whether I wish to save it. 12. The arcpy api is still insufficient.it does not allow one to identify dataset joins. It does not allow editing of field aliases. And the whole design of it seems confusing and contorted. My overall impression of this software is that features have been piled into it without sufficient regard for software bloat and performance, and without sufficient testing on various software environments which supposedly are supported. Our small company (NSW corp, private) has spent something like 35K on this software over three years for single use, desktop only, licenses. I think we should expect better than this. ESRI support is always polite, and often quite helpful on smaller issues (often with workarounds), but these big ticket performance problems and bugs are just wasting my time and preventing us from bringing to fruition the analysis which GIS is intended to help us solve. In the last three weeks or so I think I have spent more than half my time seeking help on Geonet (thanks to all for help/advice received), and that is just too much. Rob Stevens NSW Corp.

‎09-01-2015

Rebecca I today ran your script, the one that identifies broken sources, and it ran without any problem. So there does appear to be something wrong with my Python Code. I am hard pressed to figure out what. I have many maps, in structure they are very much the same, but only one causes this problem. I will compare my code with yours.

‎08-31-2015

I had already added some print, but I should have reported what I found. The script does indeed go through all the layers (there is only one frame in the map in question) and then hangs at the end (To be more precise: the 64 bit version of python hangs at the end; the 32 bit version experiences a segmentation violation). I tried changing the order of layers. The script always reports all the layers. Same behavior. The map in question has many layers, but most of them are basemap layers which display at different scales. But the others maps I have are no different in that respect. I can see nothing special about this map. All the maps I have use the same basemap layers but only this particular one (I believe) exhibits this behavior. To repeat: I have run Map Doctor and Defragger. No change.

‎08-29-2015

The code that Rebecca kindly formatted did not properly show indentation possibly because tabs and spaced were intermingled. (Though there were the 3 errors she mentioned, since fixed). I have re-posted the script here with the proper indentation. import glob import os import sys import arcpy import arcpy.mapping as mapping from optparse import OptionParser # Previous invocations of this script were used to effect the following conversions # from Program Files (x86)/ArcGIS/Desktop10.0/Business Analyst/US/Data/ # to Business Analyst/2011Data/ # from RLIS_201402 to RLIS # from RLIS_201402 to RLIS_WebMercator parser = OptionParser() parser.add_option("-f", "--fix", action="store_true", dest="f", default=False,help="fix the source name") parser.add_option("-o", "--old-src", action="store", dest="o", default="RLIS-201402", help="the old source file name, or part of a name") parser.add_option("-n", "--new-src", action="store", dest="n", default="RLIS", help="the new source file name, or part of a name") (options, args) = parser.parse_args() old_src = options.o new_src = options.n def fixLayer(lyr): lyrModified = False if lyr.isFeatureLayer: #print lyr.name if old_src in lyr.dataSource: print "\t\t" + lyr.dataSource if options.f: lyrModified = True lyr.findAndReplaceWorkspacePath(old_src,new_src,False) return lyrModified def fixMap(mxd): mxdModified = False # loop thru all data frames in the map dataFrames = mapping.ListDataFrames(mxd, '') for frame in dataFrames: print "\tframe " + frame.name # get all layers in this data frame layers = mapping.ListLayers(mxd, '', frame) # loop thru all layers in the data frame #print layers for lyr in layers: lyrModified = fixLayer(lyr) if lyrModified: mxdModified = True if mxdModified: mxd.save() def fixit(fullPath,fileName): desc = arcpy.Describe(fullPath) if desc.datatype == "MapDocument": print fileName mxd = mapping.MapDocument(fullPath) fixMap(mxd) del mxd elif desc.datatype == "Layer": print fileName lyr = mapping.Layer(fullPath) if fixLayer(lyr): lyr.save() del lyr for arg in args: if os.path.isdir(arg): for fileName in os.listdir(arg): fullPath = os.path.join(arg, fileName) fixit(fullPath,fileName) elif os.path.isfile(arg): fixit(arg,arg) else: print arg + " is not a map, nor a layer, nor a directory" exit(0)

‎08-29-2015

I have run the MXD Doctor and the Defrag utility all with no effect. No the map in question does not have any .sde or server connections -- all the data is local. One other thing I should say is that I am presently running this script to identify layers using a particular dataset. I am not changing anything (so the -f option is false). Just as an aside: this Python interface of arcpy is so slow. One notices this also when invoking Python tools from within ArcGIS. Even to bring up the dialogue screen is sluggish -- 10,15 even sometimes 20 seconds. Given that Windows itself running under Parallels on my MAcbook Pro takes a mere 5 seconds to startup, those numbers are astonishing.

‎08-29-2015

I fixed the indentation problems, and Wing seems to say all is OK. But the behavior is unchanged.

‎08-28-2015

yes, you are right. I don't know how that happened. It was correct at one time. I have no idea what Python would do with that but I will correct it and retry.

‎08-28-2015

Can someone divine why the attached python script never terminates? I am running with ArcGIS10.3.1 basic. When I run this using 32-bit python it gets to the last layer and the terminates with a segmentation violation. With a 64-bit python it never terminates. The program runs fine on other maps I have, but one in particular gives this problem. AFAIK the map in question is not in any relevant sense different from those on which the script runs ok.

‎08-25-2015

Chris The data is not by any means the crown jewels, so I am certainly willing to share it. But I am not sure how much you need. Do you need the BA2015 datasets, or do you have those? I am not very familiar with packaging up a subset of data from a file geodatabase. I would certainly be interested in what you find. I will try to figure out how to package this stuff up. Rob PS. Cat OK : did not even expend one of his 9 lives.

‎08-25-2015

Dan and Xander Thanks for your replies. Many of the things Dan had recommended were already in place: same projection; local machine; polygons pertinent to geography. Now, what is true, is that these Business Analyst datasets have many fields (~2000 in BA basic). But surely when I run one of these python tools and simply enter the name of a dataset it should not take 3 minutes simply to display a list of fields, should it? I mean, there is no real computation happening yet. There is some structure containing field data and, paf!, display it in a dialogue box ought to be instantaneous, shouldn't it?? Ok, speed aside, how is it that the tool appears to run, claims to have completed successfully, and yet does not output the new feature class I specify? If the tool is overwhelmed by computational complexity, should it now just report that and quit with an error? I think possibly the way out of my dilemma is along the lines Dan suggests: 1. Use the BA layers (tracts, block groups, zip codes..) to create just polygons with no data. Store those polygons in some file geodatabase. 2. Join the data I am interested in with those polygons using a suitable unique identifier (tract ID, block group ID, zip code, ..) 3. Then create my own custom BDS layer as outlined in this document: https://www.esri.com/library/whitepapers/pdfs/importing-and-using-your-own-data.pdf Another advantage of that is that one can use the BA reporting capabilities. I am guessing that one needs these BDS layers to be able efficiently to use all the data. Very likely normal feature classes were never intended to have so many fields.

‎08-24-2015

I am going to try not to fill this post with expletives. Preliminaries: ArcGIS 10.3.1 running on latest 2015 MacBook Pro using Parallels, Windows7, 8G memory, no other user level processes running. Using: Arc Toolbox => Analysis Tools => Overlay => Spatial Join to join points(715 features) to polygons(469 features) The two datasets have the same coordinate system, although the point dataset has a join, and the polygon dataset has a giant number of fields because it was originally Business Analyst dataset, but which was pared down to the geographic region of interest The tool fires up. I enter the name of the target feature dataset (polygons). Then name of join features.... 205 seconds later the tool acknowledges the name of the dataset (no work yet mind you, just to respond that I entered the name of a dataset). Then I wish to join 1:1 and to merge features of the point dataset. So I delete all the features which cannot meaningfully be joined. Each deletion takes about 10 seconds to register for a net total of 270 seconds. Then the remaining 4 features Itell it to aggregate as a mean. Another 20 odd seconds. I enter the name of the output dataset -- incredible, it seems to respond almost immediately; what can be wrong? I have now spent 495 seconds just to enter the data. I now hit the run button. It executes, I get the usual progress dialogue at the screen bottom, and within a few seconds the tool finishes claiming to have run without error. Only, there is no output dataset created. There is a word for software like this, which decorum dictates that I had best not use. I have had the misfortune to have been required to use ArcGIS for years; have lost count of the number of bugs personally reported; grown old looking at a spinning blue wheel while the product seemingly does nothing. The tool did not work, already bad, but what kind of software engineering underlies a user interface that takes 10 minutes to specify a handful of data items? The behavior of this tool is replicable. There, I got to the end without using any 4 letter words. Now I can go out and relieve by feelings by giving the cat a good kick.

‎07-15-2015

I will answer my own question. It occurs because roads consist of line segments and each segment gets the same label. The Maplex engine avoids this. Layer Properties=>Labels=>Placement=>Label Density=>Remove Duplicates The Options will also allow you to fine tune the label placement (eg allowing labels to repeat provided they are a certain distance apart) You can choose to use the Maplex Engine on the Frame Properties.

‎07-13-2015

I am using ArcMap10.3.1 and Business Analyst 103.1 (2014 data). I am wondering why there are so many labels needed to identify freeways. I attach a map of Portland OR, showing interstate 5 labelled 13 times on a relatively short, straight stretch of the highway. Why is that? Surely it isn't necessary. What parameter can I set to control it? When I examine the label properties of the layer (Layer Properties => Labels => Placement) I see under Duplicate Labels that the box for Place one label per feature is checked. That cannot be an accurate description of what the checkbox does.

‎07-13-2015

I have a follow up to my original question. I am now running ArcMap (10.3.1) on Apple's latest and greatest MacBook Pro (Mid 2015 2.8Ghz Intel Cire i7, 16GB 1600Mhz DDR3), but still using Parallels. 8GB of memory are allocated to the Parallels VM (running windows 7). Frankly, Arcmap still performs poorly, although certainly better than previously. One thing I have learnt, which may be of benefit to others, is that Arcmap 10.3.1 (and presumably all other Arcmap's) is a 32 bit application and, as such, cannot itself benefit from having more than 4G memory. It is true that if the VM has many other processes running that performance may improve with more memory in those cases where swapping is needed, but it is doubtful whether desktop users are doing much other than running Arcmap. In fact I don't think I have ever seen Arcmap use more than about 2.8G memory. So, although a higher end machine would help, I still find the performance poor. I may also say that it doesn't help that the slightest nudge of a magic mouse seemingly provokes Arcmap into redrawing. I'd go so far as to say that one should always halt the redrawing if one is doing anything at all to the table of contents. If one does in fact halt redrawing, the map itself gets blanked out, another undesirable feature of the product.

‎07-07-2015

I cannot now replicate the problem I had with Batch Project. I am reluctantly obliged to acknowledge that I must have made some kind of error. The comments made by Xander, Melita, and Dan are correct. Still, I am not completely convinced that all is fragrant in the State of Denmark. I had a file geodatabase on an external hard drive. I am sure I tried more than once using Batch Project to list the feature classes so I could select them all. Batch Project saw the file geodatabase, but displayed no feature classes within it. Today, I am doing the same thing and now the feature classes are showing up. I apologise to anyone who wasted time reading my original post, and my thanks to everyone who responded. Rob Stevens

Online Status	Offline
Date Last Visited	4 weeks ago

My Ideas

Latest Contributions by RobertStevens

ArcGIS Desktop. This just is not working well enough for us.

Re: Python script never terminates

Re: Python script never terminates

Re: Python script never terminates

Re: Python script never terminates

Re: Python script never terminates

Re: Python script never terminates

Python script never terminates

Re: Spatial Join Tool : does not work; slow as glaciers;

Re: Spatial Join Tool : does not work; slow as glaciers;

Spatial Join Tool : does not work; slow as glaciers;

Re: Labels on Business Analyst basemap layers

Labels on Business Analyst basemap layers

Re: ArcGIS Desktop Performance

Re: Convert the projection used by multiple feature classes in a file geodatabase

Transform from NAD_1983_HARN to WGS_1984

Shape_Length and Shape_Area of Geographic Coordina...

Re: Conda. How can it be so slow?

Conda. How can it be so slow?

Activation of Conda Environments