Latest Contributions by KimOllivier

‎01-27-2017

I never use an arcpy tool in a loop. They are not designed to do that. You could assemble your list of FIDS into a list, convert to a string and use a SQL "IN" statement to select in one step. Faster, easier to understand, easier to debug and easier to create. Why do you need to check if the parcel id exists? If it doesn't it won't matter in an IN list. # list of valid ids (do a regular expression in a list comprehension to validate print(len(FIDS)) # just to see how many, limit 50,000 should be fine, if more then partition, can use 1M if integers not strings the_list = str(tuple(FIDS)).replace('u','') # strip off any unicode tags sql = "{} in {}".format(the_field,the_list) print (sql[0:40], sql[-40:]) # check syntax # make sure FID is indexed for performance arcpy.SelectLayerByAttribute_management(parcels, "ADD_TO_SELECTION", sql)

‎01-02-2017

Yes we all got this! Install qt yourself: conda install qt-4.* Then try the spyder install again conda install spyder This will trigger yet another upgrade of qt to 5.6.0, but let it run to completion. You have to install the 4.* version first. I found that spyder then opened from the start menu shortcut. So much for conda handling dependencies! Who would have thought of this?

‎01-02-2017

This did work for me but, using this roundabout process: Could not uninstall qt, could not reinstall 4.8.7-vc10_4 because not found I could however install a variation using conda install qt-4.* which found a later version This then enabled spyder to install conda install spyder which then immediately upgraded qt to 5.6.0-vc10_0 so I didn't argue since it was not numpy or scipy So now spyder will open from the shortcut in the start menu. I am staying with Windows 7 BTW. So much easier. AVG antivirus objects to Python so I tell it to ignore it. Thanks Clinton, you gave me enough hints to blunder through without resorting to root or reinstalling the operating system.

‎01-02-2017

Yes! I got it working without going into Root with a few changes to your suggestions. I am getting the fail when trying to install spyder and the usual error message from User 'ray' who must be a developer of qt. Firstly I could not remove qt because it wasn't installed with ArcPro apparently. I could also not install qt-4.8.7=vc10_4 because it is now qt-4.8.7-vc10_9 [vc10] I got around this by using a wildcard to install qt : conda install qt-4.* Then I was able to restart the spyder install: conda install spyder, which upgraded qt to qt-5.6.0-vc10_0 anyway so I didn't argue as it wasn't numpy or scipy Now I can open spyder from the shortcut in the start menu. The next stage is to work out how to get the ARcPro licencing fixed up, but it is a pubic holiday today here. This is all distressingly complicated. I will have to set aside half a day for what goes wrong with Python setup in the new ArcPro training course.

‎07-17-2016

Ok, so you can create a Spatialite database in Arcpy. But what can you do with it? Where is the documentation on which functions and tools support Spatialite? The help is completely silent on this. The reason I even put in 10.4 was because of the new tool arcpy.management.CreateSQLiteDatabase. I assumed that there would be lots of new (hidden) support for spatialite in arcpy. But maybe I was too hopeful. Could we users need to set up a Wiki for information on this? Any suggestions for the most suitable platform? My experiments are a mishmash of SQL calls using a number of python APIs to sqlite or spatialite plus some arcpy calls. Some are excellent, others are very slow or impossible. I switch between the obvious and obscure as I explore the limits. It would be nice to have a consistent simple interface that worked the same way as the venerable file geodatabase. Perhaps a new thread discussing the limits would be better.

‎07-14-2016

It fails in either environment. Standalone is a good idea these days if you are doing batch processing because it can use the 64 bit version of Python that has more memory for those crude memory hog routines that are now so popular. By the way, partitioning is the solution for this, cartographic partitioning is now used for some tools and one can adopt the same technique to reduce memory and speed up processing.

‎07-14-2016

It is a good idea.I am already using pyspatialite to handle sqlite based tables and featureclasses. But converting back to file geodatabases needs a conversion tool. In theory this is supported by the conversion utilities, otherwise I have to complicate the workflow by using FME or a complex dump/reload. Some tables have geometry so they turn into a featureclass. Most of my customers are still more comfortable with a file geodatabase. I cannot (yet?) use database relates or store metadata for example. But I have other customers that do not have a GIS so they need data in a database exchange format. What to use? Access database have been deprecated and anyway they have 2GB limits which I have hit. File geodatabases are not an option. I could use SQL Server and attached databases if I splashed out for that. Linux based customers do not use SQL Server. PostGIS only has a crude dump. So sqlite looks ideal as a modern, compact, open, supported(!) exchange format for tables and featureclasses. Since spatialite (or geopackages) are being used for Esri Mobile applications this new 'personal' geodatabase format is gradually being supported. I have found spatialite much faster and more elegant for some operations. I switched to that format for a mirror of external data held in PostGIS. The replication has to be a 'homemade' workflow which is difficult to make robust but was easier if I also went with a similar database to PostGIS. Maybe I should have used PostGIS locally, but Sqlite is much faster and easier for a non-distributed purpose. I also found WFS unreliable so I went with CSV extracts using WKT for the geometry (on advice).

‎07-14-2016

Using a SQL filter in the TableToTable tool fails in the same way.

‎07-12-2016

It is a comment. I am grumpy. There is no error message, it is a sudden crash. Windows managed to point the finger at GdbCore.dll. Problem signature: Problem Event Name: APPCRASH Application Name: python.exe Application Version: 0.0.0.0 Application Timestamp: 5560ad83 Fault Module Name: GdbCore.dll Fault Module Version: 10.4.1.5686 Fault Module Timestamp: 57056049 Exception Code: c0000005 Exception Offset: 0009e77e OS Version: 6.1.7601.2.1.0.256.48 Locale ID: 5129 Additional Information 1: 0a9e Additional Information 2: 0a9e372d3b4ad19135b953a78882e789 Additional Information 3: 0a9e Additional Information 4: 0a9e372d3b4ad19135b953a78882e789 I have taken the time to submit a formal bug report with a script to demonstrate the problem. Yes my code is still 2.7 because it was written several years ago and was working just fine! Maybe it is a unicode error but it is not in Python because it does not get trapped with an exception. So I guess it must be deep inside GdbCore.dll which is mentioned in the windows traceback. My first effort to generate a sample using python/sqlite3 worked, but with errors, the GetCount on a view with a filter returned the total table count. So I have had to provide a link to download a larger file (288 MB) that demonstrates the behaviour. It is not the only one. I thought I was being conservative. I put 10.4 on a separate machine and used it for a week first. If it is a full release I simply buy a new machine, start again and leave the old one alone to easily go back to if there is an obscure bug. I suppose that is the purpose of a virtual environment, but I don't have any spare licences and I would expect it to be very slow.

‎07-12-2016

(Sigh) I thought that a minor upgrade from 10.3 to 10.4 would be uneventful, but unfortunately I have hit a new bug that is painful to work around. I am copying a subset from a spatialite database to a file geodatabase using a View and SQL filter. This was very elegant but now it crashes completely. The tables are large so it is time consuming to create a repeatable standalone demonstration, but if I cannot do that I can appreciate it is hard to track down so here goes as a warning to others. <CODE> db = "e:/parcel.sqlite" pre = '/main.' for src,filter,tab in [['Legal_Description',"ttl_title_no is not null","Legal_Description"]]: if not arcpy.Exists(sqlite+"/"+tab): try: print tab, filter, db+pre+src, sqlite+"/"+tab arcpy.management.MakeTableView(db+pre+src, 'view', filter) count_view = int(arcpy.management.GetCount("view").getOutput(0)) print count_view,"count_view" if not arcpy.Exists(intermediate_db+'/'+tab): arcpy.conversion.TableToTable('view', intermediate_db, tab) # crashes here count_table = int(arcpy.management.GetCount(intermediate_db+'/'+tab).getOutput(0)) print "intermediate table count",tab, count_table if count_table != count_view: sys.exit() # give up here assuming that the TableToTable does not crash first arcpy.conversion.TableToTable(intermediate_db+'/'+tab, sqlite, tab) print tab, "copied", except Exception, errmsg: arcpy.AddError("Error {}\n {}".format(tab,errmsg)) print "Error {}\n {}".format(tab,errmsg), arcpy.GetMessages() </CODE> The problem is deadlines, not reporting bugs. I have thought of three workarounds: Use another system with SQL functions that work on featureclasses eg SQLiteExpertPro, but this is a bit manual. Copy all the tables into a filegeodatabase (Table to Table works on full tables), very slow Add pyspatialite and compose sql queries in python. Works but takes time to write.

‎06-19-2016

Do you have a lot of python third party modules that you have to reload? I like using spatialite for SQL queries because they are thousands of times faster. So I have installed pyspatialite. But it does not install from pip because that assumes a C compiler is available. I cobbled a compiled DLL by copying it from an installation of osgeo4w. if I want to take advantage of python64 bit I have to install the 64 bit version of the modules into ArcGISx6410.4 as well. That is a bridge too far, so if I need a custom module I just use an explicit path to the 32bit python executable. Then there is python 3.5! Do I need to install all the modules into that as well for ArcGISPro? At least some of the common once such as Scipy are needed by Esri for spatial statistics so they are installed by default, but there are plenty of other useful ones such as SQLAlchemy, Geohash, Latlon, Reportlab, Pytz, Pygments, Requests, Mechanize, BeautifulSoup etc.

‎06-19-2016

I have done a "simple" dot release upgrade from 10.3.1 to 104.1 on Windows 7 and now I am back to slow startups again. So it does not appear to be related to Windows 10. No change to my licence server except that I loaded a new version. So I think it must be something in the licence manager. I am on a local network with a single use licence. I tried: adding a startup directory to the shortcut, deleting my mxt startup templates turning off arcgisonline on startup I note that it takes 2 minutes for ArcGIS Adminstrator to open and there are expired licences in the list that I cannot remove. Maybe that is the issue?

‎06-15-2016

Well it looks just like that function but appeared as a function of arcpy, not a separate function. It popped up in the autocompletion so I have been using it extensively in 10.3. To my surprise it has disappeared after an update. Maybe ET Tools added it? Maybe some other tool I have installed long ago and forgotten and did not realise that it extended arcpy. Is there one? What did arcapi do?

‎06-14-2016

This convenient arcpy function that was there in 10.3 has vanished at 10.4, breaking all my python scripts that use it. Thanks Esri.

‎06-12-2016

The approximation calculating the buffer distance using the perimeter looks fine to me. You don't actually have to create a featureclass if you use geometry objects that have been added to Arcpy recently. Just copy the featureclass into a polygon geometry and then use arcpy functions to buffer it. All in memory, extremely fast and easy so it should be quite ok to iterate.

Online Status	Offline
Date Last Visited	2 weeks ago

My Ideas

Latest Contributions by KimOllivier

Re: Why would my additional MakeTableView_management() calls take drastically longer?

Re: Python and ArcGIS Pro 1.3 : Conda

Re: Python and ArcGIS Pro 1.3 : Conda

Re: Python and ArcGIS Pro 1.3 : Conda

Re: New Arcpy bug at 10.4

Re: New Arcpy bug at 10.4

Re: New Arcpy bug at 10.4

Re: New Arcpy bug at 10.4

Re: New Arcpy bug at 10.4

New Arcpy bug at 10.4

Re: ArcGIS (10.4) Slow Loading on Windows 10

Re: ArcGIS (10.4) Slow Loading on Windows 10

Re: Where has AddMsgAndPrint gone in 10.4?

Where has AddMsgAndPrint gone in 10.4?

Re: buffer distance

Re: ArcGIS Pro Tips & Tricks 2026 – Share Yours!

Re: Difference between feature iteration and Row i...

Re: Adding WFS to webmap using Python API

Re: ArcGIS 3.6.2 Pro Not Closing

Re: ArcGIS for Personal use new licensing from 202...