Latest Contributions by JoshuaBixby

‎07-25-2014

I have generally had more success scaling performance with multiprocessing than with multithreading, but a lot depends on the extensions/packages being used and specific structure of the code. I believe ArcPy requires CPython. Unless functions are specifically written to address the GIL, which CPyton has, multithreading code won't always perform as people might expect. But again, it depends on the specifics. And, if locking I/O is problematic and multithreading can address it, it beats multiprocessing code that doesn't run at all.

‎07-25-2014

I believe semantics are important in all aspects of life. Whether in law, medicine, science, business, information technology, or any other field; having a common language doesn't do much good if there isn't a common understanding of the words that make up the language. Since languages evolve, maintaining a common understanding of words over time is a continual challenge. It is all too common for the familiarity of words, especially argot, to lull people into a false sense of common understanding. How many meetings have we all been to where a topic is discussed, decisions are made, and everyone walks out thinking something different in terms of what is going to happen next? Such confusion can be intentional on behalf of one or more of the parties involved in the meeting, but Hanlon's razor dictates that is the exception more than the rule. I commonly see people use the same words to mean different things when the context or participants of a discussion changes and implied qualifiers are no longer understood or known. There are times, however, when I see apathy or complacency prevent people from clarifying what is known to be misunderstood, thus willfully perpetuating confusion. It is for this reason that I have one of many worn and tattered soapboxes I must drag out of the closet from time to time. Since I rely on geospatial sciences and information technology for part of my work, I seem to find myself frequently running into semantic overloading in discussions and documentation of related software. Some of the Tilting at Globes blog posts will discuss specific examples of where failure to address known semantic overloading makes work less productive for GIS practitioners.

‎07-24-2014

I found a bit of irony in the following bug description my agency recently ran into: NIM103380 : Unable to identify or select features from a view created using the Create Database View tool on ST_GEOMETRY feature class. Status: New Workaournd : Use the ArcSDE command line to create the view. It seems removing ArcSDE command line tools makes the Esri Support toolbox a bit smaller for offering workaround suggestions.

‎07-24-2014

Also, how big are the data sets you are working with? Would it be possible for each child process to make an in-memory copy of the data set, or possibly a temporary copy on disk? If there is some kind of file locking occurring, even though you aren't editing the data, maybe giving each child process its own copy of the data to work on would work around the issue. Clunky, but you will still need a workaround if this is being caused by a bug.

‎07-24-2014

What happens if you set maxtasksperchild=1 when instantiating the pool? I realize this isn't a good idea operationally since it will kill performance, especially on Windows, but it will be interesting to see how it affects the errors.

‎07-23-2014

At least for me, it is hard to provide much feedback given the limited code snippet. Are you executing this code through the interactive Python window in ArcGIS Desktop?

‎06-09-2014

This is a fantastic idea, but one that I can't imagine we will ever see. The vast majority of Help documentation just scratches the surface, especially with developer-related topics, tools, etc.... One of the biggest hurdles I see to this type of idea is that a large number on bugs aren't even published to ArcGIS Resources. I have never figured out how they decide what bugs get published publically and what ones don't. Great idea, though. Given how common bugs are and how many string are typically attached to tools, this kind of linkage between bugs and Help documentation would be a big time saver.

‎06-09-2014

This is a fantastic idea, but one that I can't imagine we will ever see. The vast majority of Help documentation just scratches the surface, especially with developer-related topics, tools, etc.... One of the biggest hurdles I see to this type of idea is that a large number on bugs aren't even published to ArcGIS Resources. I have never figured out how they decide what bugs get published publically and what ones don't. Great idea, though. Given how common bugs are and how many string are typically attached to tools, this kind of linkage between bugs and Help documentation would be a big time saver.

‎05-14-2014

I know this thread is older, but I figured I would chime in because I have been running into problems with using in_memory workspaces. When it comes to using the in_memory workspace in the interactive python window when background processing is enabled; well, all bets are off. There are numerous tools that don't actually create GPInMemoryWorkspaces when using 'in_memory' with 'Background Processing' in the interactive Python window. I primarily work in ArcGIS 10.1, but I have also noticed the same is true with 10.2.1. If 'in_memory' doesn't mean in RAM, then performance could be much lower than expected. If using 'Background Processing' with the interactive Python windows, it pays to double check whether your 'in_memory' objects are actually in memory and not in disk-based temporary file geodatabases.

‎03-21-2014

@scottball, thanks for posting a workaround. I haven't tried it yet but will. That being said, human automation is a sign of poor software development. ArcGIS Desktop should be much more intelligent with its handling of locks.

‎03-09-2014

'sdemon' certainly did require SDE user access rights, it just passed those access rights to any connection on the port (which could be construed as a security hole). - V I am not sure playing the security card solicits the intended response for most people with this issue. I know our IT security folks won't feel better knowing Esri viewed the functionality of SDEMON for listing users as weak security, possibly a security hole, and yet never addressed the issue over years and many versions of the software. I felt better about it when I thought it was just incomplete functionality.

‎01-22-2014

Incident has been created with Esri Support with the same subject/name as this forum posting.

‎11-25-2013

I have since learned that ArcPy now supports AcceptConnections, DisconnectUser, and ListUsers as of ArcGIS 10.1. Sigh.... ListUsers requires an administrative SDE workspace/connection, SDEMON didn't, so there is still some progress to be had.

‎11-21-2013

For example, how is replacing several of the sdemon commands (-o info -I users/locks, -o pause/resume, -o kill) with tabs in ArcCatalog a step forward? I have to eat one of my earlier statements. I have since learned that ArcPy now supports AcceptConnections, DisconnectUser, and ListUsers as of ArcGIS 10.1. The sdemon functionality for listing locks and stats still appears to be missing. As an FYI, there is another related enhancement request someone referenced on the ArcGIS Ideas site: NIM095119 - SDEMON:Create geoprocessing tools or arcpy functions for operations performed when using the 'sdemon' command.

‎11-18-2013

I wasn't sure whether to post the following comments here or over in the Blogs, so I erred on doing both. I don't mean to be contrary, but I just can't get excited about replacing one suite of simple, script-able tools with an assortment of scattered and mostly GUI-driven tools. For example, how is replacing several of the sdemon commands (-o info -I users/locks, -o pause/resume, -o kill) with tabs in ArcCatalog a step forward? I realize the tabs in ArcCatalog are going to be convenient for the casual SDE administrator, and I think providing them brings some value, but providing them as the exclusive means to manage geodatabases is a step backwards. I am not necessarily advocating for keeping the command line binaries around, I just think nearly all of the functionality should exist through non-GUI means in one form or another. If Esri wants to migrate away from command line binaries to ArcPy/Python; great, I will work with that. I just can't work with managing thousands of users scattered across hundreds of geodatabases using tabs in ArcCatalog or some other GUI-only option.

Online Status	Offline
Date Last Visited	Saturday

My Ideas

Latest Contributions by JoshuaBixby

Re: MakeFeatureLayer fails in multiprocessing python

Semantic Overloading

Re: Alternatives to using SDE command line tools - Blog Discussion

Re: MakeFeatureLayer fails in multiprocessing python

Re: MakeFeatureLayer fails in multiprocessing python

Re: MakeFeatureLayer fails in multiprocessing python

Re: Update Help Files With Known Issues

Re: Update Help Files With Known Issues

Re: Insert Cursor to Table In Memory

Re: 'Ghost' Locks on Data

Re: Alternatives to using SDE command line tools - Blog Discussion

Re: mapDoc.save() fails after mapDoc.replaceWorkspaces() in MXD with ArcGIS Image Ser

Re: Alternatives to using SDE command line tools - Blog Discussion

Re: Alternatives to using SDE command line tools - Blog Discussion

Re: Alternatives to using SDE command line tools - Blog Discussion

Re: How to use ArcPy without installing ArcGIS

Re: Update the organization URL in ArcGIS Enterpri...

Re: Mobile or File Geodatabase Views: Grab earlies...

Re: import arcpy tramples datetime import

Re: Make All Fields All Capitial

Python Snippets

Esri Forestry Group (EFG)