Latest Contributions by FilipKrál

‎03-30-2016

Hi all, I wrote a Python script where I use multiprocessing.Pool.map to run a function on different parts of a large dataset in parallel (read only, results are stored in a separate directory for each process). The python sub-processes produce the expected results but they never close down so I end up with many python.exe processes running forever (even after the "master" script has finished). Each of these processes consume around 100 MB of RAM and I think I will run out of memory when I run this script on a full set of input parameters. I've been running it with 6 CPU cores but I end up with many more python processes running. How come they are not shutting down? Have you seen something like that before? I'm using ArcGIS 10.3.1 with ArcGIS Pro 1.1 and Python 3.4 (64 bit) installed for stand alone scripts. While my real script is much more complicated, this is essentially what I am doing: import arcpy import multiprocessing def worker_function(pars) """Function to be run in parallel""" wd = pars.get('wd') # ... do some work with spatial analyst ... return {"result": ["spam", "eggs"]} # parameter sets is a list of dictionaries of primite Python strings def main(): parameter_sets = [ # ... { "flow_direction_raster": '...', "weight_raster": '...', "output_raster": '...', "relative_to": '...', "extent": '...', "wd": '...' } # ... ] pool = multiprocessing.Pool(6) results = pool.map(worker_function, parameter_sets) return results if __name__ == "__main__": main()

‎03-29-2016

Good point, Curtis. String substitution aka string formatting can make the whole code much more readable. Especially considering all the options the format method offers. F.

‎01-19-2016

Haha, Thank you very much, mister Xander Bakker, that is the kind of "slide-of-hand-arcgis-trick" I was looking for. Nicely done, I'll go with that. By the way, dropping the 11 digits after the decimal place of each coordinate reduced the size of the output text file from about 150 MB to about 50 MB! To clarify the coordinate system, this was in British National Grid (EPSG:27700), i.e. projected in metres. Thanks everyone, Filip.

‎01-18-2016

Hi, I have a polygon feature class and I want to produce a text file where each feature is a row with an ID and the polygon geometry formatted as Well Known Text (WKT). To start with I am looping though the features and I use the .WKT property to produce the result. My problem is that the WKT is often formatted with coordinates reported to 12 decimal places. I need to round the coordinates to the nearest meter. Ideally I want to be able to do this with other types of geometries so I'd prefer something else then looping over the rings and vertices. Is there a way to do this? I tried to set arcpy.env.XYTolerance and arcpy.env.XYResolution to 1.0 but that had no effect. Any hints? Filip.

‎01-11-2016

My tile cache is finally fully functional again.This is what worked for me: First, I could not delete any of the cache files in Windows Explorer or using the Delete Map Server Cache tool. So I restarted the machine. Then I used Delete Cache Tiles to get rid of all the tiles. Luckily, the tool worked this time, but I had to define the caching in service properties again and I selected "update cache manually". Then I used Manage Map Server Cache Tiles tool to generate the cache from scratch. Fortunately my tile cache is quite small so it didn't take long to re-generate it. Then I installed ArcGIS 10.2.2 for Server Map Cache Consumption Patch which I believe is the one Michael Volz had in mind. I haven't had a chance to try it again just yet, but hopefully I won't run into the same issues next time. Many thanks for all your help. Filip.

‎01-08-2016

The issue I am dealing with at the moment appears to be similar to this: Manage Map Cache Tile: Errors at 10.2.2 When I left "Update cache automatically" ticked while overwriting the service, I can then see my CPUs work harder. Once the CPU load drops again (caching has finished) and I preview the service, some tiles were obviously missing (HTML Error 404). I then tried to re-create the tiles using Manage Map Cache Tile tool but it errors out saying: Output failure, error string = Error moving bundle Failed to cache extent:... and then it repeats for each process (i.e. each CPU that worked on it). Filip.

‎01-08-2016

Jennifer's answer is right. I found it to be the easiest way to update data sources or other properties of a service. See also: Overwriting a service in ArcGIS for Desktop—Documentation (10.3 and 10.3.1) | ArcGIS for Server However, things usually get complicated with tiled services. In theory it should work as the help says - if you keep "Update cache automatically", the server will be busy for some time bit it should deal with it. In practice though, I always run into difficulties with updating cached services and then spend long time fixing it. If you have any tips, please let me know. I use 10.2.2

‎12-07-2015

Hello, Since many of the tools for exporting and importing metadata are not available in ArcGIS Pro, what is the recommended way of working with metadata in geoprocessing work flows? In Python 2.7 and ArcGIS for Desktop it is possible to import/export metadata using geoprocessing tools. I have been using this in my scripts that I need to migrate to Python 3.4 Specific example is the function 'meta' here: https://github.com/NERC-CEH/arcapi/blob/master/arcapi.py#L913 Kind regards, Filip.

‎11-22-2015

Hi, Thank you for the suggestions. I ended up using the ExtracValuesToPoints repeatedly on in_memory feature classes and built the resulting feature class that way, renaming the RASTERVALU column each time (i.e. copying the RASTERVALUE column to a new column and deleting RASTERVALU column). Seems to work OK. F.

‎11-19-2015

Hello all, I need to extract values from about 30 raster datasets (each about 500 MB in size, signed 32 bit) for about 1500 points. When I use the Extract Multivalues to Points tool, it quickly consumes all my memory (16 GB) and it never finishes. I must admit I only left it running for about an hour, but that is more than the whole process took in ArcMap. It seems the tool is extremely memory intensive and I wonder if somebody has any suggestions how to get around it. An obvious option is to use the Extract Values to Points tool repeatedly. Any other ideas? Filip

‎11-18-2015

Hi, What do you mean by "attribute"? The attr(name, value) method adds an attribute to the DOM node of the graphic, not to the graphic feature. So setting feature.attr('something', 123) will result in the graphic dom node to be: <path something="123" ...></path> (although I am not sure what types of elements graphics are). In order to add an attribute to the graphic feature, in the sense of adding a new field, use the setAttributes method. Filip.

‎11-18-2015

Today I learned a new fact relevant to dissolving or intersecting large datasets. ArcGIS Pro has Parallel Overlay toolset with Parallel Dissolve and Parallel Intersect tools that I hope address this issue. I haven't tried these tools yet but the help looks promising. Filip.

‎11-18-2015

Hi, You've made quite some progress here. In order to see the tool parameters in the tool dialog, you have to configure them in the script tool properties as it is described in Adding a script tool and in Setting script tool parameters. Make sure the order of parameters in the table matches the numbers you assigned to them in the script (0 is the first parameter). You will also likely need to use the SetParameter function by the end of your script to let ArcGIS know what values should the tool return. For example, say that the end point parameter is the third parameter of your tool and its direction is "output". In your script you will write arcpy.GetParameter(2) to access the value provided by the user and eventually arcpy.SetParameter(2, 'some value') to notify ArcGIS what the resulting value is. Hope this makes sense. Filip.

‎11-13-2015

I tried this in ArcPro Python window I now am even more puzzled. When I pasted the code to the window and executed, all of the arcpy.Exists calls returned False. Even the first one so it suggested that the layer was never created. I then tried running just the MakeFeatureLayer tool and this is what I got: r = arcpy.management.MakeFeatureLayer(in_fc, lrname, w) r.getMessages() # 'Start Time: Fri Nov 13 14:30:25 2015\nSucceeded at Fri Nov 13 14:30:25 2015 (Elapsed Time: 0.03 seconds)' r.getOutput(0) # 'lr1' arcpy.Exists('lr1') # False arcpy.Exists(r.getOutput(0)) # False What is going on here?? By the way, no layer was added to the table of contents either, not even when I run MakeFeatureLayer tool manually, but that is not an issue. F.

‎11-13-2015

Hi, On line 12 I delete the layer object (variable lr) and since that didn't work I added lines 13, 14 just to show all the things I tried and didn't help. I tried setting data_type parameter to 'FeatureLayer' and also 'Layer' but that didn't help either. The help suggests the data_type parameter is "read only". I am using the standalone python 3.4 that comes with ArcGIS Pro and I am executing this as a script file from windows command line. F.

Online Status	Offline
Date Last Visited	‎11-11-2020 02:23 AM

My Ideas

Latest Contributions by FilipKrál

Python multiprocessing not shutting down child processes

Re: Question for a script!!

Re: Format WKT to nearest meter

Format WKT to nearest meter

Re: Change data Source of published map service

Re: Change data Source of published map service

Re: Change data Source of published map service

Editing Metadata in ArcGIS Pro from Python

Re: Extract Multivalues to Points consumes all memory and never finishes in ArcPro

Extract Multivalues to Points consumes all memory and never finishes in ArcPro

Re: add attribute to Graphic attributes

Re: Solution to: Dissolve tool does not fully dissolve medium-size dataset

Re: Geoprocessing service that returns features to ArcMap

Re: ArcGIS 10.3.x Python 3.4 cannot delete layer

Re: ArcGIS 10.3.x Python 3.4 cannot delete layer

Re: Remove letters and special characters from an ...

Re: Analysis of overlaying rasters

Re: Update cursor with joined tables work around w...

Re: Reference JS page from JS

Re: How to create a Polygon from a single point

ArcAPI