Latest Contributions by HugoBouckaert1

Hi  We have to process big data and for that Geoanalytics server seems to be a good solution because it supports a clustered environment. The Geoanalytics server supports arcgis libraries which themselves can make use of clustered computing (just like pyspark). However, it does not support arcpy. At the moment all my scripts are written using arcpy, using an ArcGIS Pro desktop environment and Jupyter notebooks.  The scripts are quite complex and use a lot of arcpy functions.  Before we invest in implementing a Geoanalytics server, we need to know if my scripts will run only using  the arcgis libraries without access to the arcpy libraries. I know that arcgis is web-focused and is a wrapper for the REST api, and I am happy to translate what I have written using arcpy into arcgis. I also know the two are meant to be complementary to each other rather than "like for like", but nevertheless, for the scripts, I need equivalent functionality in arcgis that is currently available in arcpy.  So my questions are:  1. Is there a URL with an "equivalence list" and "exclusion list" ? For example there is an append procedure in both, but I could not find a NEAR function in arcgis (could be there of course, but this is just an example).  Going through the documentation function by function comparing arcgis to arcpy is a huge task, so a page that lists equivalent functions would be very helpful. Similarly a page that lists functions supported in arcpy that are not in arcgis would be very useful.   2. Can we use the Geoanalytics server stand-alone without making reference to ArcGIS Online or a local implementation of Portal? 3. Are File geodatabases supported? I watched a couple of videos about Geoanalytics server and the datastores it can use. Although shapefiles are mentioned File geodatabases were not in the list of supported datasets.  Only being able to use shapefiles would be a limitation, especially because the the lenght limit of column names.  Any help will be greatly appreciated.  Thanks Hugo     ... View more

Hi  I have several arcpy scripts that run on my desktop but take days to complete. So far I have spatially gridded the data and then used parallel processing to process the spatial grids simultaneously. We are looking at a better solution however, because creating spatial grids creates its own problems, with features "cut in half", and if when kept in two grids, resulting in duplication.  Does the GeoAnalytics server support arcpy? It does not look like it, but if it does, this might be a solution. Ideally we are looking at running this code in an Azure cloud environment. In Azure, you can make use of clusters (multiple VMs) which can be scaled up to do some powerful processing. However, in Azure only pyspark libraries are supported for multiprocessing. When you use arcpy, you have to install an anaconda environment (e.g. the environment that comes with ArcGIS Pro), but any code you run cannot use Azure clustered resources and so you are back to being limited to using a single server.  Are there any solutions out there to run python code with the arcpy libraries at scale? The compute resources needed are huge, so a single server, even if very powerful, might not suffice. Note also that python by default runs on a single core, so that a single powerful machine even with 64 cores will not speed up the process because only a single core will be used.  This is why I spatially gridded the data and sent each grid to a separate core, but as I said, we would like to get away from that solution.    Any ideas or help would be most welcome.  Thanks Hugo            ... View more

Hi  I have used arcgis.learn on an aerial image, i.e. where you have a single image with many objects in it, on which you train a model and later use to detect similar objects in that same image.  What if you have hundreds of "ordinary" images i.e. non-georeferenced images taken by a camera from the ground? I have hundreds of photos of similar objects which I would like to classify and detect using arcgis.learn. However, when you use the "“Label Object for Deep Learning” tool in ArcGIS Pro, it is setup to draw around objects in a single (very large) image such as an aerial photograph. If you then export the labelled dataset  it can be used by "Train Deep Learning Model”  to train the model.  However there seems to be no facility to load, say 100 snapshots in ArcGIS Pro and create a single labelling dataset out of these 100 or so images for training purposes. Does ArcGIS Pro cater for this or is this not something the deep learning tools in ArcGIS Pro are supposed to be used for?  If this is possible, how would you do it? The question is not only about labelling multiple images and output that as a single labelled dataset, but after the model has been created, also detecting similar objects, not in a single image, but in multiple images.      Thanks Hugo  ... View more

Hi  I have line features representing overhead power lines, and for these power lines, I have height attribute values at three different locations: the start of the line, the end of the line and (using a distance attribute from start) the point of maximal sag of the line between start and end points. I do not want to create straight lines between start point, sag point and end point but a curved line of best fit between the three points. Note that the "sag" point is not always midway between the start and end point. I need to do this in a python program as I have to calculate this for a large volume of lines, so the editing "arc" tool is not really something I can use. I looked into trying to fit an ellipsoid between the points but that is no good as this would mean I have one ellipsoid between start point and sag point and another ellipsoid between sag point and end point. I need a single curve that fits all three points. I tried the spline tool, which creates a raster surface from points (which means I can perhaps get back from raster to more points), but 3 points is not enough to create a raster surface. I also tried non spatial tools such as scipy interpolate (fitting a spline between sets of np.array values), but here also, I only have 3 points which is not sufficient to do the interpolation and create the "spline" line. Anyone knows how to create a curve of best fit between three point locations? In my case these points are z (height) values so either a curved "best fit" line is calculated directly or intermediate points are calculated from which I  create a new line feature. However, in that case I also would have to find the X and Y coordinates of the newly created (intermediate) z height values.  Difficult problem, I know, I am just wondering if someone has worked this out, or can help me in any way.  Thanks very much.  Hugo       ... View more

Hi  I would like to have a map (utilised by a webapp) that caters for more than one use, because the background layers will be the same for all users. The use includes editing, but editors of layer A should not be able to edit (and see) layer B, and vice versa. I can put editors of layer A in one group and editors of layer B in another, but my understanding is that the map (and webapp item) have to be part of both groups for any of the people in the groups to even open the map or webapp. Is it possible that the same map (and webapp)  is used by different groups whereby one group can edit layer A but cannot see (and edit) layer B, and the other group can edit layer B but not see (and edit) layer A? Perhaps there is a mechanism via views, but would views allow editing? Thanks Hugo       ... View more

Hi  I have been trying to delete a field from a feature layer. I have tried "delete_from_definition" and I get a success response, and yet the field (column) is still there afterwards:  1. I have a layer object, named updater_layer (it is definitely a feature layer object, I have checked its properties).   2. I create the JSON that defines the column:  status_remove = { "name" : "STATUS", "type" : "esriFieldTypeString", "alias" : "Status", "sqlType" : "sqlTypeOther", "length" : 35, "nullable" : 'true', "editable" : 'true', "domain" : 'null', "defaultValue" : 'null' } 3. I use: updater_layer.manager.delete_from_definition(status_remove) {'success': True} but when I look at the properties i.e.  updater_layer.manager.properties The STATUS field is still there.  How do I remove a field from a feature layer in ArcGIS Online using the arcgis.gis API?  Thanks Hugo   ... View more

Hi  Is it at all possible to create a feature layer in ArcGIS Online from a GeoDataFrame? If you cannot create a feature layer directly from a GeoDataFrame, is it possible to populate an existing feature layer from a GeoDataFrame (provided the fields match of course).  I tried all sorts of methods to create a feature layer directly using GeoJSON as the intermediary  - no luck. So then I tried to insert values (attributes and geometry) from a GeoDataFrame into an existing feature layer in AGOL, using GeoDataFrame.to_dict and then trying to add that dictionary to the feature layer using edit_features(adds=<my dictionary created from the GeoDataFrame>). No luck there either.  Anyone knows how this can be done?  Thanks Hugo  ... View more

Hi  At the moment I have a python script to insert, update or delete features from a feature layer. This works well, but I have to upload the feature class containing the new records first to AGOL and publish it as a feature layer. This is because in order to update an existing feature layer, I have to create a deepcopy from the layer containing the new records, like so: Say my layer (with the new records) is named "updater_layer". I do a query on it, using a where clause : updater_layer_query = updater_layer.query(where=where_clause) # I get the features:  update_query_features = updater_layer_query.features # and create a deepcopy which I can use to update an existing layer new_features = deepcopy(update_query_features)   But to get those deepcopy records, I first have to upload and publish a new layer with new records to AGOL.  I am wondering if there is a way to do this on the desktop so that I end up with a deepcopy list from the feature class on the desktop, without having to upload the feature class first to AGOL as a feature layer. That deepcopy (derived from the feature class on the desktop) can then directly be used to update an existing feature layer in AGOL. I looked but it does not seem possible to create a deepcopy from a feature class. Is there a way to do this, perhaps via a JSON representation of the feature class or perhaps by using a searchcursor on the feature class?    Thanks Hugo  ... View more