auto-generate a class breaks renderer for a layer

davedoesgis · ‎02-01-2022

I have a hosted feature layer with a LOT of numeric fields in it. I started an AGOL webmap with the hosted feature layer. It is drawn using its default symbol - shaded by total population. What I'd like to do is clone that layer a bunch of times for the various fields.

I won't have time to go through and determine the best classification method and color ramp for each field's layer, but I just want to quickly stub something in to at least have a quick and dirty starting point. I'm thinking something like what I can do in AGOL by clicking on the highlighted parts of this screenshot:

I found the doc for arcgis.mapping.renderer.generate_renderer(), which looks like what I want, but I still have some missing pieces and OF COURSE there are no samples in the documentation!!! Are there any samples available for this?

Also, what can I use for the 'sdf_or_series' argument? I create a FeatureLayer object that works with WebMap.add_layer(). Will that work?

from arcgis.features import FeatureLayer
county_hfl_url = 'https://services.arcgis.com/[truncated]/FeatureServer/0'
county_fl = FeatureLayer(county_hfl_url)

I can write some code to figure out the min/max values in each field I guess, but again, I just want to throw the data at a default renderer and run with that, so anything to keep this simple is appreciated.

davedoesgis · ‎04-20-2022

[EDIT: After marking this as the accepted solution, GeoNet creates a copy of it at the top, but I recommend reading this odyssey, in order, below. I reference several things in previous replies that won't make sense if you read it out of order.]

This hack is basically where I landed. It's super frustrating that the ArcGIS API is broken and you need to bring in desktop software as a work-around, but that's what I resorted to.

I don't have time to create a working sample, but I tried to extract all the important pieces out of the project I was working on. Please understand that any code below is just bread crumbs pseudo-code and you will need to fiddle with it.

In an ArcGIS Pro Notebook, you can get a display of all the valid colormaps and their names from AGOL/Portal:

from arcgis.mapping import symbol, display_colormaps
display_colormaps(symbol.ALLOWED_CMAPS)

Have an offline copy of your data in an APRX map. Use ArcPy to create a variable lyr for your layer (this is ArcPy 101, lots of samples online if you don't know how to find a map layer). Then, follow the steps above with the sym variable to create a renderer with ArcPy and assign it back to your layer. Modify as desired if you want equal interval or more/fewer class breaks, etc. ArcPy docs on the topic here.

Once you have the ArcPy renderer assigned to your layer, you can create a list of the upper boundaries like this:

arcpy_breaks = [x.upperBound for x in lyr.symbology.renderer.classBreaks]

(Or probably just get them directly from the sym variable, but that wasn't my workflow.)

Then, I generate a renderer with the arcgis API. It is borked, as described in the original question, but we'll fix it with the ArcPy legend. Another dependency between the local copy in Pro and the ArcGIS API is the sdf variable, which I set in my code sample above using ArcPy and a definition query. Since we're just creating an interim renderer, I wouldn't get too hung up on perfecting the definition query, but I think you need something for this argument.

renderer = arcgis.mapping.renderer.generate_renderer(
    geometry_type="polygon", 
    sdf_or_series=sdf,
    label=<your_label>, 
    render_type="c", 
    method="esriClassifyNaturalBreaks", 
    field=<your_field>, 
    min_value=0, 
    class_count=6, # must match what you used in ArcPy
    colors=<color_scheme>)

# At this stage, I recommend you implement some checks that both the
# arcpy and arcgis renderers have the same number of classes. Data sets
# with small sample sizes broke my ability to create a renderer with 6
# classes, for example.

for i in range(len(renderer['classBreakInfos'])):
    cb = renderer['classBreakInfos'][i]
    cb['classMaxValue'] = arcpy_breaks[i]
    cb['label'] = arcpy_breaks[i]
    cb['description'] = arcpy_breaks[i]
    
    # Fix the default symbol, which ends up fat and light gray
    cb['symbol']['outline']['width'] = 0.75
    cb['symbol']['outline']['color'][0] = 50 # red
    cb['symbol']['outline']['color'][1] = 50 # green
    cb['symbol']['outline']['color'][2] = 50 # blue

Once you I have my renderer, here is my workflow to create a new layer in my webmap:

# Get the webmap
webmap_item = gis.content.get(webmap_item_id)
webmap = WebMap(webmap_item)

# This is the layer I want to symbolize and add to the webmap
template_item = gis.content.get(<layer_item_id>)
feature_layer = [x for x in template_item.layers 
                 if x.properties.name == <title>][0]

options_dict = {'title': title_out, 'visibility':False}
options_dict['opacity'] = opacity # decimal from 0-1?
options_dict['renderer'] = renderer
webmap.add_layer(feature_layer, options_dict)
webmap.update()

Lastly, while it's not exactly pertinent to this topic of legends, but fits in the theme of stuff broken in the ArcGIS API for Python, I should mention the following: There are several properties that could go in the options_dict above, but they don't seem to do anything. I have to find the webmap layer after it's created (via webmap.update()) and set them.

for wm_layer in webmap.layers:
    if wm_layer.title != title_out: continue
    wm_layer['layerDefinition']['definitionExpression'
                  ] = <def_query>
    # also set wm_layer['layerDefinition']['minScale'] and such
webmap.update()

Please expect that this pseudo-code won't execute without some tinkering, but it should be close. I'm trying to extract it from a giant workflow custom to my org and share back to the community, but don't have more time to test it.

View solution in original post

davedoesgis · ‎02-02-2022

I made some progress.

I create a FeatureLayer from the hosted feature layer on AGOL
I create a spatially enabled data frame from a local FGDB copy of my data
I use that SDF to create a renderers for equal interval, natural breaks, quantiles, and std dev.
I published that same FeatureLayer four times, once for each different renderer.

This code below runs just fine, assuming you already have an arcgis.gis.GIS object created, but the results are not what I expected.

import arcpy
import arcgis
from arcgis.features import FeatureLayer
from arcgis.mapping import WebMap
import pandas as pd
county_hfl_url = 'https://services.arcgis.com/<redacted>/FeatureServer/0'
county_fl = FeatureLayer(county_hfl_url)
out_item = gis.content.get('<item_id>') # webmap to update 
out_wm = WebMap(out_item)
fc_path = r"D:\Path\to\fgdb.gdb\County"
sdf = pd.DataFrame.spatial.from_featureclass(fc_path, 
            where_clause="State_FIPS = '53'")
ren_q = arcgis.mapping.renderer.generate_renderer(geometry_type="polygon", 
            sdf_or_series=sdf, label="Poverty", render_type="c", 
            method="esriClassifyQuantile", field="Poverty", 
            min_value=0, class_count=6, colors="Blues")
out_wm.add_layer(county_fl, {'title': 'Quantiles 6', 'opacity': 0.5, 
            'visibility': False, 'renderer': ren_q})

ren_nb = arcgis.mapping.renderer.generate_renderer(geometry_type="polygon", 
            sdf_or_series=sdf, label="Poverty", render_type="c", 
            method="esriClassifyNaturalBreaks", field="Poverty", 
            min_value=0, class_count=6, colors="Blues")
out_wm.add_layer(county_fl, {'title': 'test natural breaks 6', 
            'opacity': 0.5, 'visibility': False, 'renderer': ren_nb})

ren_ei = arcgis.mapping.renderer.generate_renderer(geometry_type="polygon", 
            sdf_or_series=sdf, label="Poverty", render_type="c", 
            method="esriClassifyEqualInterval", field="Poverty", 
            min_value=0, class_count=6, colors="Blues")
out_wm.add_layer(county_fl, {'title': 'test equal interval 6', 
            'opacity': 0.5, 'visibility': False, 'renderer': ren_ei})

ren_sd = arcgis.mapping.renderer.generate_renderer(geometry_type="polygon", 
            sdf_or_series=sdf, label="Poverty", render_type="c", 
            method="esriClassifyStandardDeviation", 
            field="Poverty", min_value=0, class_count=6, colors="Blues")
out_wm.add_layer(county_fl, {'title': 'test standard deviation 6', 
            'opacity': 0.5, 'visibility': False, 'renderer': ren_sd})

out_wm.update()

First, let me show you the legend from the layer created via the equal interval renderer. The interval is about 33,700. Everything looks kosher till I get to the top category, which has the same min/max values, that also match the max value of the category below that. I probably could fix this with code, but the values in this data set are really skewed by a few really large counties, so equal interval is really not my preference.

Next, I noticed that I get the exact same result (a broken equal interval classification) from the renderers that used other classification methods. Here are the legends for the natural breaks and quantile class breaks layers. The result is the same as the equal interval! Same for the standard deviation (not shown, but you get the idea).

I've gotten it to where I'm successfully going through all the steps and the code runs ok, but the result is definitely not right. Any ideas?

Anyone from Esri on these forums? (Random attempt to tag humans: @Anonymous User @myesri @esrirk @EsriTG @EsriSF @KPEsri @EsriEsri4 @esriasg @sgresri @esrigrk @cc_esri @ESRI___ @ESRIGIS @ELP_Esri )

davedoesgis · ‎02-15-2022

I'm still flailing on this. Anyone at Esri on these forums? We have a bug.

Anyhow, my hack work-around starts with my previous reply, where I talk about generating a renderer for Portal/AGOL using arcgis.mapping.renderer.generate_renderer(). Go read that first. That renderer is messed up, but it serves as a shell for what you want.

You will want to run this in a Pro or Jupyter Notebook to see the color ramp options for Portal/AGOL.

from arcgis.mapping import symbol, display_colormaps
display_colormaps(symbol.ALLOWED_CMAPS)

Next, I have an identical feature class in an FGDB locally and have that as a layer in an APRX. I then use ArcPy to read my APRX, find the layer (variable lyr) and classify it like this:

sym = lyr.symbology
sym.renderer.classificationField = field
sym.updateRenderer('GraduatedColorsRenderer')
sym.renderer.classificationMethod = 'Quantile'
sym.renderer.breakCount = 6
sym.renderer.colorRamp = aprx.listColorRamps('Cyan to Purple')[0]
lyr.symbology = sym  # re-run with any change

Then, within that lyr object, look for lyr.symbology.renderer.classBreaks.upperBound for the class breaks. You can also retrieve the class break labels, too. Once I have the class break from my APRX's symbology, I copy those into my broken renderer dict from generate_renderer().

As I mentioned above, I just use the color schemes from AGOL/Portal in the arcgis API. If you want to use the colors from Pro, here is how I get the color ramp names for the colorRamp property. I'm sure there's a better way, but this works.

davedoesgis · ‎04-20-2022