topic Turn Off Fields with Python in Python Questions

Turn Off Fields with Python

DylanHarwell — Tue, 02 Apr 2019 16:57:27 GMT

First of all, this is not a duplicate thread as I am not convinced that the one below has been resolved..at least the suggestions and code does not seem to work for me.

Turning fields on and off using arcpy

Another on stackexchange on this topic addresses this in ArcMap, but since ESRI decided to get rid of the mapping module and create the mp module for Pro, it doesn't solve the problem either.

https://gis.stackexchange.com/questions/230017/using-fieldinfo-setvisible-in-arcpy

We have a Pro project that houses our web maps for our organization. Recently we turned on editor tracking fields for layers coming from a specific sde, and now we need to hide those fields so they don't display in the web map pop-ups. To get an idea of the scale of this task, there are over a thousand layers that need these fields turned off. I have been scripting in Python for ArcMap for several years, but am relatively new to Pro and Python 3 and the new arcpy.mp module. The example code in the above threads uses arcpy.mapping and looks fairly straightforward - the arcpy.mapping.UpdateLayer() function does the hard work. Well this does not exist in arcpy.mp and the workaround I have seen (and suggested by esristaff) goes like this:

- create a fieldinfo describe object on the layer of interest

- loop through fieldinfo list and using setVisible hide desired fields

- make feature layer and pass in fieldinfo

- save as layer file

- replace old layer in map with new layer (plus a bunch of steps to preserve order and symbology)

This is all straightforward to me and I am not having issues structuring the code. The issue is that the temp layer created from MakeFeatureLayer is not honoring the new field info. I have tested this interactively in the Python window in Pro:

fieldInfo = arcpy.Describe(lyr).fieldInfo

Printing this shows the fields I need to turn off as visible. Then looping through and turning them off:

for i in range(0, field_info.count):

if field_info.getFieldName(i) in fields_to_turn_off:

field_info.setVisible(i, "HIDDEN")

Printing field_info now shows the fields I wanted off to be hidden. Great. But then I make a feature layer passing this field_info into it, it shows up in my TOC, I open the fields window on it and all the fields are still on!

The example code from the previous threads stops there, but to me it seems like either the field_info object is not being 'saved' or the MakeFeatureLayer function is not honoring it as a parameter. ESRI's documentation on fieldInfo and everything else is minimal at best, and I have not come up with anything else helpful on Google.

Here is an overly complicated workaround I have gotten to work.

- Save layer of interest as a layer file

- Then on that layer file create the describe object and turn fields off

- Then make feature layer passing in the new field info

- Then save as another layer file

Kind of ridiculous.

Re: Turn Off Fields with Python

JoeBorgione — Wed, 03 Apr 2019 18:23:40 GMT

Kind of ridiculous. Yep. Couldn't agree more. I've never been able to to use arcpy to turn off a field.....

Re: Turn Off Fields with Python

MarisaClaggett — Wed, 03 Apr 2019 19:51:55 GMT

Hi Dylan,

I just used your sample script and was able to hide fields in the feature layer using the updated field info. What version of Pro are you using?

Best of luck,

Marisa

Re: Turn Off Fields with Python

DylanHarwell — Wed, 03 Apr 2019 19:55:06 GMT

2.3.2

Re: Turn Off Fields with Python

MarisaClaggett — Wed, 03 Apr 2019 20:05:26 GMT

I was working in Pro 2.3.1, and upgraded to Pro 2.3.2. I was still able to hide fields successfully with the feature layer. You mentioned that you were using data coming from an sde database. Can you export a feature class, and or try on a feature class coming from a file geodatabase?

Marisa

Re: Turn Off Fields with Python

DylanHarwell — Wed, 03 Apr 2019 20:22:31 GMT

Interesting idea.. since the workaround I mentioned I first saved a layer file, then turned fields off, then made feature layer and saved a new layer file.

Guess what! Fields were off after doing it on the File GDB version of the data. To ensure I was not going crazy, I did the same thing on another layer in the project coming from SDE and sure enough, the output feature layer still had those fields turned on. Attached is the code I used in the Pro Python window.

Re: Turn Off Fields with Python

MarisaClaggett — Wed, 03 Apr 2019 20:45:49 GMT

Glad to hear that it worked!

Re: Turn Off Fields with Python

DylanHarwell — Wed, 03 Apr 2019 20:53:10 GMT

Still does not solve the problem though.. As I stated we have over a thousand layers (all in SDE) that needs these fields turned off. It's completely impractical to have to export each one to a file geodatabase, then do all these steps to turn off the fields (which is already impractical as is) then export them all back into SDE - which I think would require a delete/append as removing and adding feature classes breaks our replication setup.

The fact that it only works on file gdbs, is that not a bug or at least a flaw/limitation that should be looked into by ESRI?

Re: Turn Off Fields with Python

MarisaClaggett — Wed, 03 Apr 2019 22:04:26 GMT

I apologize, I thought you said that other SDE feature layers had those fields turned off. I misunderstood. I attempted this workflow with my own SDE database and cannot reproduce your issue. I recommend that you contact Esri Technical Support for a more in depth investigation.

https://support.esri.com/en/contact-tech-support

Best of luck,

Marisa

Re: Turn Off Fields with Python

DylanHarwell — Thu, 04 Apr 2019 16:29:50 GMT

Hmmm.. wonder if it has something to do with the feature classes coming from our downstream replica database? The way our organization is set up, we have a parent SDE database and a replica of that which all our webmaps point to. I imagine when you tested this you were working in a single sde and not a replica gdb.

I just tested this using the same workflow on the same building footprints layer, 1 coming from the replica and 2 coming from the parent database, and guess what? Fields are off on the parent sde feature class and not on the replica! Can you try and reproduce this on a replica database please? Is there some reason this should be designed to not work on the replica?

I used the same workflow as before: create a field info describe object on the layer, loop through setting the desired fields to hidden, then make a feature layer passing in the new field info.

Re: Turn Off Fields with Python

DylanHarwell — Sat, 11 Dec 2021 18:23:16 GMT

After lots and lots of testing, I imagine this is a bug and will be opening a case with tech support. If this was built into the design of arcpy, there is no reason I can think of that the FieldInfo properties should not be honored on data coming from a replica database since they are properties of the layer in the map/project and not properties of the data in the database.

I did, however, manage to figure out a workaround, cobbled together with someone else's workaround for preserving the symbology, and my own hack of the JSON code in the lyrx file to preserve other properties of our maps. The solution is rather ridiculous and overly complex for what should be such a simple task of turning off fields. I hope ESRI and the Python community can see the humor in this code

# Import modules and set workspace
import arcpy, json, os
workspace = "C:\\...\\WebLayers_COPY"
arcpy.env.workspace = workspace
arcpy.env.overwriteOutput = True

# Set up folder paths and fields to turn off
layer_folder = "C:\\...\\TempLayers"
proj_path = "C:\\...\\webLayersArcGISAPP_COPY2.aprx"
fields_to_turn_off = ['created_user', 'created_date', 
                      'last_edited_user', 'last_edited_date']

# Create project object
project = arcpy.mp.ArcGISProject(proj_path)

# List maps in project and layers in maps
maps = project.listMaps()

# Initiate loop through maps and layers
for map in maps:
    layers = map.listLayers()

    for lyr in layers:
        
        # Pass over group layers
        if lyr.isGroupLayer:
            pass
        # Pass over any data not coming from gjgisprod.sde
        elif 'gjgisprod' not in lyr.dataSource:
            pass
        # Pass over any data using Parcels
        elif 'Parcel' in lyr.dataSource:
            pass
        # Begin absurd process
        else:
            # Get field info, loop through and turn off desired fields
            field_info = arcpy.Describe(lyr).fieldInfo
            for i in range(0, field_info.count):
                if field_info.getFieldName(i) in fields_to_turn_off:
                    field_info.setVisible(i, "HIDDEN")
    
            # Make feature layer, save to lyr file, make feature layer with new
            # field info, apply symbology, save to lyr file again
            # --dumb workaround: fieldinfo not honored on replica data 
            # --must make feature layer and save as lyr twice for it to work
            # --dumb symbology/naming workarounds
            name = lyr.name        
            out_lyr_path = os.path.join(layer_folder, (name + "_1.lyrx"))        
            arcpy.MakeFeatureLayer_management(lyr, "temp_lyr", "", "", field_info)        
            arcpy.SaveToLayerFile_management("temp_lyr", out_lyr_path)
            
            arcpy.MakeFeatureLayer_management(out_lyr_path, "temp_lyr2",
                                              "", "", field_info)        
            arcpy.ApplySymbologyFromLayer_management("temp_lyr2", lyr, "", "MAINTAIN")        
            new_lyr_path = os.path.join(layer_folder, (name + ".lyrx"))        
            arcpy.SaveToLayerFile_management("temp_lyr2", new_lyr_path)
            arcpy.Delete_management(out_lyr_path) 
            
            new_lyr_file = arcpy.mp.LayerFile(new_lyr_path)        
            new_lyr = new_lyr_file.listLayers()[0]        
            new_lyr.updateConnectionProperties(new_lyr.connectionProperties, 
                                               lyr.connectionProperties)
    
            # Check leyer properties and preserve
            if not lyr.visible:
                new_lyr.visible = False
            if lyr.transparency:
                new_lyr.transparency = lyr.transparency
            if lyr.minThreshold:
                new_lyr.minThreshold = lyr.minThreshold
            if lyr.maxThreshold:
                new_lyr.maxThreshold = lyr.maxThreshold
    
            new_lyr.name = name
            new_lyr_file.save()
        
            # Hack into lyrx JSON code and collapse layer and disable popups
            with open(new_lyr_path) as json_lyr:
                data = json.load(json_lyr)
                data['layerDefinitions'][0]['expanded'] = False
                data['layerDefinitions'][0]['showPopups'] = False
            with open(new_lyr_path, "w") as json_lyr:
                json.dump(data, json_lyr, indent=4)

            # Must delete lyr and create another for the JSON changes
            del new_lyr_file
            new_new_lyr = arcpy.mp.LayerFile(new_lyr_path)

            # Finally, switch out layers in map
            map.insertLayer(lyr, new_new_lyr, "AFTER")
            map.removeLayer(lyr)

# Save project and clean up (still does not remove locks for some reason..) 
project.save()
del project
del workspace‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Turn Off Fields with Python

JoeBorgione — Fri, 12 Apr 2019 20:12:52 GMT

Anyone not amused by your comment lines lives an a bubble...

Re: Turn Off Fields with Python

Oliver_Sandoval_e — Thu, 25 May 2023 01:11:39 GMT

Thank you so much Dylan! I'm dealing with dozens of layers and Cityworks configuration. You've saved me and I'm sure many others from existential panic!

Thanks again!

Oliver

Re: Turn Off Fields with Python

HBrowning — Tue, 05 Aug 2025 22:10:46 GMT

This was so helpful, thank you for sharing it @DylanHarwell