Symbology-Add All Possible Values

[KevinJohnson]

The add values button on the symbology tab in layer properties adds only values already existing in your database.  People have to constantly add new values to their existing symbology as new value combinations get entered.  This impacts people who have published map services especailly, constantly having to republish service when ever a new combination of values get entered.  The new button "Add All Possible Values"  would add all permutations of your value fields allowing the user to set symbology for all combinations, even those that do not exist yet.   Thanks.

[DanaNolan]

This also affects people who create layers for others. When you cannot change the underlying data, it is harder to work around this problem by, for example, converting part or all of an alphanumeric field to numeric so you can use range-based symbolizations. Road types often have a letter followed by a potentially large number of numer values. This requires adding all these alpha values, which will likely change when the next version of the roads comes out. I have also noticed that sometimes not all values get added; this could be a software problem but it is also another wrinkle in this situation. You have to think about the dataset that has the widest range of values and even add values not present in that dataset when setting up a layer. Otherwise, if you try to import symbology into a sister layer, for example, another region, the added values won't be the same and adding them in again blows away your symbology unless you add them one by one (which is error-prone).

It is also disturbing to see the SQL produced by some of these Add all values operations (try importing it to the label classes to see it). It is not efficient. The SQL is very literal and hyperspecific, which is why you have to keep adding new value combinations, because it says Select myfield = x or myfield = x2, ad nauseum.
What I  enter in label classes is more like Select myfield >= x and myfield <=x99, which would presumably work across data changes.  I guess I am asking for a SQL box for Symbology, which is probably another idea on its own.

[RichardFairhurst]

Look at the ArcGIS help for Symbol Manager.  For single field symbology codes you are describing what the Symbol Manager does.  I think it has existed since 9.0, but I know that it exiists in 9.3,

Within Symbology Manager you can create your own symbol set that acts just like the ESRI symbol sets for Transportion, Mining, Hydrology, etc.  The symbols are given names that should match the code values you currently use or can possibly foresee using.  The same symbol can be used under many different code names for cases where there is more than one way to code the same symbol (a form of grouping outside a layer or a way of handling misspellings in the data sources before tehy are corrected).  You can define symbols and codes that have never been used by you or anyone else before and they will be permanently available for whenever the code might actually get used.

The symbols remain independent of any field name used in a layer that actually stores the matching symbol code values.  As long as the field chosen in a layer as a Category field stores a value that matches a Symbol name you have defined, the symbols from your symbol set will be quickly matched to the field values.   There is a Category option to load the symbols from a chosen symbology set to a layer.  The load respects query definitions on the layer and only populates relavant entries to the layer (which keeps your map legends managable without the clutter of a huge set of unused values).  When a new code appears make sure you have a matching name in your symbol set and rerun the layer Category option.  Within about 2 seconds, the stored symbol will be added to the layer.  This is better than making a huge symbol set on a layer and trying to manage all possible code through it.

The only limitation is that multi-field codes are not supported.  If you have them you will have to add a field to store the concatenated values which you then would match in your symbol set.  If you cannot alter the original data source for the layer to implement the concatenated field, you will need to use a joined table storing the concatenated values.  This definitely introduces some maintenance problems.  You will not be able to maintain a live symbol connection for the data editor on new rows and chages to the multi-field values will not be immediately reflected in your layer.  The editor would need to be trained to maintain the join table so they can operate with the symbols on the layer more or less as they edit, and should be provided with a modelbuilder model to assist them with updating the join table.  The model builder tool could also create the joinable table to SDE through Windows Scheduler and synchorized for non-work hours to be served concurrenly with your end user released data.  I have not looked at exporting the symbol set to another user or outside company, but I believe you can to try to get some of the work done on the client side.  However, trying to support outside users who are not trained to do even the simplest operations with a layer will never be an easy task.

I hope this helps.

[RichardFairhurst]

Another comment about using Symbol Maneger to help create and maintain the everything and the kitchen sink layer for an end user that needs everything imaginable.  Perform a disolve of your feature class on the symbol Code field   Choose one feature in the disolved layer and make a copy of it.  Blank out the code and copy that feature for as many additional features as you need to store all of your possible codes.  Then edit the uncoded features to store each additional imaginable code value you can think of.  When finished match the code to the Category of your dummmy feature class layer to your symbol set you have createdd with Symbol Manager.  Import the symbology from the dummy layer into your end user layer.  Ship it to the end user.

Periodically, the dummy feature class can be joined or related to the actual data source to check for new codes you need to account for that you failed to think of.  A Dissolve of the selection set of new coded values can be pasted in your dummy feature class.  Also add any more code you may for see with the secondary copy method described above.  Add the symbols listed in the new Dissolve and you other new codes into Symbol Manager and rerun the category symbol match.  Import the dummy layer symbology to your end user layer so that all used and potentially used values are available in it.  Ship the new end user layer to the end user.

Nobody can imagine every possible code that can be generated in the world, including you, me or ESRI.  You know more about your domain than they ever will, but with Symbol Manager and the above dummy feature class you should be able to maintain something that works for you.

[RichardFairhurst]

What I was actually referring to is called Style Manager, not Symbol manager and it deals with a lot more than Symboloogy.  However, I realize now that you are using the Many-Field Category symbology.  I understand the time savings potential with your suggestion, but you still have a manual way to add unused values using the "Add Values...".  button to open the "Add Values" dialog.

Under the "New Value" text box at the bottom of the dialog you can type (or paste and modify) new values that are not currently in your data.  Values you want to add need to be formatted the same existing values. 
By that I mean that they must follow the field order and values from the second and third field need to be separated from the preceeding fields values by a comma followed by a space.  When you press the "Add to List button, the value you entered will be added to the "Select the value(s) to add:" list box.  When you have entered all of the values you need, you can select them all and add them to the layer symbology.  The new symbols labels will match what you typed.  If you using a Domain value, you will have to retype the labels to translate to the Domain values.  Once this is done do not use the "Add All Values button again.  Instead use the Add Vallues... button get the Complete list and add all of the values that appear that you failed to think of.

But be warned, these kinds of layers eventually will cause a huge performance hit as your data grows.  I have large data sources (up to 3/4 million records) and find the performance hit is just too great using multi-field values and adding every symbol, including unused symbols, into a single layer.  The performance sucks because I discovered that the source table records are being repeatedly read from top to bottom for each symbol in the list, even if they are only used once or are never used.  When I use a single layer like the one you want with 30 complex line symbols on data sources like this I have experienced normal redraw times of 5 minutes pre refresh.  Totally unlivable.

To regain redraw speed I have had to create a group layer that containing as many layers of the same source to draw each symbol in its own layer.  Each individually symbolized layer has a definition query that limits the records for each layer to just those that apply to the symbol displayed.  (I also create a catch all layer just for finding new symbol values, but I never draw it).  Because each layer only reads records that actually use the single symbol for the layer, I have improved my redraw performance in such cases from 5 minutes per refresh to under 20 seconds.  Complex multi-part line symbols unavoidably cause a hit all by themselves, because each symbol part causes the table records to be reread from top to bottom, but by preliming the records to read just those that apply I minimize that hit as much as possible.  So I have abandoned all single layer symbologies like the kind you want whenever I have many symbols associated with over 10,000 records.  As a result, I just almost never use kitchen sink layers any more.

[KevinJohnson]

rfairhur24 -"I understand the time savings potential with your suggestion, but you still have a manual way to add unused values using the "Add Values...".  button to open the "Add Values" dialog."

Thats existing functionality.  Yes you could type in by hand every possible unused permutation of your symbolized fields, but that is not  practical.  Yes I agree that trying to symbolize each value differently would be a huge performance hit, but If you use grouping, its not that bad. 

A button to add all possible values would be a huge time saver, especially for people who have map services built using symbology, who expect new combinations of symbolized fields to get entered in the future.  

[MikeHelten]

I've run into the same problem- you add a feature with a not-before-seen attribute combination, and it appears as one of those nasty "Other" symbols. I took a half hour to put together a crude python snippet so I can at least copy and paste missing domain combo values into "Add Values" instead of hand typing. It's not a great solution, but it's saved me some time.

Notes:

- Fill in the first two lines, make sure the domains are in the order you want the string to appear.

- If you want to include <Null> as a possibility, uncomment the last line in the list domains part.

>>> arcpy.env.workspace = MY GEODATABASE
>>> domainSelection = [DOMAIN 1, DOMAIN 2, DOMAIN 3....]

>>> domainValuesLists = [None] * len(domainSelection)

>>> for domain in arcpy.da.ListDomains():
...     if domain.name in domainSelection:
...         domainSelectionIndex = domainSelection.index(domain.name)
...         domainValuesLists[domainSelectionIndex] = list(domain.codedValues.keys())
...         ### domainValuesLists[domainSelectionIndex] += ["<Null>"]

...
>>> def recurseCodedValues(listsOfValues, entryString):
...     listOfValues = listsOfValues[0]
...     for value in listOfValues:
...         newEntryString = entryString + ", " + str(value)
...         try:
...             recurseCodedValues(listsOfValues[1:], newEntryString)
...         except:
...             print newEntryString.strip(",").strip()
...
>>> recurseCodedValues(domainValuesLists, "")

An few example lines of the output for 3 domains:

Private, Type 2 / MH (no sump), Basin-32a
Private, Type 2 / MH (no sump), Basin-32b
PSE, Inlet, Unknown
PSE, Inlet, Basin-13a