I work for a utilities company. All of our assets have unique IDs & names. Someone has asked me to create a heat map to show the location of our assets that have certain priorities of defects listed against them. They've sent the defect file through in an excel spreadsheet, and unfortunately there are some instances where one asset has 4 defects listed against it, each it's own separate line in excel.
Is there an easy way to duplicate only those points that have more than one defect listed against them, as some have two priority 3's and two priority 2's. So they would need to be displayed twice for each priority heat map.
Any suggestions, other than doing it manually (over 800 records), would be appreciated.
Did you see?
Find Identical (Data Management)—ArcGIS Pro | Documentation
You would have to narrow down the search to the key that was duplicated (geometry or some id perhaps)
@ElaineCollinson
The Frequency geoprocessing tool might be useful. https://pro.arcgis.com/en/pro-app/latest/tool-reference/analysis/frequency.htm
You'll get a derivative table containing the number of records per type/criteria.
This can then be run through a copy cursor to create a new table with the number of points matching the number of occurrences of each instance.
Below was something I used a while ago. I'm not a coder and you can probs do it many other ways. This runs in a Notebook.
Points_A = points dataset with output of Frequency added to a field called Frequency.
It creates a new table, then adds points to it based on the count from Frequency.
You'll need to find a way to join up your records table with the resulting points so you can do the various thematics but this should give you a nice rabbit hole to explore.
import arcpy
import os
# Inputs
input_fc = r"C:\Data.gdb\Points_A"
output_fc = r"C:\Data.gdb\Points_A_Count"
frequency_field = "Frequency" # count field
# Split path and name correctly because had issues
out_path, out_name = os.path.split(output_fc)
# Create output feature class using input schema
arcpy.management.CreateFeatureclass(
out_path=out_path, #issues
out_name=out_name, #issues
geometry_type="POINT",
template=input_fc,
spatial_reference=input_fc
)
# Get needed fields
fields = [f.name for f in arcpy.ListFields(input_fc) if f.type not in ("OID", "Geometry")]
# Add geometry
fields.append("SHAPE@XY")
with arcpy.da.SearchCursor(input_fc, fields + [frequency_field]) as search_cursor, \
arcpy.da.InsertCursor(output_fc, fields) as insert_cursor:
for row in search_cursor:
freq = row[-1] # Frequency is the last field in the cursor
freq = int(freq) if freq else 1 # sanity check input freq, 1 if dodgy
insert_row = row[:-1]
for _ in range(freq):
insert_cursor.insertRow(insert_row)
print(f"Dunny: {output_fc}")
If you perform a temporary join between your base point location files (one point per location) and the defect spreadsheet, using the Unique ID as the join field, then the attribute table will create a duplicate record every time that more than one join option occurs. You can then right click the location layer and choose "Export Features" from the data menu to create a brand new point feature class with multiple points everywhere that more than one defect is listed.
@RichardHowe You're 100% correct - when it works 🤣
I've had so many instances where Pro (3.x) just doesn't want to make a join for some reason that I ended up doing things the hard way. Sometimes a restart will make it join properly, other times nada.
Often I can do the same join in ArcMap and it works - but then I don't have all the pretty symbologies......
A benefit of running Frequency is that you then have a quantitative value you can also use as a 'proper' indicator in a regular symbology; sometimes people want to compare two different periods or datasets and heatmaps are hard to make comparable much more than 'here bad, there good' 🙂
Check out using a Pivot Table in excel to see which have more than 1 occurrence.
Thank you all for the wonderful tips and feedback.
I realized I was overthinking it. I got the excel spreadsheet and used FME to link to our asset database and populated the excel spreadsheet with the coordinates of the assets. Put the spreadsheet into Pro and created points using the x, y.