nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

select duplicate records in FGDB

JustinWolff

I understand I cannot use a SQL query to select multiple records that have duplicate values within a FGDB (only personal or SDE). Does anyone have a workaround? My goal is to select features that have duplicate attribute values in a particular field and merge them together to create multipart polylines...then delete the original, separate features and replace them with the new, multipart features. (I have to do this for a point feature class as well, but I know that will take additional effort to create a multipoint feature class and integration with single point feature class is difficult.)

Right now I'm stuck on go just trying to select the duplicate records.

Thanks

Justin

Find more posts tagged with

Accepted answers

bixb0012

I think this is more a Python‌ question.

Have you looked into using Find Identical—Help | ArcGIS Desktop ? Does it not work for your situation?

All comments

Ipad1Avion

Also, exporting to Excel works well to highlight duplicates on a non-advanced license.

bixb0012

Although the reason for asking the question is being addressed in a different thread, merge features with duplicate attributes in FGDB , I thought I would share there is a fairly simple way to select distinct, duplicate, triplicate, etc... records in a layer without having to create an additional table.

The following code snippet will select non-distinct records in a layer/table view based on comparison/selection fields.

<SPAN class="keyword token">def</SPAN> <SPAN class="token function">select_plicates_layer</SPAN><SPAN class="punctuation token">(</SPAN>layer<SPAN class="punctuation token">,</SPAN> selection_fields<SPAN class="punctuation token">,</SPAN> value<SPAN class="operator token">=</SPAN><SPAN class="number token">2</SPAN><SPAN class="punctuation token">,</SPAN> condition<SPAN class="operator token">=</SPAN><SPAN class="string token">'ge'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">import</SPAN> operator <SPAN class="keyword token">as</SPAN> operator
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">from</SPAN> collections <SPAN class="keyword token">import</SPAN> defaultdict
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> isinstance<SPAN class="punctuation token">(</SPAN>selection_fields<SPAN class="punctuation token">,</SPAN> basestring<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; selection_fields <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>selection_fields<SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; plicates <SPAN class="operator token">=</SPAN> defaultdict<SPAN class="punctuation token">(</SPAN>list<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">with</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>SearchCursor<SPAN class="punctuation token">(</SPAN>layer<SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="string token">"OID@"</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">+</SPAN> selection_fields<SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">as</SPAN> cursor<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> cursor<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; plicates<SPAN class="punctuation token">[</SPAN>row<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">+=</SPAN>&nbsp; row<SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; condition_oids <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>oids 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> oids 
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">in</SPAN> plicates<SPAN class="punctuation token">.</SPAN>values<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> getattr<SPAN class="punctuation token">(</SPAN>operator<SPAN class="punctuation token">,</SPAN> condition<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">(</SPAN>len<SPAN class="punctuation token">(</SPAN>oids<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> value<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> condition_oids<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; oid_list <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>oid <SPAN class="keyword token">for</SPAN> oids <SPAN class="keyword token">in</SPAN> condition_oids <SPAN class="keyword token">for</SPAN> oid <SPAN class="keyword token">in</SPAN> oids<SPAN class="punctuation token">]</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; layer<SPAN class="punctuation token">.</SPAN>setSelectionSet<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"NEW"</SPAN><SPAN class="punctuation token">,</SPAN> oid_list<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; 
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">return</SPAN> condition_oids<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

The real value of the function is less what it selects and the results it returns, i.e., the function returns a list of OID lists. Each OID list contains the OIDs of records that share the same values for the selection_fields. The list of lists can be used to determine how many duplicates, triplicates, quadruplicates, etc... are in the layer and what the makeup of them looks like.

The code is designed to operate on layers, so any existing selection of a layer is honored. The code operators more in a "SUBSET_SELECTION" manner than a "NEW_SELECTION" manner.

JustinWolff

Thanks Joe. I've wanted to avoid adding a new field, and just use the selected, duplicates in a geoprocessing action. But, a new field may be what it takes.

JustinWolff

Yes, thanks - i meant to put this in Python, not the API

JoeBorgione

I've used this to find duplicates: Find duplicate field values in ArcGIS using Python | Don Meltz

bixb0012

I think this is more a Python‌ question.

Have you looked into using Find Identical—Help | ArcGIS Desktop ? Does it not work for your situation?

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.