nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How can I randomly select a number of point features and retain their attributes?

DarinJensen

I have >1m points in California and I want to randomly select ~10% and retain their attributes. "Create random points" doesn't work because it is creating, not selecting. I found a python script but can't make it work. Does anyone know a way using existing tools?

Najděte více příspěvků označených jako

Accepted answers

BillDaigle

How about adding a field and calculating a random number to it, then sorting by the new field and selecting the first 10%?

All comments

BillDaigle

How about adding a field and calculating a random number to it, then sorting by the new field and selecting the first 10%?

SteveLynch

SubsetFeatures, however, it requires a Geostatistical Analyst license

DanPatterson_Retired

add a numeric integer field....

I prefer to use a small code block, something like

import numpy as np
def rand_int():
    return np.random.randint(0, 10)

expression box --- > rand_int()

Now the above processes row by row, returning a random integer in the form of

"""

Return random integers from `low` (inclusive) to `high` (exclusive).

Return random integers from the "discrete uniform" distribution in the

"half-open" interval [`low`, `high`). If `high` is None (the default),

then results are from [0, `low`).

"""

>>> a = np.random.randint(0, 10, 1000000)
>>> b = a[a==5]
>>> len(b)
100247
>>>

which is pretty good, you get about 100,000 plus change (it will never exact) out of 1 million of the values equal to 5.

JoshuaBixby

If you have the license, using Geostatistical Analyst is definitely the most robust and statistically sound method of working with random subsets.

DarinJensen

So easy! Made even easier by just limiting the random numbers to 10 and then selecting by attribute for one of the numbers. This is definitely a case of me overthinking when such a simple and elegant solution became obvious as soon as Bill set me on it. Thank you, sir!

DarrenWiens2

I'm sure there are some gotchas with this, but quick & dirty, here's how you can do it without adding a new field:

>>> import random
... fc = 'bc_geoname' # your feature class
... arcpy.SelectLayerByAttribute_management(fc,"CLEAR_SELECTION") # clear selection to consider all features
... feature_count = int(arcpy.GetCount_management(fc).getOutput(0)) # count features
... percent = 0.10 # enter your desired percentage
... rnd_set = set([]) # create a set
... while len(rnd_set) < (feature_count * percent): # do until your set is full
...    rnd_set.add(random.randint(0,feature_count-1)) # make a random integer and try to add it to the set
... where = '"FID" in ({0})'.format(','.join(map(str,rnd_set))) # include set in SQL where clause
... arcpy.SelectLayerByAttribute_management(fc,"NEW_SELECTION",where) # select the FIDs in the set

This runs instantly on 40,000 features, so I assume it will work for you, unless it hits some memory limit.

BillDaigle

May fail depending on the location of your data. Oracle has a limit of 1000 items inside an "IN" clause. I'm not sure what the limits are for file gdbs or other databases.

DarrenWiens2

Shapefile, ftw.

DanPatterson_Retired

Darren has a good point and raises another important distinction.

One advantage of adding a field, is that you have 10 (in this case) samples which don't overlap and will never choose the same record, if you had to pull out several samples of about 10%. This is sampling without replacement, for doing things like t-tests, Darren's approach should be used if you need sampling with replacement and those associated tests.

Both methods have their ups and downs and one should know both.

It is also important to note that the random.randint, randominteger are but 2 of a bunch of a bunch of distributions you can draw from ie (beta, binomial, chisquare, f, gamma, geometric, gumbel, hypergeometric, laplace, logistic, lognormal,.... poisson, Cauchy, exponential, gamma, standard_t, uniform Weibull) Allowing one to pull samples out for distribution testing.

explore

import numpy as np

dir(np.random)

DarrenWiens2

I believe using a set (vs. list) ensures that this is essentially sampling without replacement, without the performance benefit of ignoring pre-selected values. Using a list would be sampling with replacement.

DanPatterson_Retired

set reduces sample size however, so it depends upon the sample size needed and the set that is drawn, plus the record is permanent.

Another way is to produce the array (potentially larger) and save the array to disk (*.npy or .npz) then draw from than by appending to a featureclass.

TimothyStoebner

Actually the Create Random Points tool does work. Just do a spatial join with your existing data and the new random points. Make sure you use your existing layer as a "Constraining Feature Class" in the creating random points tool. You could also use select by location instead of spatial join.

DarrenWiens2

I believe you'd never be guaranteed to get 10% of the points using this method because some of the new random points would be closest to duplicate original points, or vice versa. But perhaps you could provide an example.

edit: fitting to the previous discussion, this would be an example of sampling with replacement.

TimothyStoebner

By using your existing data as the constraint. the random points are created at the same location as existing points. It's also why I suggested using select by location as an alternative to spatial join, that way when your selection count equals the random count you know you don't have any duplicates or missing points.

Also, you could check for duplicate IDs in your joined output to find duplicate originals.

hope this helps.

DarrenWiens2

Aha, I stand corrected. I thought you were suggesting to use a polygon covering all points as the constraining feature class. I didn't realize you could use points as a constraint. Good answer.

JeremyNicoletti1

Arcade can do this now (9/12/2023) - making this universally accessible.

Andrew_Link

Could you share the arcade expression to perform this action.

Připojte se ke konverzaci

Přihlášení členové mohou přispívat, sledovat aktualizace a další. Jste tu noví? Zaregistrujte si bezplatný účet.

Rychlé odkazy

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.