Search limit of 10000 for AGOL items

634
3
01-18-2023 06:27 AM
Vidar
by
Occasional Contributor II

Hi,

We, as a company are facing a serious limitation which is having an impact on the add-in software we supply to our customers. We need to be able to search AGOL for items (Layers) that exceed the 10000 limit - we have around 33,000 items we want to return.

This is my code which hits the hard limit of 10,000 - it uses pagination.

private static async Task<HashSet<AgolLayer>> GetLayersFromAGOL(string search)
        {
            var portal = ArcGISPortalManager.Current.GetActivePortal();

            HashSet<AgolLayer> agolLayers = new HashSet<AgolLayer>(new AgolIdComparer());
            if (portal.IsSignedOn())
            {
                var query = PortalQueryParameters.CreateForItemsOfTypes(new List<PortalItemType>() { PortalItemType.Layer }, "", Properties.Settings.Default.CurrentGroupId, search);
                query.Limit = 100;
                query.Query = query.Query + "&sortField=modified&sortOrder=desc"; 

                //Loop until done
                var portalItems = new List<PortalItem>();
                while (query != null)
                {
                    //run the search
                    PortalQueryResultSet<PortalItem> results = await portal.SearchForContentAsync(query);
                    portalItems.AddRange(results.Results);
                    totalResultsCount = results.TotalResultsCount;
                    query = results.NextQueryParameters;
                }

                foreach (var pi in portalItems)
                {
                    AgolLayer agolLayer = new AgolLayer(pi);
                    agolLayers.Add(agolLayer);
                }
            }
            return agolLayers;
        }

Is there a way around this limitation?

 

Some links (below) back the findings I am referring to:

https://developers.arcgis.com/rest/users-groups-and-items/considerations-and-limitations.htm

Vidar_0-1674051535955.png

 

0 Kudos
3 Replies
Craig_Schellenbach
New Contributor II

Are you able to query for the total record count and then make multiple queries while storing them in a data table? I would do something like where objectid < x

0 Kudos
Vidar
by
Occasional Contributor II

Hi,

I did, in the end, do something similar to what you are suggesting. Multiple queries with guaranteed results of less that 10,000 - then store that in a list.

It was a bit more faffing about - but I did learn about clever multiple async calls to the rest endpoint - that ended up getting the results much faster than I thought. So I did gain some benefit out of it.

0 Kudos
FUngerStrabag
New Contributor II

Hi @Vidar 

may I ask how you solved that issue?

The thing is that we need to list all our ArcGIS Enterprise users as part of our license management. And of course, the number exceeds 10,000.

So the paginated call (max. number of 100 records per call) only could be made 100 times exactly yielding 10,000 users.

Many thanks!

Best,

Ferid

 

0 Kudos