Search limit of 10000 for AGOL items

[Vidar]

Hi,

We, as a company are facing a serious limitation which is having an impact on the add-in software we supply to our customers. We need to be able to search AGOL for items (Layers) that exceed the 10000 limit - we have around 33,000 items we want to return.

This is my code which hits the hard limit of 10,000 - it uses pagination.

private static async Task<HashSet<AgolLayer>> GetLayersFromAGOL(string search)
        {
            var portal = ArcGISPortalManager.Current.GetActivePortal();

            HashSet<AgolLayer> agolLayers = new HashSet<AgolLayer>(new AgolIdComparer());
            if (portal.IsSignedOn())
            {
                var query = PortalQueryParameters.CreateForItemsOfTypes(new List<PortalItemType>() { PortalItemType.Layer }, "", Properties.Settings.Default.CurrentGroupId, search);
                query.Limit = 100;
                query.Query = query.Query + "&sortField=modified&sortOrder=desc"; 

                //Loop until done
                var portalItems = new List<PortalItem>();
                while (query != null)
                {
                    //run the search
                    PortalQueryResultSet<PortalItem> results = await portal.SearchForContentAsync(query);
                    portalItems.AddRange(results.Results);
                    totalResultsCount = results.TotalResultsCount;
                    query = results.NextQueryParameters;
                }

                foreach (var pi in portalItems)
                {
                    AgolLayer agolLayer = new AgolLayer(pi);
                    agolLayers.Add(agolLayer);
                }
            }
            return agolLayers;
        }

Is there a way around this limitation?

Some links (below) back the findings I am referring to:

https://developers.arcgis.com/rest/users-groups-and-items/considerations-and-limitations.htm

[Craig_Schellenbach]

Are you able to query for the total record count and then make multiple queries while storing them in a data table? I would do something like where objectid < x

[Vidar]

Hi,

I did, in the end, do something similar to what you are suggesting. Multiple queries with guaranteed results of less that 10,000 - then store that in a list.

It was a bit more faffing about - but I did learn about clever multiple async calls to the rest endpoint - that ended up getting the results much faster than I thought. So I did gain some benefit out of it.

[FUngerStrabag]

Hi @Vidar 

may I ask how you solved that issue?

The thing is that we need to list all our ArcGIS Enterprise users as part of our license management. And of course, the number exceeds 10,000.

So the paginated call (max. number of 100 records per call) only could be made 100 times exactly yielding 10,000 users.

Many thanks!

Best,

Ferid