topic Re: Missing and duplicate results on gis.content.search in ArcGIS API for Python Questions

Missing and duplicate results on gis.content.search

CarlAlbury — Mon, 19 Aug 2019 21:44:40 GMT

I'm trying to do some content management via a jupyter notebook using the code:

from arcgis.gis import GIS
gis = GIS("https://arcgis.com", "<USERNAME>", "<PASSWORD>")
search_result = gis.content.search(query="owner:<USERNAME>", item_type="Tile Package", max_items = 750)

Which returns the correct number of tile packages, however when I loop through the results some packages are missing, others duplicated.

I other words, instead of getting:

Tile_Package_a

Tile_Package_b

Tile_Package_c

I get:

Tile_Package_a

Tile_Package_b

Tile_Package_c is there, I can find it using gis.content.get(id) but not through content.search.

Any suggestions?

Thanks

Re: Missing and duplicate results on gis.content.search

AndreasHolterman — Thu, 22 Oct 2020 16:14:54 GMT

I got the exact same issue.

Carl Albury did you find a solution in the meantime, since it is a while that you posted this question?‌

Re: Missing and duplicate results on gis.content.search

CarlAlbury — Thu, 05 Nov 2020 22:19:30 GMT

Hi Andreas Holterman, sorry for the slow reply. I did not get a solution to this issue. I had a convoluted work-around where I was able to get all of my itemid's for my tpks, which I put into a giant list and looped through that list using "gis.content.get(itemid)" to make my changes (I was modifying metadata for each item). What I can't remember is how I got that comprehensive list of itemid's. I suspect I printed them repeatedly, then combined the lists and eliminated duplicates (in Excel) until I had a list of unique id's of the correct number.

Sorry I don't have a better answer, if you figure anything out I would love to hear about it.

Re: Missing and duplicate results on gis.content.search

AndreasHolterman — Sun, 12 Dec 2021 09:27:28 GMT

Thank you Carl Albury for your reply. I have been looking into this issue a little bit deeper, and came to the conclusion that the amount of returned items always matched the expected outcome. It just inconsistently returned duplicate and missing items. My solution was to check for duplicate items in the returned list, and redo the entire search until I got a list without duplicates. So far this has done the job:

def search_gis_layers(
    gis_portal: Any,
    query: str,
    item_type: str,
    max_items: int = 1000,
    failures: int = 0,
) -> List[Any]:
    """Search layers in ArcGIS Online.
    :param gis_portal: Connection to ArcGIS Online portal
    :param query: the string to search for
    :param item_type: item type to search for
    :param max_items: the maximum numbers of items to return
    :param failures: the number of incomplete requests
    :return: list of items"""
    item_search = gis_portal.content.search(
        query, item_type=item_type, max_items=max_items
    )
    search_result = []
    mission_failed = False

    for item in item_search:
        if item not in search_result:
            search_result.append(item)
        else:
            logging.error(f"{item} is already present in list search_result")
            mission_failed = True
            failures += 1
            break

    if mission_failed and failures < 10:
        return search_gis_layers(gis_portal, query, item_type, max_items, failures)
    elif mission_failed and failures >= 10:
        sys.exit(
            "Mission failed. Tried 10 times, the returned layer list remained incomplete."
        )
    else:
        return search_result‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍