Item Graph Module: Create Dependency Graph Failing when creating graph for whole organization

[ClaytonBurns2]

I am having issues with the Item Graph module failing when running the create_dependency_graph function on my organization's ArcGIS Online in Notebook. I'm the main administrator for the organization and yet it fails with an "(Error Code: 403) Exception: You do not have permissions to access this resource or perform this operation."

When running the script on other organizations that I have administrator access to the graph gets created just fine. I have tried running the script with "outside_org=False, include_reverse=False" thinking that maybe permissions from outside organization layers were causing the create_dependency_graph function to fail. I have been following along with the following ESRI Documentation Working with an Organization-Wide Dependency Graph  Are there exceptions that I'm unaware of that could be included in the script to skip over, but list potential items that are causing issues for the create_dependency_graph function?

Below is an example of the script, with the added inputs of "outside_org=False, include_reverse=False" from the Item Graph documentation, however it continues to fail. The script always fails at graph = create_dependency_graph...., but it will work if I limit the max items to a small amount. This leads me to think there are some items that corrupt or not able to work with the graph?

Any suggestions would be very welcome. Thank you! 

from arcgis.gis import GIS
from arcgis.apps.itemgraph import create_dependency_graph, load_from_file
import datetime
import networkx as nx
gis = GIS("home")

# only run this cell the very first time you index over your org and grab all the dependencies
# this will take a long time if the org has a ton of items

import datetime
items = gis.content.search("", max_items = 1000)
timestamp = int(datetime.datetime.now().timestamp()) * 1000
graph = create_dependency_graph(gis, items, outside_org=False, include_reverse=False)
graph.write_to_file("/arcgis/home/graph_data/updating_graph.gml")
with open("/arcgis/home/graph_data/updating_timestamp.txt", "w") as file:
file.write(str(timestamp))

[PaulHaakma · Accepted solution]

Hi Clayton

I have also encountered a similar issue using the ItemGraph create_dependency_graph method. I was running it against ArcGIS Online and like you I was using an administrator account. I was running a Notebook from ArcGIS Pro.

I dug a little deeper and followed the Python error message down to a file in the ArcGIS API for Python called _item_graph.py where I added a print statement to print out which item id it was processing. In this way I was able to see that, for me, it seems to be failing on some hosted feature layer views that were created by Survey123 and are referenced in an Experience Builder application. When I took one of those item ids, I couldn't find any item with that id using the ArcGIS Online user interface, nor using the gis.content.get() method.

Interestingly though, from the Experience Builder app json I could see the service URL and when I attempted to access that with a valid token, it would say that I did not have permission. If I deliberately changed a character in the URL it would report no such service exists. Which suggests that something does exist that I don't have permission to access which I don't currently understand why not.

In order to process everything else, I added a try/except to skip those items. Obviously, this is just a workaround and I'm still trying to find out what those items are and if they are in use. If you felt confident in doing this and accept the implications, then this would be a way to at least process everything else. And give you a list of the items that didn't work.

I feel like there is either a bug here in the ArcGIS Online permissions or items, or some fringe case that the ArcGIS API for Python needs to handle. Either way, I suggest you lodge a support case with Esri to get it looked into.

[ClaytonBurns2]

Thank you for this information Paul. I was finally able to figure out getting the _item_graph.py script updated and it has provided me with 9 items with permission errors. As you mentioned they are not able to be retrieved with gis.content.get and then when trying gis.content.search it doesn't even think that the item exists in my org. Not able to find it in an ArcGIS Online content search either. After the script ran for a while I then got the following error when using the script you'd suggested, but getting the 9 error ID's was all I needed to make sure my issue seems to be the same one you have run into.

I will get a support case logged with ESRI as I agree that this seems to be a bug and some kind of exception needs to be added to the create_dependency_graph function to handle these inaccessible items. At the moment, I have no idea where these 9 items that caused error exceptions could even possibly be used at in an app or anything since I can't search for them and the dependency graph which should show me where these are broken at won't work.

[PaulHaakma]

Yeah, it's an interesting scenario. I think, technically, the problem is not the create_dependency_graph method, as the issues seem to be with the source items. So far, I've encountered a corrupted web map, items that don't exist but throw permission errors like they do exist, and some corrupted Experience Builder widget configuration - all these caused the graph method to crash.

It's not really the fault of the graph method, but without any error handling in that method, it just crashes. This makes it almost unusable when running over a large number of items such as an org-wide scan. 

You could argue that the solution is to fix all the items at the source. But then the item dependency method becomes a "problem identification tool" rather than it's intended purpose of finding dependencies. And in its current form it just crashes and doesn't even identify the offending item. At least wrapping the internal for loop in a try/except allows to print out the offending item's id and allow troubleshooting as well as the method to continue on and process the remainder of the items.

[ClaytonBurns2]

Sorry to keep bothering you, but you've given me very helpful tips and I really appreciate it! If it isn't too much to ask, how in the world are you finding the corrupted items? My AGO organization isn't too large, but it is just under 3,000 items which would be way too many to individually open the JSON for. Without being able to find the corrupted items via ID searches with .get, .search, or in content searches in AGO I feel like I'm at a loss to find them.

I'd be happy to individually correct the corrupt items (if possible, considering I may also have some 'items that don't exist but throw permission errors like they do exist'), but can't without being able to know where the issue is. Again, if this is too much I totally understand and I will be getting a support case logged for this also.

Thank you for the help and have a great day!

[PaulHaakma]

Hi Clayton.

No problem. I don't really have any special approach here. All I'm doing is, every time it crashes, I check the error message to find out which file and line it crashed on, then dive into the code and add a try except with a print statement in the exception handling block to print out the item id that it happens to be processing at the time. I think so far, I've done this in 2 or 3 places.

The print statement gives me the ids of the offending items. It doesn't do anything to help me fix them of course. But mainly, by having a try except, it at least allows the create_dependency_graph method to continue and process everything else.

The very bottom of the _item_graph.py file seems to be the entry point of the initial item loop, from what I can discern. So, at the very least, I put a print statement there. This tells me the parent item that is being processed. If it crashes on some dependency of that, at least I can re-run the create_dependency_graph method on just that one item and do some more focused testing without running over hundreds of items.

Hope this helps.

[NoahParavicini · Accepted solution]

Hi Paul and Clayton,

Noah here from the Python API Dev team. We've actually already patched this bug upon creating the graph- if you want to execute this notebook, you can update your version of the Python API. This should be able to be done through the package manager in Pro, and by running a conda call to install the newest version at the top of an Online/Enterprise notebook. Alternatively, you can create your own environment and run it from a local version of Jupyter.

Like you suggested Paul, we've wrapped it in try/except, except we don't raise an exception- it just essentially adds it as a node and moves on to the next one, meaning it will show up later as one of the "broken" items in your graph. If you're curious about the items where this specific case is happening and want to differentiate it from other items that aren't retrievable, I would recommend adding code under the "except" section to write the ID of the item to a log- that way, you can execute creation of the graph and identify each one of these items in one single go.

Interestingly enough, this is actually more a problem with our `content.get()` method, as opposed to the graph creation method, as usually an item will be returned as None if it's not retrievable- these items seem to be some sort of case not captured by that logic. We will examine whether we are correct to keep raising an error in this situation, or if we can add a warning and group it in with the others that return None to avoid errors like this.

Thank you both and hope this is helpful!

[ClaytonBurns2]

Thank you Noah this worked perfectly after running the Conda call to run the latest Python version. The graph was able to get created for the entire organization while skipping over the corrupt org items. Thanks again for the help and quick response to the issue!