Latest Contributions by PeterKnoop

‎10-21-2020

Thanks, Dan Patterson‌. model.fit() helpfully updates status information in the output cell every couple of seconds, so you can tell it is actively running... right up until you hit the 48-hour cap on ArcGIS Notebook run times.

‎10-19-2020

Thanks. I created an ArcGIS Idea for it, please add you Up Votes Scott Aulen‌ Charlie Coley‌ Provide ability to track ArcGIS Experience Builder interactions with Google Analytics

‎10-19-2020

Add support for Google Analytics to Experience Builder. Keep it simple, as was done in StoryMaps and Hub, where you have a setting or property for each Experience into which you can paste your Google Analytics Tracking ID. [Original GeoNet thread: Google Analytics with Experience Builder?]

‎10-17-2020

ArcGIS Notebooks seem like a great way to deliver a uniform environment to a large group of users, particular in higher-education, where you often have students bringing a wide variety of hardware and operating system combinations to the table. Our initial experience, however, is that the performance is terrible, particular with the "Advanced with GPU support - 4.0 runtime" kernel. For example, we have a lot of students and researchers interested in training AI models for automated plant species identification for use with Survey123, as exemplified in this Esri-provided Notebook: Plant species identification using a TensorFlow-Lite model within mobile devices. To help them get started, I envisioned offering a workshop where each user could work through this example using an ArcGIS Notebook, so that we don't have to spend time in the workshop configuring each users' local environment. Attempting to run this particular Esri example Notebook as an ArcGIS Notebook on ArcGIS Online, however, appears to require ~5.2 days to complete the training step! (Which means it will never finish, given what appears to be a 48-hour or 2-day limit on for how long a Notebook can run.) The example output provided in the Notebook suggests that step took ~12.5 hours for the Notebook author, and we see similar 10-18 hour runtimes for the training step when done on a typical, on-premise GIS workstation. Training AI models can certainly take time, however, it feels like the "Advanced with GPU kernel" is significantly under-resourced (or over-priced), or we missed something in the documentation for how we are supposed to use it... For instance, at a cost of 0.5-credits per minute, the ~5.2 day run would cost 3,744 credits. And, if you use the pay as you go pricing for developers to convert this to an approximate cost, the result is ~$375! If I spin up my own AWS g4dn.xlarge instance, which includes a NVIDIA T4 GPU, and run the example Notebook, the training step takes just under 19 hours. The cost of an On-Demand g4dn.xlarge instance is $0.71/hour, which translates to a total cost of ~$13.50 to train the model (plus a few more cents for network, storage, etc.) That is quite a price difference: $375 vs. $13.50, not to mention the time difference of days versus hours! I've looked around for information about what hardware is backing the "Advanced with GPU support - 4.0 runtime" kernel, but was unable to find any. Whatever the configuration is, however, it is not providing the sort of performance boost for tasks like AI model training that I would've expected for a GPU-backed environment. Is there some trick to getting better performance out of the "Advanced with GPU" kernel on ArcGIS Online?

‎10-07-2020

Here are a few additional thoughts, which might be helpful to folks, particular those diving deeper into cloning content between organizations. I haven't come across a one-size-fits-all solution, scripting or otherwise, for transferring *ALL* types of content from one ArcGIS Online organization to another. I think Brian's post bears that out as well. (I'd be happy to hear if others have found one!) You can get close with a Python script or Notebook that uses the clone_items() method from the ArcGIS API for Python. It handles a great many item types, however, there is still a subset that it does not handle, such as ArcGIS StoryMaps, Map Services, and Image Services. The Clone Portal users, groups and content sample Notebook Brian mentions above provides a general solution for transferring users, groups, and content. You can easily adapt it to work for a specific user or list of users, rather than all users in an organization, however, it does not handle all content types. There is also the note, "...if the item points to a web layer URL, the target item would also point to the same URL." What this means is that the workflow in their example does not update items that reference other items when they are transferred. For instance, while a user's Web Map and the Feature Layers it references are copied over to the new organization, the Web Map in the new organization will still point to the Feature Layers hosted on the old organization. I think most people tackling the problem of moving content from one ArcGIS Online organization to another have the expectation that when you copy over a Web Map and its Feature Layers, then the new Web Map will point to the new Feature Layers. The method used in the above Notebook, however, leaves you to do that swizzling manually on your own. The clone_items() method improves on the above approach, as it is capable of swizzling the items it transfers. For instance, when you use it to clone a Web Map to a different organization, then you can also have it clone the Feature Layers referenced in the Web Map, as well as swizzle the cloned Web Map on the destination organization to point to the cloned Feature Layers on the destination organization. The Cloning Content Notebook provides a lot of great examples of how to use clone_items() to transfer content between organizations. It makes a good starting point for a "transfer script" that would handle most item types, at least those that clone_items() method supports. One of the item types that clone_items() does not currently supported is ArcGIS StoryMaps; however, there is a comment in the Notebook that suggests that they are working on that! Meanwhile, as this is a common request we get, we developed a Notebook to specifically handle this content type (see Clone an ArcGIS StoryMap). It wraps additional logic around clone_items()to recursively discover and clone referenced content, and makes assumptions about what referenced content to clone based on ownership. Map Services and Image Services also cannot currently be cloned with clone_items(). As noted in the Cloning Content Notebook, the challenge is that it doesn't know enough about your system architecture to fully know your intentions. Since you do know your system, however, you can add the extra logic needed to your script to handle cloning such items. So, while ArcGIS Online currently lacks direct support for transferring content between organizations, a scripting-based approach using clone_items() will likely handle "most" situations for you. And, the remaining situations will hopefully be small enough in number that you can handle them manually on a case-by-case basis.

‎09-12-2020

The "Copy Python command" capability in the History pane is very helpful. It would be even better if it included the geoprocessing tool's parameter names in the content that it produces. It is often convenient to experiment with a geoprocessing tool in the Pro GUI, before trying to use the tool in a Notebook or script. Once one gets the settings right in the GUI, then they can use "Copy Python command" to capture the working version to their Notebook or script. What comes over, however, has no indication of which parameter is which in the tool's argument list. It would be very helpful -- especially for those new to Python and ArcGIS -- to have the names included to help one understand which parameter is which. For instance, you might need to tweak a parameter further, and labeling them would help with figuring out which one it is. Or, if you haven't looked at your code in awhile, and don't recall what is what for the tool, then the parameter labels would help you more quickly understand what is going on.

‎07-17-2020

Is is possible to use Google Analytics with Experience Builder? Is there a place to set a Google Analytics Tracking ID, like one can in StoryMaps or Hubs? Or, perhaps other, more complicated ways to tie the two together?

‎06-29-2020

Thanks for the additional info Joshua Cullen . Look forward to hearing what the ArcGIS Online teams thinks. In the meantime, it sounds like the answer for now is no, there is not a universal sharing URL for an item stored in ArcGIS Online that can be shared to anyone who has access to the item, whether they are in the same org or not. -peter

‎06-27-2020

Thanks, Owen Evans‌. I look forward to checking out the new tools! (Love to see the ability to convert or Save As a classic Cascade StoryMap as an ArcGIS StoryMap; that's our next most common challenge, however, it likely won't be a long-term one, as most of our community of users are creating new stories in the latter these days.)

‎06-17-2020

The April 2020 ArcGIS StoryMap update introduced a helpful "Duplicate story" capability, however, it only supports a "partial clone" of a StoryMap. In other words, it duplicates the direct content of the StoryMap itself, but not any of the referenced content, such as Web Maps and Web Scenes embedded in the StoryMap, and the Feature Layers referenced by those. I often find myself needing to create a "full" clone of a StoryMap, including any referenced content the user also created. The goal being for the person who created the StoryMap to keep a copy for themselves, under their control, while they also need to provide a copy of the StoryMap to someone else, who will then have control over the copy. Sometimes the copy also needs to be made to a different ArcGIS Online organization than where the original is hosted. While I am hopeful that Esri will directly address this need soon, in the meantime, below is a link to a Notebook created to handle cloning ArcGIS StoryMaps (not the Classic Story Map Apps). It relies on the powerful clone_items() method in the ArcGIS API for Python. By "full" clone, I mean copying or replicating not only the StoryMap itself, but also cloning any referenced content in the StoryMap that is owned by the same user as owns the StoryMap. Cloning referenced content is supported in the Notebook for the common content types (images, express maps, web maps, web scenes, feature layers, etc.) In the case of unsupported content types, the reference to the original item is carried through to the cloned StoryMap. Similarly for referenced content that is not owned by the same user as the StoryMap, the reference to the original item will be maintained in the clone. In Education, the need for a cloned StoryMap often comes up in the context of class assignments. The instructor needs a copy-of-record for grading purposes, which the student can not edit after the assignment is due. (You can also automate all of this with your LMS, so that clones are made automatically when assignments are submitted or due.) Meanwhile, the student wants to keep a copy of their work to perhaps continue to improve upon it, and to show off their GIS skills to prospective employers or graduate schools. Another common use case in Education is collaborative work. A student might work with a faculty member to create a StoryMap to help with disseminating the faculty member's research, or a student might create a StoryMap for a client as part of a service learning experience. In those cases the student again often would like to maintain control over a copy of their work to show off their skills, while the faculty member or client needs to have a copy of the final product over which they have control going forward. And, in the case of service learning, the student may create the StoryMap on their institution's ArcGIS Online organization, and the clone needs to live on the client's ArcGIS Online organization. The Notebook shared below handles cloning of StoryMaps both within and between ArcGIS Online organizations, assuming you have the appropriate privileges. It can also handle copying StoryMaps from public ArcGIS Online accounts to organizational ones, for when a student accidentally creates work there, without realizing they have an account through their institution. It also handles cloning most of the common types of ArcGIS Online content that can be incorporated into ArcGIS StoryMaps, such as Web Maps and Web Scenes, and their underlying Feature Layers. Please keep in mind, however, that it does not handle all possibilities. So keep your eye out for warnings and errors in the Notebook's output cells, which are there to help guide you in determining if it is safe to proceed, and if you will likely end up with the "full" clone of a StoryMap you were expecting. Also, there is no documentation for the ArcGIS StoryMap format, so Notebook depends on some reverse-engineering. Future updates to ArcGIS StoryMaps may not be compatible with assumptions about the format made this Notebook, thereby breaking the Notebook. Esri, however, nicely provides a format version to track in the typeKeywords, so the Notebook will potentially detect version-related issues, and provide you with a warning. If you've read this far, here's the Notebook: Clone ArcGIS StoryMap. I hope you find it useful...

‎06-15-2020

That is close, however, an org-neutral URL, using "www.arcgis.com" or "arcgis.com" still takes the user to a page with only the Download button present, even if they are logged into their org in another tab in their browser. The critical piece I am looking for is being able to run the Notebook right away, without the extra login steps, and to avoid the confusion of leading the user to believe they need to download/upload the notebook.

‎06-10-2020

Interesting. Your experience is not what others have reported. If you click on the link on this post, doesn't it take you to a page that lives at https://umich.maps..., which is not your ArcGIS Online organization? Therefore, the Open Notebook button is not present, as you do not have permission to run Notebooks in the umich organization. Did you click on the link in the post, and you were somehow redirected to view the Notebook's Item Details view on your own ArcGIS Online organization instead, so that you have an Open Notebook button? If that's the case, I wonder what is different for you than for others? Everyone else has reported that they click on the link in the post and are sent to the umich organization view, wether or not they are already logged into their own organization first.

‎06-03-2020

Knock, knock … Who’s there? … A lot of us! When I first began using ArcInfo and ArcView GIS many years ago as a student, I was part of a rarified group of workstation/desktop GIS users, representing a small number of departments on the university campus. And while such users are still well-served by what we know today as the ArcGIS platform, it is through its evolution, from desktop to web GIS, that a new, even larger user base has been empowered with geospatial technologies. A community that now encompasses a majority of the Schools and Colleges at the University of Michigan, across all three of our campuses in Ann Arbor, Dearborn, and Flint. With any enterprise-scale system, it is important to assess and understand its relevance to campus personnel (faculty, staff, and students). Such information is key to setting budgets and justifying expenditures, especially in this time of pandemic-driven monetary concerns and constraints. So how do we show that GIS is now used by more than just a few researchers from select disciplines and students in GIS courses? How do we illustrate the emerging impact of GIS on the university’s educational and research mission? And, most importantly, how do we communicate this message of broad adoption to our user community, as well as to the university stakeholders responsible for funding, maintaining, and supporting enterprise-wide systems, like the ArcGIS platform has become? University of Michigan ArcGIS Usage Dashboard Enter ArcGIS Dashboards, the app that conveys information by presenting location-based analytics using intuitive and interactive data visualizations on a single screen; because a picture is worth a thousand words! By combining ArcGIS platform usage data with information from our enterprise directory service, our University of Michigan ArcGIS Usage Dashboard serves to highlight the ongoing growth, depth, and breadth of GIS use at the university. Some of the dashboard’s key metrics: Registered Users -- Indicates how prevalent ArcGIS platform use has become on campus, and its growth over time. Examine the registered users serial chart and note the increasing slope of the curve. More and more people are discovering ArcGIS each year. Unique Logins -- Insight into the regularity of use of the ArcGIS platform. Individual use might range from a student leveraging ArcGIS StoryMaps for a single class activity during a semester, to a researcher using ArcGIS Pro every day. Units and Degree Programs – These pie charts illustrate the breadth of the impact of ArcGIS use across the university, in terms of both administrative divisions and academic disciplines. So what does our ArcGIS Usage Dashboard tell us about our University of Michigan community? The 5000+ registered users represent >5% of the potential user base of approximately 100,000 students, faculty, staff, and sponsored affiliates across the university. And while users come and go every year – particularly students -- the charts on the dashboard show a continuing increase in the annual user growth rate. The implication being there is still plenty of untapped potential for GIS use within the University of Michigan academic community. What does the dashboard tell our campus community and stakeholders, in terms our message above of broad adoption, “GIS for Everyone”? First, we can look at what unit a user is affiliated with; units are the individual Schools and Colleges of which the University is composed, and some users have multiple unit affiliations. The dashboard shows GIS users are drawn from 127 units, out of a possible ~175 across all three campuses. So over 70% of all Schools and Colleges have at least one GIS user among them. Second, we can examine which degree programs students are enrolled in, who are using GIS. (Keeping in mind that a student might be using GIS for a course in a unit other than the one in which they are pursuing their degree, research purposes in another unit, or self-education.) At the end of our last semester, of the 5000+ registered users, 3500+ were students. The dashboard indicates those students are drawn from 592 degree programs. With ~750 degree programs across all three University of Michigan campuses, this means nearly 80% of all degree programs have at least one student in them using GIS. Both of these metrics help illustrate that GIS can indeed be a useful tool for nearly anyone in our university community. Hence, we use Enterprise Logins and new member defaults in our ArcGIS Online organization to ensure everyone can access the ArcGIS platform, whenever they need it, without requiring manual intervention from the IT team. After all, a first-time user might be a student starting work on their assignment at 3 am the day it is due, or a faculty or staff member with some spare time on the weekend to explore what GIS might help them achieve. Whether you are in an educational organization, or one in a different industry, understanding how often the ArcGIS platform is used and by whom is important. Perhaps your organization is experiencing a similar surge in GIS usage, as web GIS expands your horizons beyond just GIS professionals. Build your own ArcGIS Usage Dashboard Are you inspired to build a dashboard to highlight the reach of GIS tools and technologies in your own organization? If so, then please check out our ArcGIS Usage Dashboard Quickstart Jupyter Notebook. [Note that the Notebook is shared with Everyone, however, due to the unfortunate limitations of ArcGIS Online's sharing model, if you click on the link to the Notebook above, then you will be viewing its Item Details page on the public ArcGIS Online instance (www.arcgis.com), so the only option you will have is to Download the Notebook. That's fine if you want to run it in Pro or your own local Notebook environment. If you want to run it on your ArcGIS Online instance, however, then avoid the download/upload process, and instead login to your ArcGIS Online instance first, then search for the Notebook by its item ID: "8cee96275677429685a28231752d9e67" (remembering to toggle the search to look outside of your own organization.) The result will take you to its Item Details page on your ArcGIS Online instance, which means you now have the "Open Notebook" button available to run the Notebook right away!] As a starting point, the Notebook provides templates for an ArcGIS Dashboard and a feature layer with a public view. It then leads you through the steps of populating the feature layer with historic usage data from your own ArcGIS Online organization using the Portal History method of the REST API. It also includes example code, on which you can base Python scripts, to automate updating your dashboard on a regular basis. What do you need to get started? Three things: The ArcGIS Usage Dashboard Quickstart Jupyter Notebook. An environment in which to run Jupyter Notebooks. Access to ArcGIS Notebooks in ArcGIS Online (Get started with notebooks), or ArcGIS Pro (Notebooks in ArcGIS Pro), or The ArcGIS API for Python and its SDK (install guide). An account on your ArcGIS Online organization with the default Administrator role, which has the required privileges for accessing your activity logs. The usage parameters tracked in this Notebook are only a starting point. You may want to track other parameters or integrate information from other authoritative systems of record in your organization. For example, our Units and Degree Programs pie charts incorporate data from our enterprise directory service. Or, you might add a chart illustrating the distribution of the number of people and how often they use the ArcGIS platform each month to get a feel for the proportion of heavy-users versus one-timers. Or, you might point ArcGIS Insights at your usage data layers, add in other data sources for your organization, and explore things in even greater detail. My hope is that this quickstart notebook will help you and your community gain deeper insight into your organization’s use of GIS. Lastly, thanks to my colleague, Abbey Roelofs, for assistance with our University of Michigan ArcGIS Usage Dashboard, and to Jim Detwiller for help testing the Notebook along the way to creating a Penn State ArcGIS Usage Dashboard. To learn more about ArcGIS Dashboards, see ArcGIS Dashboards - Useful Links. (Content also available as an ArcGIS StoryMap.)

‎05-22-2020

I would like to be able to provide a link to a Notebook in an email or blog post that will permit an ArcGIS Online user outside of my organization to directly run the Notebook. (This is assuming I have shared the Notebook with Everyone, and that they have the appropriate privileges to run Notebooks in their own ArcGIS Online organization.) Let's say I've set the sharing on a Notebook to Everyone. If I email someone the URL for the Item Details page for the Notebook (e.g., https://umich.maps.arcgis.com/home/item.html?id=4387d037ff904112963283f9a72e114a), then the only available option for them is the "Download" button. So it appears to the recipient that they have to first download the Notebook to their computer, and then re-upload it their ArcGIS Online organization, before they can use the Notebook. If instead they search for the Notebook in ArcGIS Online (remembering to toggle off limiting the search to their own organization), then when they click on it in the search results, they are taken to an Item Details view that includes the "Open Notebook (Beta)" button. This enables them to get right to running the Notebook I have shared. Is there a URL pattern to follow whereby one can construct a Notebook link that will produce the latter behavior -- the user being able to click on the link and then immediately run the Notebook -- avoiding the download/upload confusion?

‎05-21-2020

Is there an environment parameter or attribute that can be accessed from within a Notebook in order to reliably determine if it is being run in ArcGIS Online or in ArcGIS Pro? We are finding that we often have Notebooks we want to use in both environments. Rather than having to edit the GIS connection info depending on where it is being run, we would like it to figure that out on its own. If it is running in Pro, then use GIS('pro'), and if it is running in ArcGIS Online, then use GIS('home'). For now, we are making that assumption that the Notebook environment's HOMEPATH is r'/home/arcgis' when it is being run in ArcGIS Online. If is being run in ArcGIS Pro, then it will have some other value; it being highly unlikely that someone has installed it on their own machine in "/home/arcgis", especially on a Windows machine. For example: homepath = %env HOMEPATH if( homepath == r'/home/arcgis' 😞 print( "ArcGIS Online Notebook" ) gis = GIS('home') else: print( "Local Notebook" ) gis = GIS('pro') It would be nice to have something that is well-known and authoritative to use instead though, so that we could rely on it in the long-run.

Online Status	Online
Date Last Visited	Wednesday

My Ideas

Latest Contributions by PeterKnoop

Re: Performance Issues with Advanced Notebook with GPU?

Re: Google Analytics with Experience Builder?

Provide ability to track ArcGIS Experience Builder interactions with Google Analytics

Performance Issues with Advanced Notebook with GPU?

Re: Migrating User Content - Tips & Tricks

Improve "Copy Python command" by including parameter names

Google Analytics with Experience Builder?

Re: Appropriate URL for sharing Notebook outside of organization?

Re: Cloning ArcGIS StoryMaps

Cloning ArcGIS StoryMaps

Re: Appropriate URL for sharing Notebook outside of organization?

Re: Appropriate URL for sharing Notebook outside of organization?

GIS for Everyone (...and how to build your own ArcGIS Dashboard to show it!)

Appropriate URL for sharing Notebook outside of organization?

Configure Notebook connection info to support use with both Online and Pro?

Honor StoryMap URL redirect whenever going to view...

Support saving dark themed StoryMap as PDF with da...

Re: Georeference an image in 3D (VCS)

Re: Georeference an image in 3D (VCS)

Ability for Collection Description to start out ex...

ArcGIS Indoors for Higher-Ed User Group