Latest Contributions by PeterKnoop

Add support for Google Analytics to Experience Builder. Keep it simple, as was done in StoryMaps and Hub, where you have a setting or property for each Experience into which you can paste your Google Analytics Tracking ID. [Original GeoNet thread: Google Analytics with Experience Builder?] ... View more

ArcGIS Notebooks seem like a great way to deliver a uniform environment to a large group of users, particular in higher-education, where you often have students bringing a wide variety of hardware and operating system combinations to the table. Our initial experience, however, is that the performance is terrible, particular with the "Advanced with GPU support - 4.0 runtime" kernel. For example, we have a lot of students and researchers interested in training AI models for automated plant species identification for use with Survey123, as exemplified in this Esri-provided Notebook: Plant species identification using a TensorFlow-Lite model within mobile devices. To help them get started, I envisioned offering a workshop where each user could work through this example using an ArcGIS Notebook, so that we don't have to spend time in the workshop configuring each users' local environment. Attempting to run this particular Esri example Notebook as an ArcGIS Notebook on ArcGIS Online, however, appears to require ~5.2 days to complete the training step! (Which means it will never finish, given what appears to be a 48-hour or 2-day limit on for how long a Notebook can run.) The example output provided in the Notebook suggests that step took ~12.5 hours for the Notebook author, and we see similar 10-18 hour runtimes for the training step when done on a typical, on-premise GIS workstation.  Training AI models can certainly take time, however, it feels like the "Advanced with GPU kernel" is significantly under-resourced (or over-priced), or we missed something in the documentation for how we are supposed to use it... For instance, at a cost of 0.5-credits per minute, the ~5.2 day run would cost 3,744 credits. And, if you use the pay as you go pricing for developers to convert this to an approximate cost, the result is ~$375! If I spin up my own AWS g4dn.xlarge instance, which includes a NVIDIA T4 GPU, and run the example Notebook, the training step takes just under 19 hours. The cost of an On-Demand g4dn.xlarge instance is $0.71/hour, which translates to a total cost of ~$13.50 to train the model (plus a few more cents for network, storage, etc.) That is quite a price difference: $375 vs. $13.50, not to mention the time difference of days versus hours! I've looked around for information about what hardware is backing the "Advanced with GPU support - 4.0 runtime" kernel, but was unable to find any. Whatever the configuration is, however, it is not providing the sort of performance boost for tasks like AI model training that I would've expected for a GPU-backed environment.    Is there some trick to getting better performance out of the "Advanced with GPU" kernel on ArcGIS Online? ... View more

The "Copy Python command" capability in the History pane is very helpful. It would be even better if it included the geoprocessing tool's parameter names in the content that it produces. It is often convenient to experiment with a geoprocessing tool in the Pro GUI, before trying to use the tool in a Notebook or script.  Once one gets the settings right in the GUI, then they can use "Copy Python command" to capture the working version to their Notebook or script. What comes over, however, has no indication of which parameter is which in the tool's argument list.  It would be very helpful -- especially for those new to Python and ArcGIS -- to have the names included to help one understand which parameter is which. For instance, you might need to tweak a parameter further, and labeling them would help with figuring out which one it is. Or, if you haven't looked at your code in awhile, and don't recall what is what for the tool, then the parameter labels would help you more quickly understand what is going on. ... View more

The April 2020 ArcGIS StoryMap update introduced a helpful "Duplicate story" capability, however, it only supports a "partial clone" of a StoryMap. In other words, it duplicates the direct content of the StoryMap itself, but not any of the referenced content, such as Web Maps and Web Scenes embedded in the StoryMap, and the Feature Layers referenced by those.   I often find myself needing to create a "full" clone of a StoryMap, including any referenced content the user also created. The goal being for the person who created the StoryMap to keep a copy for themselves, under their control, while they also need to provide a copy of the StoryMap to someone else, who will then have control over the copy. Sometimes the copy also needs to be made to a different ArcGIS Online organization than where the original is hosted.   While I am hopeful that Esri will directly address this need soon, in the meantime, below is a link to a Notebook created to handle cloning ArcGIS StoryMaps (not the Classic Story Map Apps). It relies on the powerful clone_items() method in the ArcGIS API for Python.     By "full" clone, I mean copying or replicating not only the StoryMap itself, but also cloning any referenced content in the StoryMap that is owned by the same user as owns the StoryMap. Cloning referenced content is supported in the Notebook for the common content types (images, express maps, web maps, web scenes, feature layers, etc.) In the case of unsupported content types, the reference to the original item is carried through to the cloned StoryMap. Similarly for referenced content that is not owned by the same user as the StoryMap, the reference to the original item will be maintained in the clone.   In Education, the need for a cloned StoryMap often comes up in the context of class assignments. The instructor needs a copy-of-record for grading purposes, which the student can not edit after the assignment is due. (You can also automate all of this with your LMS, so that clones are made automatically when assignments are submitted or due.) Meanwhile, the student wants to keep a copy of their work to perhaps continue to improve upon it, and to show off their GIS skills to prospective employers or graduate schools.   Another common use case in Education is collaborative work. A student might work with a faculty member to create a StoryMap to help with disseminating the faculty member's research, or a student might create a StoryMap for a client as part of a service learning experience. In those cases the student again often would like to maintain control over a copy of their work to show off their skills, while the faculty member or client needs to have a copy of the final product over which they have control going forward. And, in the case of service learning, the student may create the StoryMap on their institution's ArcGIS Online organization, and the clone needs to live on the client's ArcGIS Online organization.   The Notebook shared below handles cloning of StoryMaps both within and between ArcGIS Online organizations, assuming you have the appropriate privileges. It can also handle copying StoryMaps from public ArcGIS Online accounts to organizational ones, for when a student accidentally creates work there, without realizing they have an account through their institution.    It also handles cloning most of the common types of ArcGIS Online content that can be incorporated into ArcGIS StoryMaps, such as Web Maps and Web Scenes, and their underlying Feature Layers. Please keep in mind, however, that it does not handle all possibilities. So keep your eye out for warnings and errors in the Notebook's output cells, which are there to help guide you in determining if it is safe to proceed, and if you will likely end up with the "full" clone of a StoryMap you were expecting.   Also, there is no documentation for the ArcGIS StoryMap format, so Notebook depends on some reverse-engineering. Future updates to ArcGIS StoryMaps may not be compatible with assumptions about the format made this Notebook, thereby breaking the Notebook. Esri, however, nicely provides a format version to track in the typeKeywords, so the Notebook will potentially detect version-related issues, and provide you with a warning.   If you've read this far, here's the Notebook: Clone ArcGIS StoryMap.   I hope you find it useful... ... View more

Knock, knock … Who’s there? … A lot of us!   When I first began using ArcInfo and ArcView GIS many years ago as a student, I was part of a rarified group of workstation/desktop GIS users, representing a small number of departments on the university campus. And while such users are still well-served by what we know today as the ArcGIS platform, it is through its evolution, from desktop to web GIS, that a new, even larger user base has been empowered with geospatial technologies. A community that now encompasses a majority of the Schools and Colleges at the University of Michigan, across all three of our campuses in Ann Arbor, Dearborn, and Flint. With any enterprise-scale system, it is important to assess and understand its relevance to campus personnel (faculty, staff, and students). Such information is key to setting budgets and justifying expenditures, especially in this time of pandemic-driven monetary concerns and constraints.   So how do we show that GIS is now used by more than just a few researchers from select disciplines and students in GIS courses? How do we illustrate the emerging impact of GIS on the university’s educational and research mission? And, most importantly, how do we communicate this message of broad adoption to our user community, as well as to the university stakeholders responsible for funding, maintaining, and supporting enterprise-wide systems, like the ArcGIS platform has become? University of Michigan ArcGIS Usage Dashboard   Enter ArcGIS Dashboards, the app that conveys information by presenting location-based analytics using intuitive and interactive data visualizations on a single screen; because a picture is worth a thousand words! By combining ArcGIS platform usage data with information from our enterprise directory service, our University of Michigan ArcGIS Usage Dashboard serves to highlight the ongoing growth, depth, and breadth of GIS use at the university. Some of the dashboard’s key metrics:   Registered Users -- Indicates how prevalent ArcGIS platform use has become on campus, and its growth over time. Examine the registered users serial chart and note the increasing slope of the curve. More and more people are discovering ArcGIS each year. Unique Logins -- Insight into the regularity of use of the ArcGIS platform. Individual use might range from a student leveraging ArcGIS StoryMaps for a single class activity during a semester, to a researcher using ArcGIS Pro every day.  Units and Degree Programs – These pie charts illustrate the breadth of the impact of ArcGIS use across the university, in terms of both administrative divisions and academic disciplines.   So what does our ArcGIS Usage Dashboard tell us about our University of Michigan community?   The 5000+ registered users represent >5% of the potential user base of approximately 100,000 students, faculty, staff, and sponsored affiliates across the university. And while users come and go every year – particularly students -- the charts on the dashboard show a continuing increase in the annual user growth rate. The implication being there is still plenty of untapped potential for GIS use within the University of Michigan academic community.   What does the dashboard tell our campus community and stakeholders, in terms our message above of broad adoption, “GIS for Everyone”?   First, we can look at what unit a user is affiliated with; units are the individual Schools and Colleges of which the University is composed, and some users have multiple unit affiliations. The dashboard shows GIS users are drawn from 127 units, out of a possible ~175 across all three campuses. So over 70% of all Schools and Colleges have at least one GIS user among them.   Second, we can examine which degree programs students are enrolled in, who are using GIS. (Keeping in mind that a student might be using GIS for a course in a unit other than the one in which they are pursuing their degree, research purposes in another unit, or self-education.) At the end of our last semester, of the 5000+ registered users, 3500+ were students. The dashboard indicates those students are drawn from 592 degree programs. With ~750 degree programs across all three University of Michigan campuses, this means nearly 80% of all degree programs have at least one student in them using GIS.   Both of these metrics help illustrate that GIS can indeed be a useful tool for nearly anyone in our university community. Hence, we use Enterprise Logins and new member defaults in our ArcGIS Online organization to ensure everyone can access the ArcGIS platform, whenever they need it, without requiring manual intervention from the IT team. After all, a first-time user might be a student starting work on their assignment at 3 am the day it is due, or a faculty or staff member with some spare time on the weekend to explore what GIS might help them achieve.   Whether you are in an educational organization, or one in a different industry, understanding how often the ArcGIS platform is used and by whom is important. Perhaps your organization is experiencing a similar surge in GIS usage, as web GIS expands your horizons beyond just GIS professionals.  Build your own ArcGIS Usage Dashboard Are you inspired to build a dashboard to highlight the reach of GIS tools and technologies in your own organization? If so, then please check out our ArcGIS Usage Dashboard Quickstart Jupyter Notebook. [Note that the Notebook is shared with Everyone, however, due to the unfortunate limitations of ArcGIS Online's sharing model, if you click on the link to the Notebook above, then you will be viewing its Item Details page on the public ArcGIS Online instance (www.arcgis.com), so the only option you will have is to Download the Notebook. That's fine if you want to run it in Pro or your own local Notebook environment.  If you want to run it on your ArcGIS Online instance, however, then avoid the download/upload process, and instead login to your ArcGIS Online instance first, then search for the Notebook by its item ID: "8cee96275677429685a28231752d9e67" (remembering to toggle the search to look outside of your own organization.) The result will take you to its Item Details page on your ArcGIS Online instance, which means you now have the "Open Notebook" button available to run the Notebook right away!]   As a starting point, the Notebook provides templates for an ArcGIS Dashboard and a feature layer with a public view. It then leads you through the steps of populating the feature layer with historic usage data from your own ArcGIS Online organization using the Portal History method of the REST API. It also includes example code, on which you can base Python scripts, to automate updating your dashboard on a regular basis.   What do you need to get started? Three things: The ArcGIS Usage Dashboard Quickstart Jupyter Notebook. An environment in which to run Jupyter Notebooks. Access to ArcGIS Notebooks in ArcGIS Online (Get started with notebooks), or ArcGIS Pro (Notebooks in ArcGIS Pro), or The ArcGIS API for Python and its SDK (install guide). An account on your ArcGIS Online organization with the default Administrator role, which has the required privileges for accessing your activity logs. The usage parameters tracked in this Notebook are only a starting point. You may want to track other parameters or integrate information from other authoritative systems of record in your organization. For example, our Units and Degree Programs pie charts incorporate data from our enterprise directory service. Or, you might add a chart illustrating the distribution of the number of people and how often they use the ArcGIS platform each month to get a feel for the proportion of heavy-users versus one-timers. Or, you might point ArcGIS Insights at your usage data layers, add in other data sources for your organization, and explore things in even greater detail. My hope is that this quickstart notebook will help you and your community gain deeper insight into your organization’s use of GIS. Lastly, thanks to my colleague, Abbey Roelofs, for assistance with our University of Michigan ArcGIS Usage Dashboard, and to Jim Detwiller for help testing the Notebook along the way to creating a Penn State ArcGIS Usage Dashboard.  To learn more about ArcGIS Dashboards, see ArcGIS Dashboards - Useful Links. (Content also available as an ArcGIS StoryMap.) ... View more