Check out this new ArcGIS Blog:

ArcGIS has a powerful sharing model that enables its members to easily share content items such as maps, apps, scenes and files with their desired audience.  Whether the audience is the entire 7.6 billion people in the world, or a few members of your company, ArcGIS Online has the infrastructure and security to get your mapping content to the right people quickly. The sharing settings of an item control both the findability and security of items in ArcGIS Online. This article will describe some common sharing scenarios and the security and findability impacts of each scenario.

Managing Security and Findability of Items with the ArcGIS Sharing Model 

The Esri development team is thrilled to announce that we have a new ArcGIS Blog website. It’s part of our overall initiative to improve your experience with ArcGIS. Although the site is new, the goal of the blog remains the same: give you deeper insight into how ArcGIS helps you get your work done.

Enhanced user experience

The new site has an enhanced design and a better experience for browsing and searching.

One of the first things you’ll notice is the new look. Articles include banners, bylines, and sticky social buttons that stay with you as you scroll. The articles can include embedded web apps including Story Maps. The site is responsive to different sized-screens and has similar branding to other Esri websites.

The site is organized around topics, each with a featured article and other articles related to that topic. Right now, a featured set of ArcGIS products makes up the topics list. Look for additional topics in the future.

You can also use the Browse All page or search with specific keywords. Search is available from all topic pages.

Redirects are in place

Our goal is to make this transition as seamless as possible for you. We’ve migrated about 2000 posts including all since 2015 and redirected old links and feeds to their new location. While redirects are in place, we suggest updating your bookmarks and feeds, especially if you had them set to specific categories or tags.

If you follow the blog using RSS, the feed for the entire blog is https://www.esri.com/arcgis-blog/feed/?post_type=blog. Feeds for specific products, categories, and tags are also enabled. Here’s an example of each:

https://www.esri.com/arcgis-blog/feed/?post_type=blog&product=arcgis-online

https://www.esri.com/arcgis-blog/feed/?post_type=blog&category=public-safety

https://www.esri.com/arcgis-blog/feed/?post_type=blog&tag=learn-arcgis

Expect the same great content

We will continue to share best practices, guidance, inspiration, and examples for using ArcGIS to accomplish your work. We will also continue to announce product updates and events, and let you know about any software patches or website maintenance. Please dive in, share these posts, and let us know your ideas.

Please note: currently the blog does not support comments. We plan to enable those soon!

Check out this ArcGIS Blog by Paul Barker

Background

Hosted feature layers come in all shapes and sizes; some start off big, some grow over time as data is updated. Regardless of their origins, it’s important that datasets remain performant, which brings me to the topic of today’s blog: response caching.  The hosted feature service team has been hard at work over the past few releases getting all the pieces in place for response caching.   Response caching might sound a bit boring on the surface but it’s pretty exciting as it allows hosted feature layers to scale even when your maps go viral.

Before I dive in, though, let’s talk about what a response is.  A response is simply the feature layer’s answer to a question you asked it.  While you can ask many questions, the most common one that is asked is: What features are in the map I’m looking at?

ArcGIS Javascript API clients, and by extension ArcGIS Online, work well with layers of all sizes.  When feature layers have a small number of features, they can be retrieved in a single request because the response from the feature layer is small. For larger layers, features are requested using tiled requests (splitting the query up into several smaller spatial queries). Tile requests have the added benefit of being consistent, even across different users and apps.  This consistency allows those responses to be cached once on the server and shared between all users. In both cases (single or tiled) those requests can also be cached on the client.  Tile requests also use quantization to improve performance but that’s a whole topic to itself that we’ll save for another blog post.

Because ArcGIS Online uses server-side response caching when multiple users request the same information, that cache kicks in.  This frees up the resources on both the server and the underlying database, allowing feature layers to scale out to millions of users and clients without the need to explicitly generate tiles ahead of time (which ArcGIS Online also supports when you need it).  These cached responses from the server are automatically invalidated as the data is edited.  This ensures that clients using the layer always get the latest information.

For many of you, the simple story is: “ArcGIS Online uses tile queries and response caching strategically to optimize performance and reduce load.”  For some of you, knowing that is enough.  But for the more curious geogeeks, here’s a deeper dive under the hood to understand all the moving parts that work in harmony to deliver performant feature layers.

Client-side caching: Your personal cache in your browser

Client-side caching is like your own personal cache of responses from the server, managed by your web browser.  When you query a feature layer, the feature service responds with a set of features that is downloaded and stored in your browser’s cache.  As you pan around the map, or zoom in and out, the browser uses those downloaded features in the browser cache whenever possible in order to avoid having to re-download the features every time the map view changes.

That browser cache has a shelf life though. If you pan around your data while it is being edited, the browser asks the servers whether anything has changed since the last time it queried. If so, the browser gets the new features and updates its cache accordingly.  This ensures that what you’re seeing on the map is current, and reduces the load on your computer and ArcGIS Online.  This browser cache only persists on the client and is not stored anywhere else. Therefor, this only benefits you and won’t make anyone else’s experience faster.   That brings us to the CDN (Content Delivery Network).  Its goal is to improve the experience for everyone.

CDN: A cache for everyone, anywhere

Content delivery networks, or CDN for short, are the backbone of a speedy internet.  For publicly shared hosted feature layers, the CDN acts similarly to client-side caching but with a few differences. First, the CDN response cache is reused by anyone using the layer and sending the same query, so everyone benefits from it, not just you.  Second, the CDN is distributed all over the world, and the cache is mirrored, which means even if the servers hosting the data are halfway around the world, the cache is most likely much closer to you.  So, in addition to being cached, the data has fewer hops over the internet to reach you so it’s faster to download.

Server-side caching: Shared and stored in ArcGIS Online

As mentioned in the previous section of this post, only publicly shared layers use the CDN, so layers shared within your organization do not make use of the CDN response cache.  This ensures that your private data is not cached on external servers around the world.  Server-side caching caches the responses to tile requests within ArcGIS Online so that other users in your organization and the public can reuse the cache when the browser cache and the CDN can’t be used.  This cache is shared between all authorized users and maintained internally as part of ArcGIS Online’s infrastructure.  As a result, queries come back quickly, put less load on the underlying databases, and keep everything running smoothly at scale even under heavy load.  As your data changes, the feature tile cache is invalidated to ensure that clients using the feature layer always see the most current information.

Wrap up: Response caching in a nutshell

When you put all the caching methods together for hosted feature layers, the workflow looks something like this:

ArcGIS Blog by Bernie Szukalski

ArcGIS Online has just been updated with the following new features and enhancements. This release includes a more flexible user experience for managing and discovering content, Insights for ArcGIS, administrative enhancements, and other improvements throughout the site.

For additional information see the What’s new help topic or ArcGIS blog posts specific to this release.

*
User experience

Start Page

You can specify the start page that you first see when you sign in to your account. The setting is managed in your Profile.

The specified page appears each time you sign in to your site. For example, set your start page to Gallery to go directly to your organization’s gallery page, or set your start page to Map to open the Map Viewer automatically after signing in. See Start page for details.

*
App Launcher

Administrators can now make web apps available to organization members by adding apps to the app launcher. The app launcher appears in the upper right next to Search after signing in.

Members can personalize their view of the app launcher by dragging and dropping apps to change the order in which they are displayed.

*
Item Status

Items can now be marked as authoritative or deprecated in the item settings. Administrators and content curators in your organization—those with privileges to update organization content—can now designate items as Authoritative. Users can deprecate items they own. Setting the status makes it easier for members of your organization to find reliable, authoritative items while discouraging the use of items that are not as reliable or becoming out-of-date. Items with these status indicators are identified by a corresponding badge on the item page.

The Status filter can be used when members search or browse for content. Organization members can also restrict layer searches to authoritative layers when authoring maps using the Map Viewer. The Authoritative status is currently only visible within your organization, Deprecated status is also visible outside your organization.

*
Item information

A new status bar and interactive checklist on item pages help item owners and administrators improve item information, making it easier for others to find, understand, and use their items.

As item details are completed, the status bar shows the current item information completion status, and offers suggestions for improvement. See Item information for more details.

*
Mobile devices

ArcGIS Online has been enhanced to better support mobile devices. Open layers in Map Viewer using the new open link on the layer’s item page. Sign in with your enterprise or social network credentials and get help with your password and user name.

*
Insights for ArcGIS

Insights for ArcGIS is a web-based, data analytics workbench that allows you to perform iterative and exploratory data analysis. A new version is now available for ArcGIS Online, previously Insights was only available with ArcGIS Enterprise.

Using interactive linked cards, you can gain new discoveries about your spatial and nonspatial data. Linked cards allow you to click on a map or chart and see related data light up on another card. In the example above, crimes are explored by type, volume, and density to see if patterns change over time.

For more information, see the Insights for ArcGIS website or Insights for ArcGIS help.

*
Map Viewer

Map Viewer is a built-in app that enables you to view and author maps, navigate, see map details, edit, perform analysis, and more (depending on your privileges). New for this release:

Analysis tools include a new Find Centroids, allowing you to find the geometric center of multi-point, line, or area features.

Aggregate Points and Summarize Within have a new option to generate custom square or hexagon bins instead of summarizing your data within an input area layer.

Light up your map with a new Firefly symbol set. A firefly point symbol is a dot with a colorized glow effect, useful for creating dramatic thematic maps. These symbols are especially effective when used with the World Imagery (Firefly) basemap.

See more Firefly symbols and basemaps in the Living Atlas of the World.

You can now use geometric operations in Arcade expressions defined for smart mapping styles, labeling, and pop-ups. Use advanced functions to normalize by area, test spatial relationships, compute distances between defined areas, and more.

You can now update the URL of layers in your map from HTTP to HTTPS, or from staging versions of your ArcGIS service layers to production versions. This is available via the Settings tab for the map item. ArcGIS Online inspects each layer in the map. If any layers use HTTP, ArcGIS Online attempts to make a request to the same layer using HTTPS, and then updates the map and any associated layer items you own or administer. If a layer does not support HTTPS, you are notified and the layer URL is not updated. See Layer Settings for more details.

Arrows are now available to indicate direction when styling line features. See Change symbols for more information.

You can now add GeoJSON files to your map. GeoJSON is a widely used open standard format for encoding a variety of geographic data structures in JavaScript Object Notation (JSON) format. See Add layers from files.

*
Scene Viewer

Scene Viewer is an app built into ArcGIS Online for creating and interacting with 3D scenes. New for this release:

You can now measure distances in 3D between two points in Scene Viewer. Choose different measuring units and leverage interactive laser lines to calculate the direct and vertical distances in your scenes. See Measure your scene.

Scene layer performance has significantly improved in this release. Buildings and integrated mesh scene layers now load faster using the correct priority order. Additionally, when zooming to scene layers, data is temporarily cached enabling the layers redraw much faster.

3D symbols from ArcGIS Pro can be used in your scene via style files published from Pro. In addition, organization Administrators can now configure custom web styles for Scene Viewer.

Other enhancements in Scene Viewer include the inclusion of 3D ArcGIS Marketplace subscription content, and 3D mobile support (still in experimental phase).

*
Data collection & management

You can now use CSV and Microsoft Excel files stored in Microsoft One Drive and Dropbox to publish hosted feature layers. If you update the the data in the file on the cloud drive, you can overwrite the data in the hosted feature layer using the updated cloud data. Google Drive, including Google Sheets, will be supported soon after this initial release. For more information, see Add and publish files from a cloud drive.

Administrators and owners can now append data to layers in existing hosted feature layers without needing to overwrite the entire hosted feature layer.

You can now define an area of interest for hosted feature layer views to limit access to features within a particular area.

New editing options are available for hosted feature layers and hosted feature layer views, providing more fine-grained control over what operations editors can perform.

After enabling editing, choose the editing capability allowed on the feature layer. See Manage editor settings for details.

*
Living Atlas

ArcGIS includes Living Atlas of the World, a global collection of authoritative, curated content from Esri and the GIS community. Living Atlas is an integrated part of the ArcGIS platform, providing an easy way to access basemaps, maps, layers, and more covering thousands of topics.

Esri vector basemaps have been updated to include additional Esri Community Maps content and style refinements. New vector basemaps are also available. Shown below is Nova – a recently introduced vector tile layer that provides a detailed basemap for the world featuring a dark background with glowing blue symbology.

See the Vector Basemaps blog posts for more information on how to take advantage of these new basemaps. For more information on Nova, view the item details.

A new version of the SSURGO Downloader has been released, providing quick access to ready-to-use map packages filled with useful soil data with over 150 attributes derived from the SSURGO dataset.

To use this application, navigate to your study area and click the map. A pop-up window will open. Click download, and the map package will be copied to your computer.

World Imagery basemap has been updated with additional sets of DigitalGlobe imagery for several countries, including sub-meter resolution Basemap +Vivid imagery for Australia and the United States. In addition, sub-meter resolution Basemap +Metro imagery has been published for a few hundred additional cities around the world.

Updates have been made to the demographic maps for Germany and several other countries. The demographic maps now feature the latest demographic data for Germany from Nexiga. In addition, updates have been released for several other countries using the latest Michael Bauer Research demographic data.

Detailed local boundaries, based on authoritative data from national government sources, have been published for a few countries. These boundary layers can be used for visualization and analysis in ArcGIS Online.

For the latest information, see the Living Atlas blog posts.

*
ArcGIS Hub

ArcGIS Hub is a platform for public engagement and citizen collaboration. It makes a smart community even smarter, giving context and purpose to data and organizing people, processes, and technology to meet the needs of citizens and governments.

ArcGIS Hub has recently been updated with the following:

Site and Page improvements include new cards that enable you to quickly and easily configure and customize your sites. Site Themeshave an improved editor, and now include 7 configurable theme options. Initiatives include better tools for managing and monitoring team members, and improvements to the Community Dashboard. Citizen Perspectives enable community members to sign-in to your Hub to analyze data and contribute local perspectives. Finally, Hub now supports 31 languages, and includes accessibility and mobile device improvements.

For more information, see ArcGIS Hub updates for December 2017.

*
Administration

Custom roles include the privilege to link to enterprise groups. To take advantage of the new privilege in existing custom roles, edit the role, add in the new privilege, and save the role.

Members with Level 1 accounts can now join external groups if assigned to a custom role granting this privilege.

Members who have the privilege to update member account information can now also reset passwords for other members. Resetting passwords is no longer a privilege reserved for default administrators.

A new setting allows administrators to prevent all ArcGIS Pro licenses in the organization from being taken offline.

The invitation process has been improved; you can now enable Esri access when inviting and adding members. A member whose account has Esri access enabled can use My Esri and GeoNet Community and Forums, access e-Learning on the Training website, and manage email communications from Esri.

Organizations can now create groups whose membership is controlled by SAML-based enterprise groups.

*
App builders

App builders enable anyone to to build compelling applications, no coding required. App builders have been updated with the following enhancements:

*
AppStudio for ArcGIS

AppStudio for ArcGIS lets you convert your maps into consumer-friendly mobile apps ready for Android, iOS, Windows, Mac OS X, and Linux, and publish them using your own brand to all popular app stores – no developer skills required. See Palm Springs Map Tour as an example.

AppStudio has released major enhancements across the board, including an improved user interface for desktop tools and additional features in templates. AppStudio now uses Qt 5.9.1 and supports ArcGIS Runtime 100.1. For more information, see What’s new in AppStudio for ArcGIS.

*
Story Maps

Story Maps let you combine authoritative maps with narrative text, images, and multimedia content, making it easy to harness the power of maps and geography to tell your story. Continuing commitment to the Web Content Accessibility Guidelines 2.0, additional accessibility improvements have been made to Story Map Journal.

You can now specify alternative text for your story’s media, plus new heading tags and landmarks provide better semantic structure for assistive technologies. Further improvements have been made to the keyboard navigation controls added in the September 2017 update. For more information, see the Story Maps blog posts.

*
Web AppBuilder for ArcGIS

Web AppBuilder offers a way for you to easily create HTML/JavaScript apps that run on any device, using a gallery of ready-to-use-widgets. You can customize the look of your apps with configurable themes, and can host your apps online or on your own server.

Web AppBuilder now adds two new widgets and several enhancements to existing widgets. The new Cost Analysis (beta) allows you to sketch a proposed design and get an estimated cost of the project. Use the new Public Notification to define a collection of properties and create mailing labels for the owners or occupants. End-users can now build their own filter expressions with Filter. Filter expressions now support relative dates such as today and last week. Infographic has eight new templates and now supports custom colors.

Other widget enhancements include support for local KML in Add Data, layer visibility in Bookmark, layer selection control in Select, layer selection and zoom to the swipe layer in Swipe, and saving all edits to attributes with one commit in Edit.

For more information, see What’s new in Web AppBuilder for ArcGIS.

*
Configurable apps

Configurable apps allow you to choose from a comprehensive set of templates to quickly author an app. For this release, the configuration pane in configurable apps has improved validation. For more information, see the configurable apps blog posts.

*
Apps for the field

ArcGIS field apps help you improve coordination and operational efficiencies in field workforce activities. Field app enhancements since the last ArcGIS Online update include the following:

*
Collector for ArcGIS

Collector for ArcGIS enables your field workforce to use smartphones or tablets to collect and update information in the field, whether connected or disconnected. Collector has updated since the last ArcGIS Online release with the following:

• Trimble Catalyst is supported on Android, allowing you to use Trimble’s new subscription-based software GNSS receiver capable of collecting data at centimeter accuracy.
• For organizations that require their accuracy be calculated using a 95 percent confidence interval, you can require the same in Collector.
• Collector’s custom URL scheme lets you include information to start a collection when launching Collector.

For more information, see What’s new in Collector for ArcGIS

*
Operations Dashboard for ArcGIS

Operations Dashboard for ArcGIS is an app that enables you to monitor, track, and assess your assets and daily operations.

Operations Dashboard for ArcGIS now supports authoring dashboards in the browser. Configure highly interactive dashboards composed of charts, gauges, maps, and other visual elements. Use better maps, with support for smart mapping, stream layers, and labels. Create powerful layouts with grouping and stacking. Get revamped data visualizations and improved date-based filtering. Manage your dashboards in the dashboard home page. For more information see Operations Dashboard for ArcGIS.

*
Survey123 for ArcGIS

Survey123 for ArcGIS is a simple and intuitive form-centric data gathering solution that makes creating, sharing, and analyzing surveys possible in three easy steps.

Survey123 has had an incremental release since the last ArcGIS Online update. The release features the following:

Survey123 Connect now includes the ability to add fields to an existing feature service, disable the Sent box, and ensure fields are created in the feature service for note questions. Camera zoom has been added to image capture in the field app and barcode scanning defaults have been improved.

The Survey123 website has been partially redesigned to provide a more intuitive, user-friendly experience. The website now includes support for modifying field names, the ability to delete a survey without deleting the associated feature service, and support for nested repeats and groups, as well as having improved performance for custom reports.

For more information, see What’s new in Survey123 for ArcGIS.

*
Workforce for ArcGIS

Workforce for ArcGIS is a mobile solution that uses location-based decision making for better field workforce coordination and teamwork. It is composed of a web app used by project administrators and dispatchers in the office, and a mobile app used by mobile workers on their devices.

Workforce released an update focused on enhancing communication between Workforce and other apps, and between dispatchers and mobile workers. You can configure your Workforce project so that information provided in the assignment can be passed to Collector for ArcGIS or Survey123 for ArcGIS when launching those apps through Workforce.

Mobile workers can add attachments while they are working in the field, and dispatchers can access those attachments through the website. Dispatchers now see the date an assignment last changed state, showing how recently assignments have progressed. Project owners now have better ways to identify project members who are no longer part of the organization.

For more information, see What’s new in Workforce for ArcGIS.

*
Apps for the office

ArcGIS Apps for the Office enable you to put powerful, user-friendly tools and data into the hands of data analysts to gain location-based insights and make decisions that save money and time. Some apps for the office have been updated, highlights include the following:

*
ArcGIS Business Analyst

ArcGIS Business Analyst helps you make smarter market planning, site selection, and customer segmentation decisions by combining demographic, lifestyle, behavioral, and spending data, maps, infographics and reports with analytics.

Business Analyst Web App now allows you to set validation on location attributes. Setting validation on location attributes helps you apply specific constraints on the site attribute values, and control what kind of data your field users capture. For example, prospective locations must have greater than 100 parking spaces or the building condition should be chosen from a drop down with options – good, average, poor.

This will be possible to set up while creating projects in the Business Analyst Web App. Both Business Analyst Web and Mobile apps will honor the validations defined and show informative messages when the entered numerical value is out of the defined range, or not in the dropdown options for text fields.

There’s also a new Executive Summary infographic template, and the ability to use maps in custom reports, plus enhancements throughout the user experience. For more information, see Business Analyst Web App What’s New.

*
ArcGIS Maps for Power BI

ArcGIS Maps for Power BI geo-enables Microsoft Power BI with spatial analytics, and offers enhanced mapping capabilities, demographic data, and more visualizations. The latest update introduces the new Plus subscription feature. Upgrading to a paid monthly Plus subscription allows you to add enhanced content and capabilities, including higher geocoding limits, more basemaps, global demographics, and access to Esri Living Atlas maps and layers to use as reference layers. For more information, see ArcGIS Maps for Power BI.

*
ArcGIS Maps for Adobe Creative Cloud

ArcGIS Maps for Adobe Creative Cloud is intended for design and communications professionals, delivering access to data-driven maps inside Adobe Illustrator and Photoshop.

You can now sign in to ArcGIS Online using your enterprise login or use your email address to access ArcGIS Maps for Adobe Creative Cloud as an anonymous user. Anonymous users have access to all public content hosted on ArcGIS Online. You now also have more control over the use of layers; you can manage labels to improve the overall appearance of your map and add or modify existing filters (definition queries) to refine what data is visible, based on attribute queries. New, large-scale vector content for the United States and the world is available, and you can now download CSV and TXT file data into the compilation window. For more information, see the ArcGIS Maps for Adobe Creative Cloud help.

*
Drone2Map for ArcGIS

Drone2Map for ArcGIS turns your drone into an enterprise GIS productivity tool by enabling you to create orthomosaics, 3D meshes, and more from your drone-captured still imagery.

Drone2Map has been updated with new features and performance improvements. The latest release adds the ability to create compressed zLAS point clouds, rolling shutter correction, GCP photo support, Integrated Windows Authentication, and flight altitude adjustment tools. For more information, see the Drone2Map for ArcGIS help.

*
Other enhancements

ArcGIS Runtime app developers can now prepare map areas ahead of time for offline use. This provides a faster and more streamlined offline experience.

Slovenian is now a supported language in the website.

*
For more information about the December 2017 release

For more information see the What’s new help topic or ArcGIS blog posts specific to this release.

The ArcGIS Online website has the latest functionality and enhancements. For this reason, the What's new help topic only contains information about the latest update. But for those who want a quick reminder of what we released when, here's an archive of past announcements. For fun, we've included a 2009 video showing the original ArcGIS Online website. Enjoy.

Previous what's new announcements

March 2019

December 2018

September 2018

June 2018

December 2017

June 2017

March 2017

December 2016

September 2016

June 2016

March 2016

November 2015 and earlier

A Look Back at the 2009 ArcGIS Online Website

In 2009 ArcGIS Online expanded from a map library to a website for sharing and collaboration. Step into the Esri Holistic Lab circa 2009 for a demo of the original ArcGIS Online website. 

You can display hundreds, thousands, tens of thousands, and bazillions of features in interactive web maps and apps using ArcGIS . This blog explains technical details about how Map Viewer and Web Apps interact with feature services. Sometimes when large datasets are displayed in their entirety, they can become visually confusing or stop performing optimally. When a feature layer stops performing optimally, an orange exclamation mark displays in the bottom-left corner of Map Viewer, indicating that the “layer did not completely draw”. 

Uncle Ben Parker (Spiderman) once said, “With great power comes great responsibility.” Keep this in mind when creating maps and apps with large datasets. Although it is possible to show as many features as you have in a web map, consider the map reader and the strategies outlined in this blog for displaying large datasets in creating the most appropriate information product.

Let’s get technical

Map Viewer and ArcGIS web apps request data from the feature layers that result in efficient display of features.  Map Viewer determines whether to request all of the features in small feature services in a single request (Snap Shot Mode) or to make many small requests to the same service (On-Demand Mode) for larger feature services.

When a feature layer is initially requested, the web map sends a query request specifying outStatistics to verify whether the layer has more than 2000 polygons, 4000 points, or 250,000 vertices. If the limits are exceeded (value of 1), On-Demand Mode is used. (For a refresher in monitoring web requests, check out this blog: https://community.esri.com/groups/cats/blog/2017/05/04/monitoring-web-service-requests-from-web-apps)

When On-Demand Mode is used to request features from a feature service, the web map divides the map into a virtual tiling scheme and makes individual requests for specific extents within the web map. This results in individual requests being sent for each tile extent determined within the map. Each request can have a maximum number of points returned. This value is different depending on the source of the feature layer. By default, hosted feature services can have a maximum of 8,000 points returned by the server efficiently, and1000 points for ArcGIS Server services. If one of the virtual tiles exceeds this limit, the orange exclamation mark appears in Map Viewer.

This map shows the 4 virtual tiles that were used to query this Web App of earthquakes in 2017, which has 15,284 features. No virtual tile requests exceed 8,000 points, thus no error message appears and all of the features are displayed.

1. Geometry: -20037508.342788905, 0.000004857778549194336, 0.000004857778549194336,20037508.342788905 , Request, Points Returned: 7763
2. Geometry: -0.000004857778549194336, 0.000004857778549194336, 20037508.34277919,20037508.342788905, Request. Points Returned: 3085
3. Geometry: -0.000004857778549194336, -20037508.34277919, 20037508.34277919, 0.000004857778549194336, Request, Points Returned: 2135
4. Geometry: -20037508.342788905, -20037508.34277919, -0.000004857778549194336, 0.000004857778549194336 Request, Points Returned: 2301

This web map shows the 4 virtual tiles that were used to query this Web App of earthquakes between 2014 and 2017, which has 95,039 features (not quite a bazillion, but you get the point). As each virtual tile returns more than 8,000 features, the orange exclamation mark appears and 8,000 features are displayed per virtual tile.

1. Geometry: -20037508.342788905, 0.000004857778549194336, 0.000004857778549194336,20037508.342788905 , Request, Points Returned: 8000
2. Geometry: -0.000004857778549194336, 0.000004857778549194336, 20037508.34277919,20037508.342788905, Request, Points Returned: 8000
3. Geometry: -0.000004857778549194336, -20037508.34277919, 20037508.34277919, 0.000004857778549194336, Request, Points Returned: 8000
4. Geometry: -20037508.342788905, -20037508.34277919, -0.000004857778549194336, 0.000004857778549194336 Request, Points Returned: 8000

Preventing the Warning

When exclamation mark indicating the layer was not completely drawn is displayed, there are two methods to ensure that all of the data draws without the error.  

1. Set up your map so that requests to the feature service won’t exceed the limits of the feature layer. You can reduce the amount of data requested in on demand mode by setting the visible scale range or applying filters.
2. If you need to display a large number of features that exceeds the request limits described above, use a tiled service. This will result in a performant layer that visually displays all of the data using pre-rendered image format.

This application displays the same data as a feature service (right) and a tile service (left). Both look very similar but you will notice the drawing performance speed is much faster in the tile service (left). With a keen eye, you can also notice that the tile layer has more features (West Coast of Greenland). Although tile service allows all of the data to draw without error, is it the best way to show large numbers of features for your audience. Check out the companion blog for Strategies for Displaying Large amounts of data.

Additional Notes about Layer Drawing Warning:

• When the layer drawing error displays, an incomplete set of the data is drawn in the web map. Additionally, the error message is suppressed in web applications, therefore your users may be unaware that only part of the dataset is displayed.
• The warning will also display if there is an error in the response from the service. It you receive an error and not features are drawn, monitor the web requests to investigate the issue further.

Check out this ArcGIS Blog by Diana Lavery

Have you ever had a dataset that contains too many points to see a clear pattern at the scale you wish to view your map? With today’s new release of ArcGIS Online, clustering is now available to help in this very situation, allowing you to find new patterns that are otherwise hidden.

For example, here’s a look at Tennessee Public Cooling Centers without clustering:

Compare that to Tennessee Public Cooling Centers with clustering:

Here’s a simple guide to clustering for data exploration:

Step 1: Simply select clustering to cluster your points. If your feature layer contains points, the clustering option will appear in the More menu:

Next, use the slider to decide how much clustering you want to use:

Less clustering (less generalization/aggregation/grouping) yields more dots on the map, and conversely, more clustering (more generalization/aggregation/grouping) yields fewer. As you zoom in and out, the clusters update based on your current map scale. Clusters are proportionally sized by the count of features to visually cue the map reader that there are more items clustered in one location vs. another.

Before clustering, there were very few options to display this information: labeling, in which the map reader had to read every label to even begin to assemble the picture; or heat mapping, which is great for lots of coincident points, but qualitatively different from depicting a cluster of points near each other. “Near” can mean different distances at different scales, and clustering naturally adjusts the size and quantity of clusters based on the current map scale.

Step 2: Explore your data’s attribute fields.

Categorical Attributes

Say you have a points layer of police stops with an attribute containing the type of police stop each is. Click on the Change Style menu, choose an attribute to show, and select the Types (Unique Symbols) option to pick your colors. Next, select clustering in the More menu.

After selecting clustering, the default pop-up will tell you how many features one cluster represents. If a small cluster only represents one feature, the traditional pop-up containing that feature’s information is displayed. Here, since our points are symbolized by categories, the predominant category in each cluster will appear. For example, in our layer of Minneapolis Police Stops, we see this:

Numeric Attributes

Clustering works with numeric attributes as well. To map a numeric attribute, click on the Change Style menu, select the numeric attribute you want to display. In this layer of Los Angeles Transit Stops, one of the attributes is the number of trips per hour in the morning rush hour (between 7am and 9am). Each stop has a number of stops per hour in this field, and as the points get clustered, we can get an average to display in the pop-up. Just as with the categorical fields above, whatever is currently symbolized will appear automatically in the clusters’ pop-ups. Here, we see the average number of stops displaying in the pop-up.

To truly customize your clusters’ pop-ups, click Configure Clustering Pop-up in the Clustering menu.

Step 3: Add a filter to subset your data and clean up the noise.

Using clustering with filters can be a powerful way to see more nuance in your data. Going back to the Minneapolis Police Stops, say you’re only interested in stops for curfew violations – a small percentage of total stops – which does not show up on the predominance cluster map.

First, simply filter your data such that only stops for curfew violations are displayed, then select clustering. Now we see clusters where there are a lot of police stops for curfew violations only, something that got lost in the first predominance map!

vs.

Advanced Tips and Tricks with Clustering

One way to make your clusters pop is to use an image as your symbol. For example, say you are creating clusters of all U.S. Dunkin’ Donuts locations. You could use a pink or orange circle/diamond/square, but to take your map to the next level, why not use the logo to get a more interesting map?

To do this, simply upload a small image as an item in your ArcGIS Online Content. Share and copy the link to the hosted image. Then, in the Change Style menu, select Use an Image, and paste the link to your image. Experiment a bit with the symbol size, and let your map sing.

A great way to display more context to your map readers is to display two layers: your clustered layer, and the original layer in a subtle, small symbol. Here we have a map of Hurricane Harvey SOS Rescues. The clusters are the proportionally-sized squares, and the small dots are the individual features. You can even adjust the transparency of the original features to make them more subtle compared to the clusters.

This shows the map reader the clustering, but also the geographic distribution of the original points. It gives the map reader an automatic sense of how many points one cluster might represent, as well as how spread out the points are.

A final advanced tip is to use clustering with Arcade Expressions. Arcade expressions allow you to construct new attributes from your existing data and map your ideas. Say we have a layer of arrests in San Francisco. We’ve already applied a filter for the specific crime of interest to us: public drunkenness. We’re interested in creating a categorical field (morning, afternoon, evening, and night) based on our existing field: time. We can use Arcade to write an expression and create a new field:

Now that our map is symbolized by this new categorical attribute, when we apply clustering, we’ll get the predominant category to appear:

Putting it all together

Some of the best maps will combine these tips and tricks. For example, this map of sales volume for store locations of a particular business uses both an image to symbolize, and the ability to display an average in the pop-up:

More Benefits of Using Clustering

Clustering “forgives” imperfect x/y or lat/long location data. Imperfect x/y or lat/long values imply that your points are more precise than they really are. Clustering makes this lack of precision irrelevant. Clustering uses multiple locations to place the cluster, so the error in one feature will cancel out the error in another feature. Of course, this assumes the error across your locations is independent and identically distributed (not systematically erring in one direction, for example).

The biggest benefit of clustering is that now, map authors have one more tool for data exploration and visualization. Data exploration and visualization is often just the first step to working with data, and can inspire ideas for actual analysis. Let the inspiration begin!

Clustering works with any hosted feature layer or feature layer served from ArcGIS Server 10.3.1 or higher with pagination support. Right now, clustering works with feature layers containing up to 50,000 features. All features are downloaded and clustered on the client. In a future release, clustering will be an option that can also be seamlessly enabled on the service itself, eliminating the need to download the entire dataset to the client. Clustering is currently only supported in ArcGIS Online 2D web applications along with the upcoming 3.22 version of the ArcGIS API for JavaScript. Support across the platform (Pro, Runtime, JSAPI 4.x, & ArcGIS Apps) will come in 2018. If you author a web map that has clustering enabled and view it in a client that does not currently support clustering, the features will display as if they were not clustered.

For a full interactive guide on using clustering in ArcGIS Online, visit this story map on Clustering.  The official clustering help page and a quick video tutorial are also available.