Implementing ArcGIS Blog - Page 4

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Latest Activity

(183 Posts)
SridharKarra
Esri Contributor

The Managed Cloud Services team in Professional Services is pleased to announce a new series that will be highlighting various tools and best practices for implementing ArcGIS Enterprise using modern methodologies.

System implementation, configuration and management of the deployment is a fun challenge, similar to Tetris. As an ArcGIS Enterprise or ArcGIS Server administrator, you are likely tasked with standing up new systems, ensuring they are configured correctly and maintaining them over time. Traditionally, these tasks were done manually where an administrator would work through procuring a new virtual machine, install the required software, work through the configuration steps needed and then ensure users were able to access it. Over time, the needs of the users may change and as such, the administrator would need to further modify the system and its software to meet those needs.

Here at Managed Services, we are faced with managing 100's of customer implementation. This series will cover best practices we developed overtime. Each entry will cover a specific aspect of automating the deployment, configuration & life-cycle management (updates, monitoring, scaling, etc) of both infrastructure and the ArcGIS suite. 

automation, devops, implementation , engineering‌, cloud‌

more
12 4 4,690
JeffDeWeese
Esri Contributor

As cloud adoption evolves from Web GIS to full GIS deployments, questions continue to be raised such as, “What about the desktops?”. That is, when moving desktops to the cloud, what technologies should be used to support Esri desktop GIS? The cloud offers multiple desktop options and the following will provide some high-level guidance as to how and when these technologies should be used. It is important to realize that each deployment is unique and deciding on which of these technologies to deploy involves multiple factors. The purpose of providing this information is primarily to share information regarding all of the potential solutions and high-lighting some of their key characteristics. However, they will not be ranked in any way as deciding on one approach over the other requires more detailed analysis and discussion based on specified requirements, costs, and constraints. Further, this list can likely be expanded but the solutions below represent the most common options that Esri has encountered. 

Note: Assuming ArcGIS Pro will be used for rendering, ArcGIS Pro requires a GPU-enabled machine type for the underlying host VM. Examples include the NVv4 for Azure and a Graphics Bundle for AWS WorkSpaces.

 

Virtual Machines - Azure and AWS

  • Use Case: Typically used to support administrative functions or small number of desktop users
  • Client Connectivity: Utilizes the Remote Desktop Connection client and the RDP protocol
  • User Experience: Published desktop with growing visual latency as geographic distance increases
  • Scalability: Limited due to no more than two concurrent users per VM
  • Management: Typically deployed without a base image
  • User Profiles: Locally stored per VM

 

Remote Desktop Services  - Azure and AWS

  • Use Case: Supporting users at scale where the users are not globally distributed
  • Connectivity: Utilizes the Remote Desktop Connection client and the RDP protocol
  • User Experience: Published desktop or apps with growing visual latency as geographic distance increases
  • Scalability: Limited for ArcGIS Pro based on the number of concurrent sessions that can share resources (e.g., GPU)
  • Management: Can be used with snapshot technology to create a base image
  • User Profiles: Roaming profile or equivalent, assuming at least two servers deployed

 

Citrix Virtual Apps and Desktops (XenApp) - Azure and AWS

  • Use Case: Supporting users at scale where the users could be globally distributed
  • Connectivity: Utilizes the Citrix Workspace app and the HDX protocol
  • User Experience: Supports both published desktops and apps and performs well with high-latency
  • Scalability: Limited for ArcGIS Pro based on the number of concurrent sessions that can share a GPU
  • Management: Can be used with snapshot technology to create a base image
  • User Profiles: Roaming profile or equivalent, assuming at least two servers deployed
  • Other: Can utilize Citrix Cloud to manage the "back-end" (e.g., Controllers/Licensing)

 

Amazon WorkSpaces - AWS

  • Use Case: Supporting users at scale
  • Connectivity: Utilizes either a desktop or web client with either the PCoIP or WSP protocol
  • User Experience: Supports a published desktop to an assigned WorkSpace instance
  • Scalability: Can scale as needed as users increase but is 1:1 user to VM assignment
  • Management: Cannot be used with snapshot technology so each WorkSpace is an independent deployment
  • User Profiles: Locally stored on each WorkSpace

 

Amazon AppStream 2.0 - AWS

  • Use Case: Supporting user applications at scale
  • Connectivity: Utilizes either a desktop or web client with the NICE DCV protocol
  • User Experience: Supports published desktop applications
  • Scalability: Can scale as needed as back-end infrastructure capacity is managed by AWS
  • Management: Based on creating base images for different application configurations as needed
  • User Profiles: Saved to a Virtual Hard Disk (VHD) and synchronized to Amazon S3
  • Other: Esri / AWS AppStream 2.0 Deployment Guide

 

Azure Virtual Desktop - Azure

  • Use Case: Supporting users at scale
  • Connectivity: Utilizes the Remote Desktop Connection client and the RDP protocol
  • User Experience: Supports a published virtual desktop
  • Scalability: Can scale as needed assigning users to virtual desktops
  • Management: Can be used with snapshot technology to create a base image
  • User Profiles: Roaming profile or equivalent, assuming at least two servers deployed (e.g., vie FSLogix)
  • Other: The only solution supporting multi-session Windows 10/11

 

more
8 4 5,754
JeffDeWeese
Esri Contributor

As the Esri platform continues to evolve, it is critical that organizations maintain a capable GIS system architecture that will support new GIS/IT capabilities and scale to support growing user demand. There are further key considerations that we have seen most recently such as moving to the cloud, migrating to ArcGIS Pro, and expanding the adoption of GIS capabilities within the organization. Esri Architects can work with you to understand your organization’s needs and provide guidance. We will be available at UC2019 at the Guiding Your Geospatial Journey area of the Esri Showcase and participating in several sessions throughout the conference. Meanwhile, have a look at the Architecture and Security section of GeoNet where you will discover valuable information related to GIS system architecture!

Technical Workshops

Tuesday, July 9                                                                                                         Location: SDCC - Rooms

8:30 am

Esri Best Practices: Architecting Your ArcGIS Implementation

SDCC, Room 33 C

8:30 am

Moving to a Managed Cloud Services Environment

SDCC, Room 30 A

Wednesday, July 10                                                                                         Location: SDCC - Rooms

8:00 AM

Designing an Enterprise GIS Security Strategy

SDCC, Room 30 A

1:00 pm

Moving to a Managed Cloud Services Environment

SDCC, Room 31 A

4:00 pm

Esri Best Practices: Architecting Your ArcGIS Implementation

SDCC, Room 31 A


Thursday, July 11                                                                                             Location: SDCC - Rooms

10:00 am

How to be Successful with Esri Managed Cloud Services

SDCC, Room 16 A

 

Spotlight Talks                                               

Tuesday, July 9                             Location: SDCC – Expo: Guiding Your Geospatial Journey Spotlight Theater      

11:15 am

ArcGIS Enterprise: Architecture Best Practices

 

1:00 pm

ArcGIS in the Cloud

 

1:30 pm

Build Security into Your System

 

Wednesday, July 10                          Location: SDCC – Expo: Guiding Your Geospatial Journey Spotlight Theater      

10:00 am

Are You Cloud Ready?

 

11:15 am

Designing a Robust Environment - Workload Separation

 

1:30 pm

Considerations for a Highly Available Enterprise

 

4:30 pm

Designing a Robust Environment - Environment Isolation

 

5:45 pm

Distributed Web GIS - A Modern Approach to Sharing

 

Appointments         

Tuesday, July 9 – Thursday, July 11  Location: SDCC – Expo: Guiding Your Geospatial Journey Spotlight Theater                                             

Architecture Maturity Review

Get expert advice and feedback on your enterprise implementation, including best practices. Leave with recommendations for meeting security and architecture needs.

Schedule an appointment

 

Stop by to connect 1-on-1 with Esri Staff that can talk more about your system architecture plans in our Guiding Your Geospatial Journey area.

Tuesday, July 9               9:00 AM–6:00 PM

Wednesday, July 10        9:00 AM–6:00 PM

Thursday, July 11            9:00 AM–4:00 PM

more
3 0 548
AdamCarnow
Esri Regular Contributor

If you're headed to the 2019 Esri International User Conference and are interested in sessions for GIS Managers, here is a link to the GIS Manager Track:

https://userconference2019.schedule.esri.com/schedule?filters=1964269831

more
0 0 499
AdamCarnow
Esri Regular Contributor

Are you a GIS manager, leader or other executive headed to the 2019 Esri International User Conference (UC)?  I know it can be a challenge creating your personal agenda for the world's largest GIS conference, so I created this flier to assist.  It covers suggested events and activities you should consider when deciding how to spend your valuable time at UC.  I hope you have a productive UC experience, and I hope to see you there!

UPDATED 6/24/2019 - Added GIS Manager Track and Get Advice from Esri Services section.

UPDATED 6/13/2019 - Corrected the name of the Implementing ArcGIS area in the Expo to “Guiding your Geospatial Journey”

FYI there are other Esri UC fliers here: https://community.esri.com/community/events/user-conference/content?filterID=contentstatus%5Bpublish...

more
3 0 2,306
AhmadAbdallah
Esri Contributor

Introduction

To cloud or not to cloud is the question that many organizations are currently facing. While on-premise data center technology is not necessarily on the verge of extinction, cloud computing is an option with many benefits including scalability, agility and cost efficiency.

The ArcGIS platform is supported on both on-premises or in a cloud environment like Microsoft Azure or Amazon Web Services (AWS). By leveraging these components, you can expose GIS content and capabilities as web services and consume those services in your apps. This enables users to access and apply useful GIS resources in their work.

Moving to the cloud is more than just upgrading your servers and software. It represents a radical change in the way that you manage technology. This shift gives you a unique opportunity to align technology with your overall business vision, which in turn can help you grow your business, improve productivity, and reduce costs.

Before starting, it is a must to document your current infrastructure, this includes compiling a list of your servers, storage systems, network components, off the shelf software, bespoke software, and subscriptions to gain a full picture of your current technology. This information is critical for helping you determine the best path forward.

Analyzing your current infrastructure against your user workflows is the next step.  From assessing your desktop workflows to assessing your integrations, your security policies, SLAs, data storage requirements, authorization, and authentication providers all these points will play a huge factor in designing your cloud infrastructure. We will discuss these topics briefly in this article and will go into the specifics of each separately in future blogs posts.

 

General Database Considerations

Moving your Geodatabase to the cloud is a big step, you need to consider where your clients accessing the data exist, are they running from the cloud or on-premise, and what is the type of work is being performed (read-only, frequent edits, etc.).

 

General Network Bandwidth and Speed

Moving all the infrastructure to the cloud will need a resilient and reliable Internet connection, most importantly with low latency. Low bandwidth can be a serious problem when you factor in a large number of employees sharing the same network. Sometimes cloud solutions don’t offer enough bandwidth, especially when it comes to uploading larger files such as raw satellite imagery. However, latency is the primary concern as network latency is the key factor in determining cloud architecture decisions, for example, because of the latency it is likely not practical to keep desktops on premise and move the database into the cloud.

 

Assessing ArcGIS Desktop Workflows

First and Foremost, do you need to provide this functionality through ArcGIS Pro or ArcGIS Desktop, or can we have it replaced via a web-based app? That would save us setting up a dedicated machine or application instance on the cloud.

Is the data you are processing 2D or 3D? ArcGIS Pro required a GPU and the resourcces are even more demanding if processing 3D data. GPU enabled machines on the cloud cost considerably more than machines without a GPU. Do you have advanced Geoprocessing workflows that require hefty computing power, do these workflows run overnight? Check out https://pro.arcgis.com/en/pro-app/get-started/virtualization-overview.htm for more information about running ArcGIS Pro on the cloud.

 

ArcGIS Server Sites and Portal for ArcGIS Sites

The least difficult method for starting to use the cloud for hosting your own software is perform a lift-and-shift migration, where you move software currently running on-premises to the cloud host. The behavior of the software is precisely the same after moving into the cloud however now it is running with the benefits of underlying cloud infrastructure, such as having affordable and reliable virtual machines that don’t require high upfront purchasing costs and little ongoing maintenance of the infrastructure.

Using specialized deployment tools can make it easier to install and configure the software on certain cloud platforms. You can also create your own machine images to host Esri software in cloud environments. These deployment tools don’t just install and configure the software; they also provision and set up the underlying infrastructure, including the virtual machines, load balancers, networking, and storage.

 

Bespoke GIS Applications

Whether you have Geocentric applications, Geo-enabled applications, or Composite applications you will need to reassess the architecture of these applications and ensure the compatibility with the new cloud environment. check out developers.arcgis.com for information about our wide coverage of SDKs and APIs.

No single application integration pattern fits all situations. You can use the application pattern that best combines capabilities from ArcGIS and your business system to deliver the greatest impact.

 

Integration to Other Systems

Application integration lets you deliver solutions that combine data and tools from different systems—including your GIS as well as business systems like permitting, licensing, and asset management systems. With integrated solutions, you can improve cross-functional business processes and provide decision-makers with integrated views of your organization’s information. Insuring that these systems integrate prevents information duplication and enables having the right data available when you need it. You need the option of deploying your integration technology so that it works in both environments. To decide whether your integration technology will reside in the cloud or in a rack in your data center, you have to start with the specific requirements of your business.

 

Security

Security is not only an important decision to avoid becoming a statistic, but also one that may be government regulated. Depending on your industry, market, and location, you may have to abide by an array of rules determining how you use and store sensitive data. Cloud security has come a long way, though, and is arguably as secure as most private data centers. In either case, security and regulations of your industry are something to think about when considering cloud and on-premise. With either option, security is granularity configurable to meet your security needs. Consider Esri best practices for configuring secure ArcGIS Server and Portal for ArcGIS environments as well as your organization's needs and policies when designing your ArcGIS Enterprise site security.

 

High Availability and Redundancy

Designing the HA environments in Azure or AWS are somewhat the same design principles as of on-premises however more functionality is provided by cloud providers such as scalability on demand. You can configure your site so that ArcGIS Server machines are added in response to certain triggers, such as CPU usage. New servers can be created in a matter of minutes, allowing your site to gracefully respond to abrupt spikes in traffic. When you no longer need the instances, you can destroy them and incur no further infrastructure charges for them. The installation is much easier as now there are images already provided for faster installation times.

 

Backups and Disaster Recovery

Cloud computing has led to a new way of preparing for IT disasters by providing secondary environments for backing up and restoring data and failing over business applications. These disaster recovery services are cost-effective and straightforward to set up. ArcGIS Server and Portal for ArcGIS include utilities that you can use to create backups and restore your sites on Azure and AWS.

 

Access Management Providers

When implementing a cloud-based application it can be a challenge to ensure that information security and access management controls are applied to the cloud application.

Organizations should ensure that the single sign-on (SSO) access management is implemented in the cloud. In particular, organizations need to understand how employees can seamlessly access the various cloud-based applications using their existing access management protocol.

Portal for ArcGIS is compliant with SAML 2.0 and integrates with identity providers that support SAML 2 Web Single Sign-On. The advantage of setting up SAML is that you do not need to create additional logins for users to access your ArcGIS Enterprise portal; instead, they use the login that is already set up in an enterprise identity store. For example, Microsoft Azure Active Directory is a SAML-compliant identity provider. You can configure it as your identity provider for enterprise logins in Portal for ArcGIS on-premises and in the cloud.

I hope you find this helpful, do not hesitate to post your questions here: - Arcgis Architecture Series : Moving to the Cloud

more
10 0 9,269
JeffDeWeese
Esri Contributor

Often the term "VDI" is used to define ArcGIS Desktop/ArcGIS Pro deployed as a virtual application. The challenge is understanding what specific virtualization technology is actually being referenced when using this term since VDI, or "Virtual Desktop Infrastructure", represents just one of several desktop virtualization options. So, the intent of this article is define the options and differentiate what VDI truly means.

ArcGIS Desktop has been delivered virtually for over 20 years using what is referred to as "hosted virtual applications" which includes technologies such as Citrix XenApp (recently renamed to Virtual Apps) and Microsoft Remote Desktop Services (RDS). This approach is referred to as "hosted"  because it is being hosted by a singular operating system which users share by initiating individual user sessions. This technology option represents a many-to-one relationship in terms of users and virtual machines. Further, the shared operating system is a server OS, such as Windows 2016 and not a desktop OS, such as Windows 10. Hosted virtual applications provides a means to share a singular server with multiple users and is an attractive option since since each user doesn't require their own dedicated virtual machine. For this approach, system resources are shared including processors, memory, and GPU and there isn't a practical way to assign resources at the individual user session level.

A more recent innovation is to provide individual virtual machines to users as "virtual desktops" where each user accesses a remote desktop deployed with a desktop operating system such as Windows 10. This includes technologies such as Citrix XenDesktop (recently renamed to Virtual Desktops) and VMware Horizon. This approach represents the true meaning of "VDI" as it is defined by a one-to-one relationship between users and virtual machines. Though this approach increases per-user deployment costs, it also provides a more isolated deployment in terms of resources since processors, memory, and GPU resources can be assigned accordingly. The ability to manage GPU resources for the virtual desktops has made this approach an attractive option for ArcGIS Pro which requires a GPU.

So, the next time you hear the term "VDI" used for delivering ArcGIS to users, know that this implies that each user is being presented with their own individual Windows desktop virtual machine with a set of assigned resources as opposed to users accessing a singular server-based virtual machine and sharing it with multiple users, including sharing the server's assigned system resources. 

more
6 0 2,055
JacobBoyle412
Esri Contributor

Amazon and Esri recently published a whitepaper outlining the steps needed to setup and configure Amazon AppStream 2.0 and ArcGIS Pro. 

Through testing, Esri and AWS outline the various classes of AppStream hosts:

ArcGIS 2D Workloads – stream.compute.large, stream.memory.large. Compute and Memory optimized instances are perfectly suited for ArcGIS Pro workloads that does not require a GPU.

ArcGIS 3D Workloads (Normal) – stream.graphics-design.xlarge. Graphics Design instances are ideal for delivering applications such as ArcGIS Pro that rely on hardware acceleration of DirectX, OpenGL, or OpenCL. Powered by AMD FirePro S7150x2 Server GPUs and equipped with AMD Multi user GPU technology, instances start from 2 vCPU, 7.5 GiB system memory, and 1 GiB graphics memory, to 16 vCPUs, 61 GiB system memory, and 8 GiB graphics memory.

ArcGIS 3D Workloads (High res) – stream.graphics-design.2xlarge or stream.graphics-pro.4xlarge. The Graphics Pro instance family offers three different instance types to support the most demanding graphics applications. Powered by NVIDIA Tesla M60 GPUs with 2048 parallel 4 processing cores, there are three Graphics Pro instances types starting from 16 vCPUs, 122 GiB system memory, and 8 GiB graphics memory, to 64 vCPUs, 488 GiB system memory, and 32 GiB graphics memory. These instance types are ideal for graphic workloads that need a massive amount of parallel processing power for 3D rendering, visualization, and video encoding, including applications such as ArcGIS Pro.

Please find the full whitepaper here: https://d1.awsstatic.com/product-marketing/AppStream2.0/Amazon%20AppStream%202.0%20ESRI%20ArcGIS%20P...   

more
1 3 3,558
JacobBoyle412
Esri Contributor

What is System Log Parser?

System Log Parser is an ArcGIS for Server (10.1+) log query and analyzer tool to help you quickly quantify the "GIS" in your deployment. When run, it connects to an ArcGIS for Server instance on port 6080/6443/443 as a publisher (or an administrator), retrieves the logs from a time duration (specified as an input), analyzes the information then produces a spreadsheet version of the data that summarizes the service statistics. ArcGIS Monitor uses the command-line version of System Log Parser (slp.exe) for data capture.

System Log Parser supports the following service types:

  • Feature Services
  • Geoprocessing Services
  • Network Analyst Services
  • Geocode Services
  • KML Services
  • Stream Services
  • GeoData Services
  • Map Services
  • Workflow Manager Services
  • Geometry Services
  • Image Services

 

  • Globe Services
  • Mobile Services

 

System Log Parser (github.com), a free-standing application or Add-on for ArcGIS Monitor, is an effective tool for diagnosing and reviewing infrastructure functionality.

Getting Started

This section will configure ArcGIS Server to collect logs at the level needed for the tool and set up System Log Parser to generate a report (MS Excel).

1.   Ensure the following conditions are met on the machine you’ll be running System Log Parser from:

  1. 64-bit Operating System:
    1. Windows 7 (64 bit), Windows 8.x, Windows 10
    2. Windows Server 2008 64 bit, Windows Server 2012, Windows Server 2016
  2. RAM: 4 GB
  3. Microsoft .NET Framework 4.5 or 4.6
  4. Microsoft Excel 2010 or newer (or appropriate .xlsx viewer).

2.   Set your ArcGIS Server logs to Fine on EACH server you’d like to get metrics on. Complete instructions on how to change ArcGIS Server log levels can be found here:  Specify Server Log Settings

Note:    I recommend running the logging at FINE for AT LEAST one week prior to running System Log              Parser. This should give you a fairly clear picture of a typical weeks load.

3.   Download and extract System Log Parser here: github.com

4.   Extract the .zip file.

Note:    This is BOTH the user interface and the Add-on for ArcGIS Monitor.  We will be focused on the user interface version for this exercise.

5.   Launch System Log Parser

6.   Browse to the location you extracted System Log Parser

7.   In the System Log Parser for ArcGIS folder, locate and launch SystemLogsGUI.exe

System Log Parser GUI

Note:    You may be prompted that Windows has protected your PC.  If you do get this prompt, please click              More info and then click Run Anyway.

Configuring System Log Parser

The following outlines the configuration required to set up System Log Parser to analyze a week's worth of logs.

Note:    The System Log Parser will automatically access logging for all clusters that are part of an ArcGIS              Server Site. If you have multiple ArcGIS Server Sites configured

Click the ArcGIS Server (Web) button to display the following:

Fill out the above form as indicated below:

1.   Enter the Server URL.

  1. The typical syntax with ArcGIS Server 10.2 or higher is: https://<host_name>:<port_number>/arcgis  
  2. The typical syntax with ArcGIS Server 10.1 is: https://<host_name>:<port_number>/ArcGIS
Note:    If your URL structure is different, enter it.

2.   Enter the ArcGIS Server Manager user name with a publisher or better permissions. 

3.   Enter the user's password

4.   Check this box if you are accessing a Site federated to Portal for ArcGIS

Note:   Consider using a web adapter address for the Server URL:  https://<webadaptor_name>/server
Note:   If accessing over the internet, this assumes that the web adapter was registered with administrative access to ArcGIS Server

5.   Check this box if you use IWA(Integrated Windows Authentication)

6.   If needed, specify a token(advanced option)

7.   Select an End Time (Now)

8.   Select Start Time (1 week)

9.   Select Analysis Type (Complete)

  1. Simple: Provides only the Service Summary page data. 

    Note: This mode will also generate a list of the underlying data source by service and layer in the service. 

  2. WithOverviewCharts: Provides the Service Summary page plus charts of Request Count, Average Request Response Time, Instance Creation Time, Wait Time (Queue Time), and Max Request Response Time.

  3. Complete: Provides a Service Summary page plus all data and charts in separate tabs for all services.

  4. ErrorsOnly: Provides a report of just the errors.
  5. VerboseMode: Provides full verbose log analysis (Limited to 12 hours).

10.   Select Report Type (Spreadsheet)

11.   Specify where to output the report (Default is your My Documents location)

Click Analyze Logs. Analyze Logs

This process can take a few minutes or longer; this all depends on the number of transactions logged.

Review the System Log Parser report

 

When System Log Parser finishes running, it will open the report in Excel if present.  If you run this from a machine without Microsoft Excel, move it to a computer with Excel and open it.

You will note a summary tab and several tabs listed across the bottom of the spreadsheet.  We'll cover each in further detail below, by tab.

Summary

When the Excel report opens, you will see the Summary tab. The below screen grab shows what server this was run against and some summary statistics.

Summary

Statistics

On the bottom of the Excel report, select the Statistics tab to view a table of all services by layer and service types. This is where we'll spend most of our time.  Please read the rest of this post, then click here.

Resources

On the bottom of the Excel report, select the Resources tab to view several charts:

  • Top 20 Resources by Count
  • Top 20 Resources by Average Response Time
  • Top 20 Resources by Maximum Response Time

Methods

On the bottom of the Excel report, select the Methods tab to view several charts:

  • Top 20 Methods by Count
  • Top 20 Methods by Average Response Time
  • Top 20 Methods by Maximum Response Time

Queue Time

At the bottom of the Excel report, select the Queue Time tab to view any services that had to wait for an ArcSOC to return a result. In an ideal setting the below is the desired value:

Queue Time Stats

Users

At the bottom of the Excel report, select the Users tab to view a chart of the top 20 users by request count.

Time

At the bottom of the Excel report, selects the Time tab to view a chart of requests by day.

Throughput per Minute

At the bottom of the Excel report, select the Throughput per Minute tab to few a minute-by-minute breakdown of requests.

Below is a sample of what information can be found on the tab:

Throughput Per Minute

Elapsed Time of All Resources

At the bottom of the Excel report, select the Elapsed Time of All Resources tab to view a chronological listing of all requests from the time period the System Log Parser report was generated.

I'd also like to thank Aaron Lopez‌ for his help and continued development of this invaluable tool. 

Note: The contents presented above are recommendations that will typically improve performance for many scenarios. However, in some cases, these recommendations may not produce better performance results, in which case, additional performance testing and system configuration modifications may be needed.

I hope you find this helpful, do not hesitate to post your questions here: ArcGIS Architecture Series: Tools of an Architect.

more
11 9 13.3K
JacobBoyle412
Esri Contributor

What is System Log Parser?

System Log Parser is an ArcGIS for Server (10.1+) log query and analyzer tool to help you quickly quantify the "GIS" in your deployment. When run, it connects to an ArcGIS for Server instance on port 6080/6443/443 as a publisher (or an administrator), retrieves the logs from a time duration (specified as an input), analyzes the information then produces a spreadsheet version of the data that summarizes the service statistics. The command-line version of System Log Parser (slp.exe) is used by ArcGIS Monitor for data capture.

 

Note:   This post is a second in a series on System Log Parser, please see ArcGIS Server Tuning and Optimization with System Log Parser to learn how to setup your server for System Log Parser and an overview of the report.

 

Introduction to Statistics Used In System Log Parser

 

There are several statistical categories you should be familiar with when using System Log Parser. (definitions from Wikipedia)

 

Percentile (P) - a measure used in statistics indicating the value below which a given percentage of observations in a group of observations falls. For example, the 20th percentile is the value (or score) below which 20% of the observations may be found. 

 

Average (avg) -   is a single number taken as representative of a list of numbers. Different concepts of average are used in different contexts. Often "average" refers to the arithmetic mean, the sum of the numbers divided by how many numbers are being averaged. In statistics, mean, median, and mode are all known as measures of central tendency, and in colloquial usage, any of these might be called an average value. 

 

Maximum (Max) - the largest value of the function within a given range.

 

Minimum (Min) - the smallest value of the function within a given range.

 

Standard Deviation (Stdev) -    measure that is used to quantify the amount of variation or dispersion of a set of data values. A l...

 

Fields of the Statistics Collected

 

Field
Definition

Resource Requested resource or service (Service REST endpoint)
Capability The ArcGIS capability of the resource
Method The function performed by the resource (What was accessed)
CountThe number of requests for this resource
Count Pct Count percentage based on total service requests
Avg The average time (in seconds) spent processing request
MinThe time (in seconds) of the shortest request
P5, P25, P50, P75The percentile grouping of the time (in seconds)
P9595% of all responses occur between 0 seconds and the value displayed in this column per service
P9999% of all responses occur between 0 seconds and the value displayed in this column per service
MaxThe time (in seconds) of the longest request
StdevThe standard deviation of time (in seconds)
SumThe total time (in seconds) spent processing requests per resource
Sum PctThe total time (in seconds) spent processing requests

 

We're going to focus on 2 key statistics, P95 and Max.  As we learned above, P95 signifies the response time for the fastest 95% of all requests and Max signifies the maximum draw time per request per service and method.

 

Identifying Opportunities to Tune Service Performance

 

In the example below, I've sorted P95 and Max values over 1/2 second.  User experience drops the longer your draw-time takes. 

 

I've highlighted any Max draw time over 1/2 second in red and any P95 draw time over 1/2 second in yellow.  These are the services and layers I'd focus on cleaning up, focusing first on getting the P95 value below 1/2 second first. 

In the next section you'll find starting points to tune and optimize your services.

 

Another column worth reviewing is the Sum Pct.  this column factors in the number of requests for each service and the respective average time, then weights that in against all the other services.

 

Sum Pct

 

For example:   

  1. One service may have thousands of more requests than all others but it has fast times (Sum Pct should be low)
  2. Another service may have just a small handful of requests but very slow times (Sum Pct should be high). In this case, this service would be a good candidate to for tuning.

 

Best Practices for Services

 

Below are some links to get you started on service tuning and SOC management.

         

In addition to the above, data source performance should be looked at if adjustments to the service do not help enough. You can look at:

 

I hope you find this helpful, do not hesitate to post your questions here: https://community.esri.com/thread/231451-arcgis-architecture-series-tools-of-an-architect

 

Note: The contents presented above are recommendations that will typically improve performance for many scenarios. However, in some cases, these recommendations may not produce better performance results, in which case, additional performance testing and system configuration modifications may be needed.

more
15 13 12.5K
123 Subscribers