- Home
- :
- All Communities
- :
- Services
- :
- Implementing ArcGIS
- :
- Implementing ArcGIS Blog
- :
- Automating System Log Parser from the Windows Comm...
Automating System Log Parser from the Windows Command Line (Beginner/Intermediate)
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
System Log Parser's GUI and Command Line
The System Log Parser (SLP) utility is great for being able to analyze many different "ArcGIS Enterprise" log sources to help quantify the usage of your Site. It is free and includes an easy-to-use graphical user interface (GUI) called SystemLogsGUI.exe. But, did you know it can also be run from the command line via the slp.exe that is included with the download?
Command line capable execution makes a good fit for automation through PowerShell or good old fashioned bat scripts.
The slp.exe executable includes all of the same capabilities. However, you can override many of the limits imposed by the GUI such as: the end time and start time. You can even access features that are not available from the GUI like: searching with specific UTC dates, customizing the name of the report, and enabling the application debug log (for troubleshooting).
The Most Popular Log Sources
The exact command and options will vary slightly depending and the log source you are wanting to use for the analysis. Let's start with the most popular log sources using some recommended options:
ArcGIS Server Log Query (File System)
C:\SystemLogParser> slp.exe -f AGSFS -i "\\myserver.domain.com\c$\arcgis\arcgisserver\logs\MYSERVER.DOMAIN.COM" -eh now -sh 30day -a optimized -validate true
or
C:\SystemLogParser> slp.exe -f AGSFS -i "\\myserver.domain.com\c$\arcgis\arcgisserver\logs\MYSERVER.DOMAIN.COM" -eh now -sh 30day -a optimized -validate true
The -f AGSFS parameter specifies the system log parser read function...ArcGIS Server File System.
The -i "[location of logs]" parameter specifies the file system or network location of the folder containing the logs.
The -eh now and -sh 30day parameters specify the end hour and start hour. In this case, the end time (most recent time) will stop "now" (defined as the date time of whenever the command is executed) and the start time (furthest back in time) will begin 30 days ago.
Note: There is no authentication with this method, but access to the logs is granted via the network share or local file system permission. Only read permission is required.
Note: The file system is the fastest and most scalable way to read the ArcGIS Server logs.
Note: For ArcGIS Server Log Query (File System), it is possible to read logs from multiple servers into one report. This can also be accomplished with the "-i" option by separating each path with a comma. For example:
C:\SystemLogParser> slp.exe -f AGSFS -i "\\myserver1.domain.com\c$\arcgis\arcgisserver\logs\MYSERVER1.DOMAIN.COM,\\myserver2.domain.com\c$\arcgis\arcgisserver\logs\MYSERVER2.DOMAIN.COM,\\myserver3.domain.com\c$\arcgis\arcgisserver\logs\MYSERVER3.DOMAIN.COM" -eh now -sh 30day -a optimized -validate true
ArcGIS Server Log Query (Web)
C:\SystemLogParser> slp.exe -f AGS -s https://myserver.domain.com/server -u siteadmin -p AdminP@ssword -eh now -sh 7day -a optimized -validate true
or
C:\SystemLogParser> slp.exe -f AGS -s https://myserver.domain.com/server -u gisadmin -p Myp@ssword -pfl true -eh now -sh 7day -a optimized -validate true
The -f AGS parameter specifies the system log parser read function...ArcGIS Server Web.
The -s [ArcGIS Server instance URL], -u [administrator user], and -p [administraor password] parameters specify the options needed to authenticate to your ArcGIS Server.
Use the -pfl true parameter if user performing the query is using a Portal Federated Login.
The -eh now and -sh 30day parameters specify the end hour and start hour. The end time will stop "now" (defined as the date time of whenever the command is executed) and the start time will begin 7 days ago.
Note: While technically possible to read 30 days worth of logs through the web, it is not recommended, even when using the Optimized report. The reason is because for busy Sites, the overall process can be very resource intensive on the server machines with tokens or individual log requests timing out.
Note: Unlike reading the logs from the file system, you do not need to specify each individual server in the Site as the ArcGIS Server's REST Admin API does this automatically.
Internet Information Services Log Query
C:\SystemLogParser> slp.exe -f IIS -i \\myserver.domain.com\c$\inetpub\logs\LogFiles\W3SVC1 -eh now -sh 30day -a optimized -validate true
The -f IIS parameter specifies the system log parser read function...Internet Information Services.
The -i "[location of logs]" parameter specifies the file system or network location of the folder containing the logs.
The -eh now and -sh 30day parameters specify the end hour and start hour. In this case, the end time (most recent time) will stop "now" (defined as the date time of whenever the command is executed) and the start time (furthest back in time) will begin 30 days ago.
Note: There is no authentication with this method, but access to the logs is granted via the network share or local file system permission. Only read permission is required.
Note: The values included in them are a nice representation of "response times" which can help describe the experience the users encountered with respect to performance.
AWS ELB (Amazon Web Services, Elastic Load Balancer)
C:\SystemLogParser> slp.exe -f ELB -s mybucket1 -reg USWest2 -u ABC123DEF456 -p n4b5n74v75 -eh now -sh 7day -a optimized -validate true
The -f ELB parameter specifies the system log parser read function...Elastic Load Balancer.
The -s [bucket name], -u [access key], -p [secret key], and -reg [AWS region] parameters specify the options needed to authenticate to your account.
The -eh now and -sh 7day parameters specify the end hour and start hour. In this case, the end time (most recent time) will stop "now" (defined as the date time of whenever the command is executed) and the start time (furthest back in time) will begin 7 days ago.
Note: System Log Parser can query large time spans (e.g. greater than 7 days) of ELB logs, but to significantly improve the download concurrency and reading performance for such large queries (e.g. 30days worth of logs), it is recommended to run slp.exe through the Get-SLPReport.ps1 PowerShell script provided with the System Log Parser zip. This script utilizes multiple PowerShell threads to quickly download and unzip the logs. It then uses the ELBFS log source function (-f ELBFS) to read them from the local disk.
Note: The Get-SLPReport.ps1 script requires at least PowerShell version 7 to run.
Useful Universal Options
The -a optimized parameter
All of the examples commands above use the Optimized report which is the recommended Analysis Type.
Note: The Optimized report has tremendous memory savings over the other analysis types, especially if you are reading 30days worth of logs.
The -validate true parameter
To assist in troubleshooting, the -validate true is passed into the slp.exe command to print any warnings or errors to the console that occur. Without it, the default mode of slp.exe is to run silently.
The -o false parameter
When running log query through a scheduled script (e.g. Task Manager), it is usually desirable to suppress the automatic opening of the generated report. Setting -o to false will create the report but not open it when slp.exe completes.
The -apploglevel DEBUG parameter
For in-depth troubleshooting, you can optionally pass in -apploglevel DEBUG. This will create a unique run log of the slp.exe execution...with a large amount of detail. The log file (for an ArcGIS Server file system log query) would typically be found in a location similar to the following:
C:\Users\gisadmin\Documents\System Log Parser\Logs\Application_agsfsx2_20220702T220628_k6a7xrhb.log
The -d "C:\Users\[username]\Desktop\my slp reports" parameter
slp.exe allows the location of report to be set at generation time with the "-d" switch. If the "my slp reports" directory does not exist, it will be created.
The user running slp.exe will need write permission to "C:\Users\gisadmin\Desktop", otherwise an error will be thrown.
Note: In the case above, using -d "C:\Users\[username]\Desktop\my slp reports" will create the folder, if it does not exist. However, if it does not exist and a slash is appended (-d "C:\Users\[username]\Desktop\my slp reports\"), slp.exe will throw an error.
The -n "myreport123.xlsx" parameter
slp.exe allows the name of the report to be set at generation time with the "-n" switch.
Note: The file extension (e.g. ".xlsx") is not added automatically so be sure to include it if using the -n option.
The -endstring "2022/05/21 12:00:00 PM" and -startstring "2022/05/21 9:30:00 AM" parameters
slp.exe allows for surgical time ranges. You can pass exact end and start times (in UTC) which can be helpful for troubleshooting or when looking for specific date in the past. For examples, a particular spike in response times that occurred 10 days ago but only lasted for a few hours. In this case, you most likely don't need everything from 10 days ago until right now, but instead want to focus just on the event.
Note: Both the -endstring and -startstring should be passed strings that define time in the Coordinated Universal Time or UTC timezone.
Other Log Sources
slp.exe supports several other log sources such as Azure (-f azure), CloudFront (-f cloudfront) and Apache Tomcat (-f tomcat). However, these sources do not yet support the Optimized Analysis Type and querying large durations of times against busy Sites can result in some time consuming and resource intensive (on the machine running slp.exe) log parsing.
Apache Tomcat
A typical tomcat installation does not usually have the access logs enabled by default. However, they can be activated with a relatively simple change to the server.xml file.
- Open server.xml in your favorite text editor
- Ensure the the following is added and/or not commented out:
- <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" fileDateFormat="yyyy-MM-dd.HH" pattern="%h %l %u %t "%r" %s %b %v %p %D" prefix="localhost_access_log." resolveHosts="false" suffix=".txt"/>
- Save edits
- Restart Tomcat
Note: In this example, the suffix is set to ".txt" but using ".log" will also work with slp.exe
Latest Version
Bug fixes and new features are always being added to SLP.
The latest version can be found here: System Log Parser
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.