Allow deleted services to be ignored from logging

[stevegourley]

ArcGIS Server logs two items when a request for a service does not exist; A java exception and a nicer log that X service was not found. While I understand it can be important to know when requests are being made for services that do not exist, it would be nice to remove some of these requests entirely as they produce a lot of noise.

We delete services when they are no longer required by an application. We specifically tell people to not take them on as a dependency in their own applications, maps, and other systems. Yet, we find our logs piling up with messages that service x no longer exist; 1000's per week. It is almost impossible to find and notify these people to update their services based on the logs provided.

We are able to block REST requests at our load balancer so they do not show up in our arcgis server logs but SOAP requests are more difficult to block.

If there was an admin configuration endpoint area where we could maintain a list of services we know to be deleted and do not want to be warned about any longer, we could reduce the amount of noise in the logs to help find errors that are more important.

[berniejconnors]

Are you using a web adaptor? You should be able to filter your web adaptor logs for records that include your deleted map services. Those records will include the ip address of the user trying to access the deleted map service. I have used this method to find users trying to connect to a deleted map service.

I have also used the web adaptor logs to find long running requests and help my users create more efficient queries.

But that's not really your problem. You have a communication problem. You need to do a better job communicating with your users. We have created a process for retiring a map service. Step one is tell your users that a service is being retired on a certain date. Step two you remove the data from the map service and replace it with a single polygon with a label that says this service is being retired and possibly the url of a replacement map service. Keep this message in place for several months and then finally delete the map service and block traffic to the deleted map service.

Bernie.

[stevegourley]

@berniejconnors no we do not use a web adapter and instead we use a load balancer. How do you find and notify a public facing user of a service if all you have is an ip?

I appreciate your concern for our communication problem. We have a public policy and public process for deprecating everything that we do. It's all visible on our website and on our tracking site.

Thank you for you feedback although it was rather unhelpful.

Changing the log level from SEVERE and WARNING for these 404's to an INFORMATION would be also be an improvement in my opinion.

[berniejconnors]

@stevegourley You are right, an IP address is useless.  I meant to say I used the referrer URL.  I was able to use the referrer URL to find several poorly configured Web AppBuilder sites that were sending long running queries for simple Parcel ID searches.  The referrer URL included the AGOL item ID so I was able to find the apps and contact several municipal GIS teams to help them make corrections to the search widget config.

Thanks for sharing the SGID links.  They were very informative.  And it does not look like you have a communication problem.  Your github repo is a great tool to document changes to your datasets and services - but you cannot force your users to read it 😉

Bernie.