arcgisinput folder: What is it for?

[jcarlson]

I'm doing some maintenance on our Enterprise machines, and I happened to notice that some of the directories in our ArcGIS Server machine were not clearing out automatically. This meant that the output folder was getting quite large, when it should have been dropping files older than 10 minutes.

While I was dealing with that, I noticed that the directory arcgisinput was also quite large. According to the description, this folder

Stores source files (such as maps and data) for services that you publish.

It also appears to have no cleanup mode defined, which would imply that this folder is not meant to be cleared out. But I don't really understand what these source files are doing, if anything, and can't find any good documentation that explains it in more detail.

Key question: if I remove these files, will anything be impacted?

Most of our services are hosted. The Portal machine holds all the service definition files used to publish the services, so is anything using the raw files in the input directory?

In particular, I'm thinking about a few hosted imagery tile layers we have published. The service.sd file is about as big as the corresponding directory in arcgisinput. If I tell the service to build tiles, isn't it looking at the service definition when it does that?

It just seems like keeping these files as well as the service definitions leads to unnecessary duplication and requires extra storage space.

[DavidColey · Accepted solution]

So the input folder is used by server to store source copies of the services that you publish from a registered data-store location, e.g. from a file gdb in a registered folder or an sde connection registered as a database in the server data store. 

Its just the way it works, server needs copies of the source documents in the directories it needs them in:

The domain arcgisserver system account handles all writes to the input folder when you publish or overwrite a service, not users.  When you delete a map service, the service's input folder and subfolders should be deleted as well.  If the input service name folder and subfolders do not delete when you delete a map, image or gp service, then you have or had other problems, probably related to sharing, where the arcgis server domain account could not fully access all the directories.

If you have an 'orphan' directory, make sure the reference is no longer in the portal content (if you are federated) before deleting it manually.

There should be no cleanup mode set on the input folder. 

For the output folder, all files created by the arcgis server service account (via the REST endpoint are cleared out by the time set, usually 10 minutes.  What I have found is files in the output directory that are not auto-cleaned are lyr files created by SOAP when someone at sometime connected to a service in ArcMap.  

 Hope this helps-

David Coley

Sarasota County GIS

[HenryLindemann]

I hade a look at what is in there, from what I do on a day to day deploy, audit, recover , migrate.
I would need the data in these folders for example the v101 and p20 p14 folders contain the original mapx and mxd these always get lost, and the manifest provides information on where the data originated from the users etc.

and there might be some other features as well.

In my chase the folders are less than a 1 MB, but I see you have SD files and these can be large, in ArcGIS Enterprise 10.8.1 - 10.9 the sd files are moved to portal and then you can delete them but you can also use them to rebuild the service should it crash

if there are still folders after you removed a service that is not right,  there might be a script problem if you use that I don't know how you remove services currently.

Kind Regards

Henry 

[DavidColey]

Ok, so  on a limb here - I think it's fine to know what's in the system folders for server, but they are system folders for a reason.  The system account needs to manage these, not the administrator.  In almost every case I can think of, going into these folders an manually deleting, updating, writing to or adding files is totally unadvisable.  

[jcarlson]

I would agree with that, and I don't usually touch system files unless I'm 100% sure of what I'm doing. I only happened to notice it when I was clearing out some stuff that didn't delete when I removed a service.

As @HenryLindemann notes, it's good to hang onto SD files, but in the case of raster imagery, I've got a 10GB file, and unless I switch to keeping that file offline somewhere, I basically have to allocate 20GB of storage to publish the service, and that's before caching any tiles. But I guess that's just the way things are.

Another question, then: does DataStore not store raster imagery, just vector layers?

[DavidColey]

no it doesn't, not like a file gdb or an enterprise rdbms can store a blob or raster data type.  Like when you create a tile cache for a hosted feature layer, that tile cache is going to reside in the cache directories, not in the relational datastore.  At least that's my understanding....

For imagery, we use the image server role on our federated site.  The source imagery resides in an enterprise file share, like a san, and the mosaic datasets that source the image services reside in a file gdb on the federated sites' data-store directory