topic Re: Added scale to image service cache, but tile generation refuses to complete in ArcGIS Image Server Questions

Added scale to image service cache, but tile generation refuses to complete

deleted-user-zpcJw1u0IXiO — Mon, 15 Aug 2016 17:39:54 GMT

I have an image service I've successfully cached previously. It had four empty scales 1:50, 1:100, 1:250, and 1:500 I created in order that the Geocortex viewer would permit zooming closer than the cached levels.

I have found need to populate the cache at 1:500, and was able to get perhaps a third of the service extent to do so. The table below shows the Cache Status where it is stuck, despite that Level 17 is currently underway for the dozenth or so attempt.

Level

Scale

Size

Expected Tiles

Completed Tiles

Percent

In Progress

0	1,000,000	0.18 MB	6	6	100
1	500,000	0.61 MB	18	18	100
2	250,000	2 MB	55	55	100
3	125,000	4.36 MB	210	180	85.71
4	100,000	6.76 MB	312	275	88.14
5	64,000	9.87 MB	760	624	82.11
6	50,000	13.92 MB	1,200	960	80
7	32,000	27.75 MB	2,886	2,356	81.64
8	25,000	39.45 MB	4,653	3,744	80.46
9	16,000	65.76 MB	11,242	9,150	81.39
10	10,000	162.53 MB	28,405	23,183	81.62
11	8,000	228.27 MB	44,352	36,058	81.3
12	5,000	471.78 MB	113,390	91,868	81.02
13	4,000	742.54 MB	177,408	143,395	80.83
14	2,500	1.91 GB	453,560	366,520	80.81
15	2,000	2.79 GB	708,400	572,390	80.8
16	1,000	11.01 GB	2,832,450	2,285,258	80.68
17	500	4.47 GB	11,327,500	2,583,061	22.8
18	250	0 MB	45,310,000	0	0
19	100	0 MB	283,126,752	0	0
20	50	0 MB	1,132,434,765	0	0

The trouble is the tile count never changes, despite my running Manage Map Server Cache Tiles with each attempt using an Update Extent shapefile covering a progressively smaller fraction of the area needed. So after three hours on an extent of just 30 sq. miles, the job details URL reports 0% completion on each line, with a different Estimated Time Remaining that grows larger and larger before dropping back to begin growing larger again.

The Logs in AGS Manager report what I suspect may be the reason for no tile creation:

SEVERE

Aug 15, 2016, 1:21:23 PM

Error exporting image Wait time of the request to the service 'public/swoop2015_mosaic.ImageServer' has expired.

Rest

SEVERE

Aug 15, 2016, 1:21:23 PM

Unable to process request. No instances for 'public/swoop2015_mosaic.ImageServer' were available for 60.006 seconds. Wait timeout exceeded.

public/swoop2015_mosaic.ImageServer

SEVERE

Aug 15, 2016, 1:20:41 PM

Error exporting image Error handling service request :Processing request took longer than the usage timeout for service 'public/swoop2015_mosaic.ImageServer'.

Rest

SEVERE

Aug 15, 2016, 1:20:38 PM

Processing request took longer than the usage timeout for service 'public/swoop2015_mosaic.ImageServer'. Server request timed out. Check that the usage timeout is appropriately configured for such requests.

public/swoop2015_mosaic.ImageServer

I don't know what to try next. I have already restarted the image service with no change in behaviour.

Short of contacting Esri tech support, what should I try to correct this issue?

Thanks,

Justin

Re: Added scale to image service cache, but tile generation refuses to complete

JenniferMcCall4 — Mon, 15 Aug 2016 17:52:30 GMT

Could it be that another process is holding on to the instances need to cache this? Have you tried re-starting the ArcGIS Server service in windows?

What does your CPU sit at over the 3 hours?

Re: Added scale to image service cache, but tile generation refuses to complete

deleted-user-zpcJw1u0IXiO — Mon, 15 Aug 2016 17:59:33 GMT

Thank you Jennifer. I didn't notice the CPU usage before cancelling the job shortly after my post. How can I determine what other process might be holding on to the instance, which I assume you mean to be the image service? Perhaps it might be that I publish the same service dynamically to handle the larger, non-cached scales. Should I stop that service to see if that affects things?

To restart ArcGIS Server service in windows, do you mean in Manager, or in Windows Services?

Re: Added scale to image service cache, but tile generation refuses to complete

deleted-user-zpcJw1u0IXiO — Mon, 15 Aug 2016 18:06:30 GMT

I've launched the MMSCT again, after stopping the dynamic image service, and CPU usage ranges between 20% and 65%, but the server has active map users. I've still not restarted any services.

Re: Added scale to image service cache, but tile generation refuses to complete

JenniferMcCall4 — Mon, 15 Aug 2016 18:07:17 GMT

Hi Justin,

I was referring to the ArcGIS Server service within Windows Services. Stopping and then starting it should remove any potential locks from other services or even from processes previously run on this image service. After starting the windows service again, wait a minute or two until ArcGIS Server and all arcgis services (ex; the image service) are back up and running. Then try to cache just level 17 again and see if it makes a difference.

When you publish the service, are you checking the box to manually create the cache after the service is published?

Another source of information you could check is the Status.gdb within the cache folder. It will only be visible through Windows Explorer, so copy this gdb to another location (ex; your Desktop) and then you can open the GDB and feature classes within ArcMap. This may provide more information on what might be happening with your service. I would recommend copying the GDB after you have tried to re-cache level 17 so any information about the job will be populated in the feature classes.

Re: Added scale to image service cache, but tile generation refuses to complete

JenniferMcCall4 — Mon, 15 Aug 2016 18:12:49 GMT

Is this percentage where you expect the CPU to sit while caching, or while there is no caching and active users?

Re: Added scale to image service cache, but tile generation refuses to complete

deleted-user-zpcJw1u0IXiO — Mon, 15 Aug 2016 18:13:57 GMT

okay thank you. I'll try restarting as you described. I realized just moments ago the mistake of turning off the dynamic image service (as I said I did), because the cached service needs the dynamic one as its source. And AGS logs screamed at me for the source service not being found. Silly me.

I looked at status.gdb last week, and that's how I knew which areas remained uncached, and used that to create polygons to use as the small update extents.

Re: Added scale to image service cache, but tile generation refuses to complete

deleted-user-zpcJw1u0IXiO — Mon, 15 Aug 2016 18:43:16 GMT

yes, the service was already set for manual cache creation.

I did restart the Windows ArcGIS Server service, but neglected to cancel caching before doing so. I clicked Cancel Job in the Job Details page, but after 10 minutes, it still says esriJobCancelling, so I don't know what more to do to make it actually stop so that I may restart it

Re: Added scale to image service cache, but tile generation refuses to complete

JenniferMcCall4 — Mon, 15 Aug 2016 18:46:54 GMT

Stopping the ArcGIS Server service should kill the job, not leave it cancelling. How long did you wait between stopping it and starting it up again. It's beneficial to wait at least 30 seconds.

Re: Added scale to image service cache, but tile generation refuses to complete

deleted-user-zpcJw1u0IXiO — Mon, 15 Aug 2016 19:04:01 GMT

I had left it only a second or two before starting it again. So I've stopped it once more, this time waiting more than 30 seconds, but with all services back up, the job still reports as being in progress (in Manager Cache Status).