nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

10.5.1 Memory Usage

DaveTenney

All,

i was curious if anyone else encountered an interesting behavior once upgrading desktop and server to 10.5.1....

We have noticed that after upgrading and running the "manage map server cache tiles", esri ultimately maxes out the memory, and this usually happens 90min after starting the tool. We have run this process many times on the data and not until after we upgraded to 10.5.1 are we now experiencing this.

Machine Specs

Intel Xeon CPU E5-4620 v2

Windows Server 2012 R2

16gb Ram

4 Cores

thanks,

dave

Encontre mais publicações marcadas com

Accepted answers

All comments

George_Thompson

What version where you on before the upgrade?

How many services are on the machine?

DaveTenney

10 services total: 2 cached map service, 1 dynamic map, 7 locator services

we were running 10.2.2, when we upgraded we rebuilt all the services manually using 10.5.1. we were able to rebuild the services manually just fine. now we create an area of interest that is built from a delta check and we get to about level 16 out of 19 and the cachingtools begin consume massive amounts of memory.

we start seeing entries in the cachingtools logs that mention "out of memory", then the process fails. with a similar message:

Data access failure, layer = Image, error string = Could not access data for layer or table Image during drawing§TilesWorker: Data access failure, layer = Image, error string = Could not access data for layer or table Image during drawing§TilesWorker: Data access failure, layer = Image, error string = Could not access data for layer or table Image during drawing Failed to cache extent:........

George_Thompson

Thanks for that info, how many instances on the caching services?

How many instance are running on the other 10 services?

I have seen other posts (cannot find right now) that users had something similar with newer versions consuming more memory than previous ones.

Is the machine (VM or physical) actually out of memory when the error occurs?

MichaelSchoelen

Could it be a setting with the pooling parameters?

DaveTenney

Caching services were set to a max of 3, we decided to take the n-1 approach (n = # of cores) in order to prevent the resources from being maxed out on the machine. As for all the services, they run at a max of 2.

yes, i can sit and watch the gptool process run at the same time tracking the machine stats and before too long, the memory just starts jumping up like crazy and ultimately maxing out.

DaveTenney

Michael,

i doubt it is pooling as we have taken a conservative approach with the caching services.

we are running the gptools at a max of 3 on a 4 core machine, we did this because we did not want to take the chance to max out the machine. unfortunately, we are now seeing something that runs wild with the available memory.

thanks

dave

MikeVolz

What kind of data is changing in your problematic cached mapservice (e.g. parcels, road centerlines, other)?

How often are you upating the cache?

On average, how many features are changing per cache update?

What is your largest scale that you are caching? Can you see if the problem still occurs if you don't include the 3 or 4 largest scales, just as a test?

Is this process occurring when there is low or no load on the AGS server?

Dynamic mapservices perform much better in 10.5.1 so you might consider removing layers from cache and make them dynamic.

DaveTenney

1) we have roads, addresses, other individual point, and line features.

2) this update process runs on a weekly basis

3) on average we see 20k-50k changes total

a. this is why we do the delta b/c so little changes that we don't want to waste time rebuilding everything

4) when we rebuild tiles we do it in two steps

a. forced scales - these are level 0-15, very little data total so we just rebuild them all (usually only 10min)

b. selective scales - this is levels 16-19, this is where we implement our area of interest based on the delta results

1) it is within the selective scales that we see the memory usage spike even faster and ultimately failing

5) the sole purpose of this machine/server is to build tiles, nothing else runs on this machine, we did not want anything interrupting this process or bogging down the machine to slow the process.

note: i really wanted to separate the vector data from the raster data, i feel that the sheer size of all the data due to them being combined is an issue in itself. the raster data only changes once a year, whereas the vector data changes weekly. the reasoning behind combining them is so we would not have to make two separate requests for the vector and raster data, thus bogging down the bandwidth of the system.

dave

DaveTenney

I wanted to share more info with everyone and hopefully shed some more light on why this is very perplexing...

i have two images attached...

1) AOI - this is running the "managemapservercachetiles" on level 19 whit an area of interest polygon

a. there is very little work to be done with recreating all tiles within the area of interest.

2) ALL - this running the "managemapservercachetiles" on level 18 and level 19

a. there is a lot of work to be doing recreating all tiles in these two levels

AOI - memory is running up like crazy and has almost maxed out the machine and is about to fail

a. why would this smaller amount of work cause the memory usage?

ALL - the memory usage has stayed consistent at this level throughout the entire process, the cpu usage jumps up and down but i have always seen that behavior.

what in the world is going on?

MikeVolz

Can you provide a screenshot of either the tool with the parameters filled in or the python call to ManageMapServerCacheTiles_server with all of your parameters?

It's been a long time since I had this process running in 10.2, but I thought the parameters for this call changed between 10.2 and 10.5.1 which could be part of the problem you are seeing.

DaveTenney

here is the screenshot of the tool:

DaveTenney

not sure why it showed up this way, but when we run the tool we leave "wait for job completion" checked, I believe that is the default. again, not sure why in this screenshot it is not checked.

MikeVolz

Can you take a screenshot of the results showing Inputs from a cache update that would not take that long just to get an idea of what that looks like?

Have you made sure that the AOI geometry in the delta.gdb is reasonable and your script to create the AOI is not creating an area much larger than what you are expecting?

DaveTenney

here are the inputs as seen from geoprocessing results.

DaveTenney

all,

would installing and utilizing background geoprocessing assist in memory usage?

I wasn't sure if the background geoprocessing was utilized for the ManageMapServerCacheTiles gptool

thoughts?

dave

Participe da Conversa

Membros conectados podem postar, seguir atualizações e mais. Novo aqui? Registre uma conta gratuita.

Links Rápidos

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.