Create Map Tile Package tpk faster!

9877
10
Jump to solution
03-13-2015 11:15 AM
forestknutsen1
MVP Regular Contributor

We are using tpk files as basemaps for a Android application. Which works great. But they take a very long time to generate. The last one was around 8 or 10 days as I recall. I have one building now. It is on day 7 and at around 2.2GB. I would guess that in the end it is going to be over 5GB someplace. So, my question is how can we speed this up. Is it depended on cpu speed? Would getting a more powerful machine help? The source data is in a gdb on the machine's local drive and the output location is also on the local drive.

Right now I have it running on a Lenovo ThinkPad with:

Intel i7-4800MQ @ 2.70GHz

16GB RAM

NVIDIA Quadro K2100M

ArcGIS 10.2

Looking at task manager it does not look like I am using much of the machine's resources...

Capture.PNG

1 Solution

Accepted Solutions
JasonHarris2
New Contributor III

I have to do something very similar for a Android application every month...except I have to create several dozen very large TPK files.  It started to become unmanageable time wise. 

What I ended up do is creating a straight up ArcGIS Server Map cache for the area that I need the TPK for.  This way, you can crank it up to use every last bit of CPU you have available (I never could get the Parallel Processing Factor var to work right in py).  Then use the Export Map Server Cache tool to dump it out to a TPK. 

Scripted all out, this ended up being on the magnitude of 50x faster than just creating a TPK file the normal way.  Of course the caveat is that you need a ArcGIS Server available.  But, all you would need is a EDN level license to do it.  Hope that helps.

View solution in original post

10 Replies
MicahBabinski
Occasional Contributor III

Hi Forest,

Many tools within the Tile Caching toolset honor the "Parallel Processing Factor" environment setting. It may be worth boosting this factor if you're hoping to apply more of your computer's resources to the task. If executing these tools from Python, add

arcpy.env.parallelProcessingFactor = "100%"

to the top of your script. Otherwise, set this in the environment settings of the tool dialog in the stand-alone tool or in ModelBuilder. Hope this helps!

Micah

forestknutsen1
MVP Regular Contributor

Micah,

Thanks for the input - I was unaware of "Parallel Processing Factor" environment setting. I am going to take a look at this.

Thanks,

Forest

0 Kudos
forestknutsen1
MVP Regular Contributor

Well it looks like the tpk generation does not support the Parallel Processing Factor. It does not say that it is supported in the documentation. Also, I have run a small test and the generation time is the same with it set to 100%. But next time I do a big raster project I am going to start setting this. Thanks for the help Micah.

forestknutsen1
MVP Regular Contributor

I have started a new tpk job on a new machine with 10.3. This one is going quite a bit faster. It has overtaken first one in outout size in four days. The first on has been building for about 12 days.

As to why this it is going faster this time; I do not know for sure but I have done a few things differently:

1. I reprojected all of the source data to the SRS of the tpk. Before I was letting Arc reproject on the fly.

2. I changed layer transparency to colors. I am unclear how this would help with the generation.

AndySmith
Esri Contributor

You may also wish to think about if you need to have all of the information in the tiles packages and at all of the levels as well: Tips for creating tile packages—Help | ArcGIS for Desktop

0 Kudos
forestknutsen1
MVP Regular Contributor

Right, I am using a mxd that has scale levels set for the layers so that everything does not try to draw at all scale levels. I also have a tiling custom scheme that I have made up for the area....

0 Kudos
JasonHarris2
New Contributor III

I have to do something very similar for a Android application every month...except I have to create several dozen very large TPK files.  It started to become unmanageable time wise. 

What I ended up do is creating a straight up ArcGIS Server Map cache for the area that I need the TPK for.  This way, you can crank it up to use every last bit of CPU you have available (I never could get the Parallel Processing Factor var to work right in py).  Then use the Export Map Server Cache tool to dump it out to a TPK. 

Scripted all out, this ended up being on the magnitude of 50x faster than just creating a TPK file the normal way.  Of course the caveat is that you need a ArcGIS Server available.  But, all you would need is a EDN level license to do it.  Hope that helps.

forestknutsen1
MVP Regular Contributor

That sounds interesting. We do have ArcGIS Server available. Thanks Jason.

0 Kudos
JasonHarris2
New Contributor III

You'll be quite pleased at how fast it is.