Export tiles from ESRI service to tile package

9569
10
06-08-2015 04:12 PM
MichaelRich
New Contributor III

Hi,

I want to create a tile package .tpk from this ESRI map service.  How can I do this?

My main goal is to create one tile package file that I can use to side load multiple mobile devices that will be using Collector app.  I realize I can setup each device to individually download the basemap, but this does not seem efficient, especially when the tpk is 5Gb+ and I have over 5 devices.

Also, I would like to try to modify the derived tpk to constrain tiles to a defined boundary.  My thought here was, use the Tile Cache toolset to import the tpk to a cache dataset, manage the cache and constrain, then export the cache dataset back to tpk.  Does this sound right also?

most important to me is figuring out how to download a .tpk from the ESRI basemaps without having to browse to the REST endpoint and figure out how to input all the correct parameters.  It would be nice if ArcMap/arctoolbox had some good tools for doing just this.

edited June 12

CALLING ALL Devs and AGOL Admin Tool developers !!    what about an admin tool for this concept?  or in arcGIS Server Manager?  still need to know answer to "How do I do this?".  Am I stuck with just REST endpoint?  Also, I thought I saw at UC years ago (like 3) ESRI R&D demoing this functionality in an early dev build of Portal.  It had the ability to create "Mobile Applications" just like we can currently create Web Mapping Applications in AGOL now.  The Mobile Application became a content item which could then be used to provision many devices within your organization.  if that exists somewhere or if one of you devs could PLEASE!​ build something, I'd be forever grateful.

thanks, Mike

10 Replies
MichaelRich
New Contributor III

UPDATE:

After buchering example here:

http://server.arcgis.com/en/portal/latest/administer/windows/example-prepare-esri-basemaps-for-use-i...

I've got this python code put together.  After studying up on the API, it appears I need to generate both a portal token and server token, which I got figured out.  However, code still gives me Invalid Token on the exportTiles method.  What am I doing wrong?

thanks, mike

# Requires Python 2.7+
#
# ! NOT COMPATIBLE WITH PYTHON 3+!
# taken from ESRI python sample for ARcGIS Server REST api
#
import urllib
import json
import sys


def generatePortalToken(username, password, portalUrl, refererUrl):
    '''Retrieves a token to be used with API requests.'''
    parameters = urllib.urlencode({'username' : username,
                                   'password' : password,
                                   'client' : 'referer',
                                   'referer': refererUrl,
                                   'expiration': 5,
                                   'f' : 'json'})

    response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?',
                              parameters).read()
    try:
        jsonResponse = json.loads(response)
        if 'token' in jsonResponse:
            return jsonResponse['token']
        elif 'error' in jsonResponse:
            print jsonResponse['error']['message']
            for detail in jsonResponse['error']['details']:
                print detail
                
    
    except:
        print 'An unspecified error occurred.'


def generateServerToken(portalUrl, token, serverURL):
    '''Retrieves a token to be used with API requests.'''
    parameters = urllib.urlencode({'token': token,
                                   'serverURL': serverURL,
                                   'f' : 'json'})
    response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?',
                              parameters).read()
    try:
        jsonResponse = json.loads(response)
        if 'token' in jsonResponse:
            return jsonResponse['token']
        elif 'error' in jsonResponse:
            print jsonResponse['error']['message']
            for detail in jsonResponse['error']['details']:
                print detail
                
    
    except:
        print 'An unspecified error occurred.'
        
def submitExportTilesJob(serverToken, mapService):
    '''download TPK'''
    ''' ** TEST ** '''
    parameters = urllib.urlencode({'token' : serverToken,
                                   'f' : 'json',
                                   'tilePackage' : 'true',
                                   'exportBy' : 'LevelID',
                                   'levels' : '10-13',
                                   'exportExtent' : '-13578942,5406262,-13510153,5490705'})


    
    response = urllib.urlopen(service + '/exportTiles?', parameters).read()
    return json.loads(response)




# Run the script.
if __name__ == '__main__':


    portal='https://www.arcgis.com'
    agoAcct='***'
    agoPassword='***'
    service='http://tiledbasemaps.arcgis.com/arcgis/rest/services/USA_Topo_Maps/MapServer'
    outFolder='C:/Temp'
    federatedServer='http://tiledbasemaps.arcgis.com/arcgis'
    
##    portal = sys.argv[1]
##    agoAcct = sys.argv[2]
##    agoPassword = sys.argv[3]
##    service = sys.argv[4]
##    outFolder = sys.argv[5]
    


    # Get a token for the Portal for ArcGIS.
    print 'Getting PORTAL token for ' + portal
    pToken = generatePortalToken(username=agoAcct, password=agoPassword,
                          portalUrl=portal,refererUrl=service)
    #print 'PORTAL TOKEN = ' + pToken
    print 'Getting SERVER token for ' + federatedServer
    sToken = generateServerToken(portalUrl=portal, token=pToken, serverURL=federatedServer)
    #print 'SERVER TOKEN = ' + sToken
    
    try:
        print 'attempt tile export'
        result = submitExportTilesJob(serverToken=sToken, mapService=service)
        print result
        if 'jobId' in result:
            print 'Successful request'
            print result['jobId']
        elif 'error' in result:
                    
            print 'Error, did not work'
            print result['error']['message']
            for detail in result['error']['details']:
                print detail
        else:
            print 'Error, bad'
            print 'An unhandled error occurred.'
    except:
            print 'Error even badder..'
            print 'An unhandled exception occurred HERE.'


    

  
0 Kudos
JonathanQuinn
Esri Notable Contributor

It looks like you generate a Portal token using the http referer approach, which I believe is the only way to generate a token.  What that means, though, is that every time you use the Portal token, the only way it's going to be valid is if you include a referer header equal to the referer parameter when you generated the token in every request where the token is used.  That's the only way the token is valid.  If you're not including the referer header in the request using the Portal token to generate the Server token, then maybe the Server token is not being generated correctly.

MichaelRich
New Contributor III

Hi Jonathon,

I changed and added a line for the 'referer' parameter in my Generate ServerToken block.  it still gives me "invallid Token" error when i run the script and it goes into submitExportTilesJob block.  thanks for your help

def generateServerToken(portalUrl, token, serverURL):
    '''Retrieves a token to be used with API requests.'''
    parameters = urllib.urlencode({'request': 'getToken',
                                   'serverURL': serverURL,
                                   'token': token,                                   
                                   'referer': 'www.arcgis.com',
                                   'f' : 'json'})
    response = urllib.urlopen(portalUrl + '/sharing/generateToken?',
                              parameters).read()
0 Kudos
JonathanQuinn
Esri Notable Contributor

If you use the "referer" option when generating the token, you need to pass the "referer" header within any request that uses the token, otherwise the token will be invalid.  You can use the urllib2 module to add the referer header and set it to whatever URL you used for the referer parameter when creating the token.

DougBrowning
MVP Esteemed Contributor

10.1 will let you export Esri layers to a tpk.  It was taken out after and was supposed to be back for 10.3.1 but was not.

0 Kudos
RenatoSalvaleon
Occasional Contributor III

Oh yes, ArcGIS Mobile and Mobile server 10.0.  I wonder what the future is for ArcGIS Mobile?

0 Kudos
DougBrowning
MVP Esteemed Contributor

I meant ArcMap 10.1

0 Kudos
simoxu
by MVP Regular Contributor
MVP Regular Contributor

Any update on this? it will be good if ESRI can provide a tool to create tpks from its online basemaps.

DougBrowning
MVP Esteemed Contributor

ArcPro 1.2 is allowing it for me.  Some others told me about it but they have since lost their access all of a sudden.  Mine is still working for now.

0 Kudos