topic Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script in Python Questions

Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

GeoffreyWest — Sat, 11 Dec 2021 20:47:43 GMT

I have a script which creates a vector tile package. I am receiving the error: ERROR 001856: Cached scale doesn't match tiling scheme only when I run the tool from a standalone Python script. I use the same parameters in the GUI of ArcGIS Pro and the tool works fine. Is this an ArcPy bug? If I don't use a min and max scaling the tool runs successfully but takes forever due to the size of the dataset.

Here is my script:

from arcgis.gis import GIS
import os
import arcpy



#set environment settings
gis = GIS("myportal")
arcpy.env.overwriteOutput = True
outputPath = r"\\myOutPath\"
arcpy.env.parallelProcessingFactor = "66%"


# Loop through the project, find all the maps, and
#   create a vector tile package for each map,
#   using the same name as the map
p = arcpy.mp.ArcGISProject(r"\\CPVektrTile.aprx")
for m in p.listMaps():
    print("Indexing " + m.name)
    arcpy.CreateVectorTileIndex_management(m, outputPath + m.name + '.shp', "ONLINE", "", 10000)
    print("Packaging " + m.name)
    arcpy.management.CreateVectorTilePackage(m, outputPath + m.name + '.vtpk', "ONLINE", "", "INDEXED", 100000, 3000, r"\\CPAreas.shp")

print ("Adding VTPK to Test Portal")
vtpk_item = gis.content.add({}, data=outputPath + m.name + '.vtpk', folder='packages', overwrite=True)

print ("Publishing & Overwriting VTPK to Test Portal")
vtpk_layer = vtpk_item.publish(overwrite=True)
vtpk_layer.share(groups="SCG - Cathodic Protection")

 ‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Screenshot of successful run from geoprocessing tool in ArcGIS Pro:

Code sent to Python Window from history:

arcpy.management.CreateVectorTilePackage("CPAreas", r"C:\db\vtpktest_2.vtpk", "ONLINE", None, "INDEXED", 100000, 3000, r"\\\CPAreas.shp", None, None)
‍

Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

Anonymous User — Tue, 04 Jun 2019 14:36:35 GMT

Hey Geoffrey,

Were you ever able to figure this out? I am having this issue as well. I would like to be able to set the min and max scales in Arcpy due to the size of my dataset, but am seeing this same error. I have a large number of layers and would really like to script it. Thanks!

Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

GeoffreyWest — Tue, 04 Jun 2019 14:54:27 GMT

Hi Ethan,

I don't remember what my solution was with this, a colleague has since

taken over the project. I will take a look at her code and get back to you.

Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

Anonymous User — Tue, 04 Jun 2019 20:24:32 GMT

Thanks, no worries if you cannot find it!

Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

Anonymous User — Wed, 05 Jun 2019 20:28:34 GMT

That makes sense, and is a good idea. Thanks for this!

Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

TraceStanford1 — Wed, 04 Sep 2019 22:45:06 GMT

Did you get this script to work? I've tried, but keep getting the 'Please provide parameter for config file' message. Where do you plug in your project?

Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

DanDeegan — Wed, 16 Oct 2019 18:32:00 GMT

We don't use vector tiles very often in web maps because they won't show up in legends. I do use vector tiles for contour lines and some other things with tons of geometry. Which don't need to be updated, so I don't need to automate it.

the config part is something I am trying to use now. You set parameters in a text file and get them out. So you need to create one. I am guess this one uses json.

Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

GeoffreyWest — Sat, 11 Dec 2021 20:47:46 GMT

Hi Ethan,

Here is the refactored code. It looks like the script gets the map properties. Hope it helps.

import os
import logging
import logging.handlers
import time
import configparser
import SendEmail
from arcgis.gis import GIS
import arcpy
import sys
def getMapProperties(mapPropsList):
 mapProperties = {}
 for i in range(0, len(mapPropsList)):
 props = mapPropsList[i].split(",")
 mapname = props[0]
 mapnameupper = mapname.upper()
 mapProperties[mapnameupper]={}
 mapProperties[mapnameupper]['mapname'] = props[0]
 if len(props) > 1:
 mapProperties[mapnameupper]['minscale'] = props[1]
 else:
 mapProperties[mapnameupper]['minscale'] =""
 if len(props) > 2:
 mapProperties[mapnameupper]['maxscale'] = props[2]
 else:
 mapProperties[mapnameupper]['maxscale'] =""
 if len(props) > 3:
 mapProperties[mapnameupper]['vtpkindex'] = props[3]
 else:
 mapProperties[mapnameupper]['vtpkindex'] = ""
 return mapProperties
def createVtpk(m, mapProperties, indexGDBPath, vtpkPath, logger):
 mapName = m.name
minscale = ""
 maxscale =""
 vtpkindexfile = ""
 mapNameUpper = mapName.upper()
 if mapNameUpper in mapProperties:
 minscale = mapProperties[mapNameUpper]['minscale']
 if minscale !="":
 minscale = float(minscale)
 maxscale = mapProperties[mapNameUpper]['maxscale']
 if maxscale !="":
 maxscale = float(maxscale)
vtpkindex = mapProperties[mapNameUpper]['vtpkindex'] 
 if vtpkindex != "":
 vtpkindexfile = os.path.join(indexGDBPath, vtpkindex)
 
 logger.info("Creating Vector Tiles for " + mapName)
 arcpy.management.CreateVectorTilePackage(m, vtpkPath, "ONLINE", "", "INDEXED", minscale, maxscale, vtpkindexfile)
return vtpkPath
 return ""
def uploadVtpk(gis, vtpkName, vtpk, logger):
 
 vtpkFile = gis.content.search(query="title:"+vtpkName, item_type="Vector Tile Package")
 for i in range(0, len(vtpkFile)):
 item = vtpkFile[i]
 if item.title.upper() == vtpkName.upper():
 logger.info("Deleting Old VTPK Package From Portal")
 item.delete()
 break
 
 logger.info("Uploading VTPK package")
 vtpk_item = gis.content.add({}, data=vtpk)
if vtpk_item == None:
 logger.error("Error adding " + vtpk + " to Portal")
 
 return vtpk_item
def publishVtpk(gis, vtpkName, vtpk_item, portalGroups, logger):
 vtpkLayer = gis.content.search(query=vtpkName, item_type="Vector Tile Service")
 for i in range(0, len(vtpkLayer)):
 vLayer = vtpkLayer[i]
 if vLayer.title.upper() == vtpkName.upper():
 logger.info("Deleting Old VTPK Service From Portal")
 vLayer.delete()
 break
logger.info("Publishing VTPK service")
 vtpk_service = vtpk_item.publish()
 vtpk_item.delete()
logger.info("Sharing to Group in Portal")
 for group in portalGroups:
 cp_group = gis.groups.search('title:' + group, max_groups=15)
 for i in range(0, len(cp_group)):
 cpgroup_share = cp_group[i]
 if cpgroup_share.title.upper() == group.upper():
 vtpk_service.share(groups=cpgroup_share.id) 
 break;
 
 return vtpk_service
def SendErrorNotification(emailTo, emailFrom, msg):
 for email in emailTo.split(";"):
 SendEmail.email_error_msg(emailTo, emailFrom, msg)
def main():
 if len(sys.argv) == 1:
 print("Please provide parameter for config file")
 sys.exit()
if not os.path.isfile(sys.argv[1]):
 print("Config file does not exist")
 sys.exit()
 
 cfile = configparser.ConfigParser()
 cfile.read(sys.argv[1])
 portalUrl = cfile.get('Portal', 'PortalUrl')
 portalGroupsList = cfile.get('Portal', 'PortalGroup')
 portalGroups = portalGroupsList.split(";")
 workspace = cfile.get('Folder', 'Workspace')
 outputPath = cfile.get('Folder', 'OutputFolder')
 logFile = cfile.get('Folder', 'LogFile')
 emailTo = cfile.get('Notification', 'EmailTo')
 emailFrom = cfile.get('Notification', 'EmailFrom')
 vtpkIndexFolder = cfile.get('VTPK', 'VtpkIndexFolder')
 project = cfile.get('VTPK', 'ProProject')
 mapProps = cfile.get('VTPK','MapProperties')
 mapProperties = getMapProperties(mapProps.split(';'))
logger = logging.getLogger('vectortiles_oca')
 logger.setLevel(logging.INFO)
 formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
 handler = logging.handlers.RotatingFileHandler(logFile, maxBytes=100000, backupCount=5)
 handler.setFormatter(formatter)
 logger.addHandler(handler)
gis = GIS(portalUrl, verify_cert=False)
 
 arcpy.env.workspace = workspace
 arcpy.env.overwriteOutput = True
 arcpy.env.parallelProcessingFactor = "66%"
logger.info("XXXXXXXXXXXXXXXXXXXXXXXX=Start")
 p = arcpy.mp.ArcGISProject(project)
 maps = p.listMaps("*")
 for i in range(0, len(maps)):
 try:
 if maps[i].name.upper() in mapProperties:
 vtpk = os.path.join(outputPath, maps[i].name + '.vtpk') 
 if os.path.exists(vtpk):
 backupPath = os.path.join(outputPath, "backup")
 newname = os.path.join(backupPath, maps[i].name + '_'+ datetime.datetime.today().strftime('%Y%m%d%H%M') + '.vtpk')
 logger.info("Renaming existing VTPK to " + newname)
 os.rename(vtpk, newname)
 tryagain = True
 count = 0
 while tryagain:
 count = count + 1
 if count == 5:
 break
 try:
 vtpkPath = createVtpk(maps[i], mapProperties, vtpkIndexFolder, vtpk, logger)
 vtpk_item = uploadVtpk(gis, maps[i].name, vtpk, logger)
 publishVtpk(gis, maps[i].name, vtpk_item, portalGroups,logger)
 tryagain = False
 except Exception as e:
 logger.error(e, exc_info=True)
 if count ==3: 
 SendErrorNotification(emailTo, emailFrom, "Encountered an error - " + maps[i].name + ". trying again." + str(e))
 except Exception as e:
 logger.error(e, exc_info=True)
 SendErrorNotification(emailTo, emailFrom, "Vector Tiles Script has failed for map " + maps[i].name + ": " + str(e)) 
 del gis
 logger.info("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=Complete")
if __name__ == '__main__':
 main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Create Vector Tile Package Different Results in ArcGIS Pro and Standalone Python Script

DanDeegan — Sat, 11 Dec 2021 20:47:49 GMT

Thanks for following up with on your original comment. I am going to be attempting to do something similar and spent some time formatting the code as I tried to follow it.

Here it is. I think I got the indentations right...

if anyone notices errors let us know.

I assume there is a way to update the vtpk source in portal.

# -*- coding: utf-8 -*-
"""
Created on Wed Jun 26 13:10:43 2019

******not my code*****
source https://community.esri.com/thread/210061-create-vector-tile-package-different-results-in-arcgis-pro-and-standalone-python-script

"""

import os
import logging
import logging.handlers
import datetime  # used to be time
import configparser
import SendEmail
from arcgis.gis import GIS
import arcpy
import sys


def getMapProperties(mapPropsList):
    mapProperties = {}
    for i in range(0, len(mapPropsList)):
        props = mapPropsList[i].split(",")
        mapname = props[0]
        mapnameupper = mapname.upper()
        mapProperties[mapnameupper] = {}
        mapProperties[mapnameupper]['mapname'] = props[0]
        if len(props) > 1:
            mapProperties[mapnameupper]['minscale'] = props[1]
        else:
            mapProperties[mapnameupper]['minscale'] = ""
        if len(props) > 2:
            mapProperties[mapnameupper]['maxscale'] = props[2]
        else:
            mapProperties[mapnameupper]['maxscale'] = ""
        if len(props) > 3:
            mapProperties[mapnameupper]['vtpkindex'] = props[3]
        else:
            mapProperties[mapnameupper]['vtpkindex'] = ""
    return mapProperties


def createVtpk(m, mapProperties, indexGDBPath, vtpkPath, logger):
    mapName = m.name
    minscale = ""
    maxscale = ""
    vtpkindexfile = ""
    mapNameUpper = mapName.upper()
    if mapNameUpper in mapProperties:
        minscale = mapProperties[mapNameUpper]['minscale']
    if minscale != "":
        minscale = float(minscale)
        maxscale = mapProperties[mapNameUpper]['maxscale']
    if maxscale != "":
        maxscale = float(maxscale)
        vtpkindex = mapProperties[mapNameUpper]['vtpkindex']
    if vtpkindex != "":
        vtpkindexfile = os.path.join(indexGDBPath, vtpkindex)

    logger.info("Creating Vector Tiles for " + mapName)
    arcpy.management.CreateVectorTilePackage(m, vtpkPath, "ONLINE", "", "INDEXED",
                                             minscale, maxscale, vtpkindexfile)
    return vtpkPath
    return ""


def uploadVtpk(gis, vtpkName, vtpk, logger):
    vtpkFile = gis.content.search(query="title:"+vtpkName,
                                  item_type="Vector Tile Package")
    for i in range(0, len(vtpkFile)):
        item = vtpkFile[i]
        if item.title.upper() == vtpkName.upper():
            logger.info("Deleting Old VTPK Package From Portal")
            item.delete()
        break

        logger.info("Uploading VTPK package")
        vtpk_item = gis.content.add({}, data=vtpk)
        if vtpk_item is None:
            logger.error("Error adding " + vtpk + " to Portal")

    return vtpk_item


def publishVtpk(gis, vtpkName, vtpk_item, portalGroups, logger):
    vtpkLayer = gis.content.search(query=vtpkName, item_type="Vector Tile Service")
    for i in range(0, len(vtpkLayer)):
        vLayer = vtpkLayer[i]
        if vLayer.title.upper() == vtpkName.upper():
            logger.info("Deleting Old VTPK Service From Portal")
            vLayer.delete()
        break
        logger.info("Publishing VTPK service")
        vtpk_service = vtpk_item.publish()
        vtpk_item.delete()
        logger.info("Sharing to Group in Portal")
    for group in portalGroups:
        cp_group = gis.groups.search('title:' + group, max_groups=15)
    for i in range(0, len(cp_group)):
        cpgroup_share = cp_group[i]
        if cpgroup_share.title.upper() == group.upper():
            vtpk_service.share(groups=cpgroup_share.id)
        break  # used to be a ; here
    return vtpk_service


def SendErrorNotification(emailTo, emailFrom, msg):
    for email in emailTo.split(";"):
        SendEmail.email_error_msg(emailTo, emailFrom, msg)


def main():
    if len(sys.argv) == 1:
        print("Please provide parameter for config file")
        sys.exit()
    if not os.path.isfile(sys.argv[1]):
            print("Config file does not exist")
    sys.exit()

    cfile = configparser.ConfigParser()
    cfile.read(sys.argv[1])
    portalUrl = cfile.get('Portal', 'PortalUrl')
    portalGroupsList = cfile.get('Portal', 'PortalGroup')
    portalGroups = portalGroupsList.split(";")
    workspace = cfile.get('Folder', 'Workspace')
    outputPath = cfile.get('Folder', 'OutputFolder')
    logFile = cfile.get('Folder', 'LogFile')
    emailTo = cfile.get('Notification', 'EmailTo')
    emailFrom = cfile.get('Notification', 'EmailFrom')
    vtpkIndexFolder = cfile.get('VTPK', 'VtpkIndexFolder')
    project = cfile.get('VTPK', 'ProProject')
    mapProps = cfile.get('VTPK', 'MapProperties')
    mapProperties = getMapProperties(mapProps.split(';'))
    logger = logging.getLogger('vectortiles_oca')
    logger.setLevel(logging.INFO)
    formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
    handler = logging.handlers.RotatingFileHandler(logFile,
                                                   maxBytes=100000,
                                                   backupCount=5)
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    gis = GIS(portalUrl, verify_cert=False)

    arcpy.env.workspace = workspace
    arcpy.env.overwriteOutput = True
    arcpy.env.parallelProcessingFactor = "66%"
    logger.info("XXXXXXXXXXXXXXXXXXXXXXXX=Start")
    p = arcpy.mp.ArcGISProject(project)
    maps = p.listMaps("*")

    for i in range(0, len(maps)):
        try:
            if maps[i].name.upper() in mapProperties:
                vtpk = os.path.join(outputPath, maps[i].name + '.vtpk')
            if os.path.exists(vtpk):
                backupPath = os.path.join(outputPath, "backup")
                newname = os.path.join(backupPath, maps[i].name + '_' +
                                       datetime.datetime.today().strftime('%Y%m%d%H%M')
                                       + '.vtpk')
                logger.info("Renaming existing VTPK to " + newname)
                os.rename(vtpk, newname)
                tryagain = True
                count = 0
            while tryagain:
                count = count + 1
                if count == 5:
                    break
                try:
                    vtpkPath = createVtpk(maps[i], mapProperties, vtpkIndexFolder,
                                          vtpk, logger)
                    vtpk_item = uploadVtpk(gis, maps[i].name, vtpk, logger)
                    publishVtpk(gis, maps[i].name, vtpk_item, portalGroups, logger)
                    tryagain = False
                except Exception as e:
                    logger.error(e, exc_info=True)
                    if count == 3:
                        SendErrorNotification(emailTo, emailFrom,
                                              "Encountered an error - "
                                              + maps[i].name
                                              + ". trying again." + str(e))
        except Exception as e:
            logger.error(e, exc_info=True)
            SendErrorNotification(emailTo, emailFrom,
                                  "Vector Tiles Script has failed for map " +
                                  maps[i].name + ": " + str(e))
    del gis

    logger.info("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=Complete")


if __name__ == '__main__':
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍