AnsweredAssumed Answered

Overwrite Hosted Feature Layer Overflow error

Question asked by Koelker12 on Mar 3, 2020
Latest reply on Mar 20, 2020 by arexroad

Hey all,

 

I'm trying to script out an overwrite workflow using the method found here:

 

*Relevant code from above link (Overwrite Feature Layer section near the bottom):

from arcgis.features import FeatureLayerCollectioncities_flayer_collection = FeatureLayerCollection.fromitem(cities_item)
#call the overwrite() method which can be accessed using the manager propertycities_flayer_collection.manager.overwrite(os.path.join('data', 'updating_gis_content',                               'updated_capitals_csv', 'capitals_1.csv'))

 

I've used this in the past for other scripts and it works swell. However, I now have a need to create a single "master" feature layer comprised of multiple datasets that have been merged together, and it comes out as a pretty big file size when zipped together in a fgdb for upload (2.1 GB). When I manually execute the Overwrite using the menu buttons on the item detail page in ArcGIS Online, it uploads and overwrites the feature layer from my zipped fgdb with no problem.

 

When I try to replicate this via the scripting method above, I get the following error:

 

Traceback (most recent call last):
  File "C:/Users/user_user/.PyCharmCE2019.3/config/scratches/DRP_Update_SNAPP_Master.py", line 146, in <module>
    flayer.manager.overwrite(newdata)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\managers.py", line 1268, in overwrite
    if related_data_item.update(item_properties=params, data=data_file):
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 8165, in update
    large_thumbnail)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\portalpy.py", line 2228, in update_item
    resp = self.con.post(path, postdata, files)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py", line 1106, in post
    resp = opener.open(req)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 526, in open
    response = self._open(req, data)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 544, in _open
    '_open', req)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\urllib\request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 1075, in _send_output
    self.send(chunk)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\http\client.py", line 996, in send
    self.sock.sendall(data)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py", line 975, in sendall
    v = self.send(byte_view[count:])
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py", line 944, in send
    return self._sslobj.write(data)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\ssl.py", line 642, in write
    return self._sslobj.write(data)
OverflowError: string longer than 2147483647 bytes

 

I haven't found any sort of note within the API reference stating that the overwrite uploads are limited to 2GB -- and it works when I do it manually -- so I'm confused what I'm running into here, and unsure how to work around it. Is it a limitation of the urllib or ssl stuff that the .overwrite function uses? 

Outcomes