KeyError: 'attachmentGroups' when cloning forms and associated hosted feature services

[JamesGustine]

Pretty simple cloning operation here. Just want to test cloning Survey123 forms and there associated feature service

from arcgis import GIS
from arcgis.mapping import WebMap 
import arcpy
from arcpy import env


#set portal and define resources

source = GIS('https://source.southernute.com/portal', "admin", "password")
target = GIS('https://target.southernute.com/portal', "admin", "password")

print (source)
print (target)

##get source info
#sourceItem = source.content.search('owner:arcgisportaladmin',item_type = "Form", max_items=500)

sourceItem = source.content.get('f0979f620b30416db90bf6d0de5d2866')

print (sourceItem)

target.content.create_folder(("Survey-"+sourceItem.title).rstrip())

f = ("Survey-"+sourceItem.title).rstrip()

print ('cloning...')
clone = target.content.clone_items([sourceItem], folder=f, copy_data=True)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Keep getting this error? It happens on forms with and without attachments? Any ideas?

Traceback (most recent call last):
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py", line 1629, in clone
 self._add_features(new_layers, relationships, layer_field_mapping, feature_service.properties['spatialReference'])
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py", line 1130, in _add_features
 attachments = original_attachment_manager.search()
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\managers.py", line 153, in search
 for result in results['attachmentGroups']:
KeyError: 'attachmentGroups'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "D:\source\code\admin\clonetest.py", line 32, in <module>
 clone = target.content.clone_items([sourceItem], folder=f, copy_data=True)
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 3121, in clone_items
 return deep_cloner.clone()
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py", line 525, in clone
 results = loop.run_until_complete(self._clone(executor))
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\asyncio\base_events.py", line 467, in run_until_complete
 return future.result()
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py", line 510, in _clone
 raise result
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\concurrent\futures\thread.py", line 55, in run
 result = self.fn(*self.args, **self.kwargs)
 File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py", line 1638, in clone
 raise _ItemCreateException("Failed to create {0} {1}: {2}".format(original_item['type'], original_item['title'], str(ex)), new_item)
arcgis._impl.common._clone._ItemCreateException: ("Failed to create Feature Service SUU - Confined Space Reclassification Certificate\n: 'attachmentGroups'", <Item title:"SUU - Confined Space Reclassification Certificate
" type:Feature Layer Collection owner:arcgisportaladmin>)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[ChrisFox]

Hi James,

As a troubleshooting step if you set copy_data=False during the clone does it work successfully? I am just curious if it is failing during the creation of the feature service or if the problem occurs when attempting to copy the data between the services.

-Chris

[StephenPatterson1]

I am running some test code that copies content from ArcGIS Online to a new Portal account using the ContentManager.clone_items() method like above.  

It fails for me as well when trying to copy data.  If I set copy_data=False, it copies the definitions fine without the data. 

Here is the error I am receiving when using clone_items():

_ItemCreateException: ('Failed to create Feature Service AJ1770141: Expecting value: line 1 column 1 (char 0)'

Not sure if these issues are the same, but I didn't find much else for this type of error.

[ChrisFox]

Hi Stephen,

What version of the api do you have installed? We fixed some things that may have caused this problem in 1.5 that was just released.

-Chris

[StephenPatterson1]

I upgraded the Python API to 1.5 and it is still giving me _ItemCreateException errors.  Below is the latest error message.  It happens when trying to copy a hosted FeatureLayer from ArcGIS Online to a Portal account.  I also noticed that the char number in the exception continues to go up every time I run my script and it fails. 

I saw on this post(https://community.esri.com/message/775171-error-when-cloning-items-from-one-portal-to-another ) that they noticed FeatureLayer types failing to clone.  I am looping through my personal content, which has many different item types, and trying to pass each item one at a time to the clone_items() method.  It copies a Service Definition fine, but fails on the second item, which is a FeatureLayer. 

Copy_Content_to_Portal

JSONDecodeError                           Traceback (most recent call last)C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py in clone(self)   1697                 if self.copy_data and not self.is_view:-> 1698                     self._add_features(new_layers, relationships, layer_field_mapping, feature_service.properties['spatialReference'])   1699  C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py in _add_features(self, layers, relationships, layer_field_mapping, spatial_reference)   1087             for layer in original_layers:-> 1088                 features[str(layer.properties['id'])] = self._get_features(layer, spatial_reference)   1089         else:C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py in _get_features(self, feature_layer, spatial_reference)   1066         while offset < record_count:-> 1067             features = feature_layer.query(out_sr=spatial_reference, result_offset=offset, result_record_count=max_record_count, return_z=return_z, return_m=return_m).features   1068             offset += len(features)C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\layer.py in query(self, where, out_fields, time_filter, geometry_filter, return_geometry, return_count_only, return_ids_only, return_distinct_values, return_extent_only, group_by_fields_for_statistics, statistic_filter, result_offset, result_record_count, object_ids, distance, units, max_allowable_offset, out_sr, geometry_precision, gdb_version, order_by_fields, out_statistics, return_z, return_m, multipatch_option, quantization_parameters, return_centroid, return_all_records, result_type, historic_moment, sql_format, return_true_curves, return_exceeded_limit_features, **kwargs)    576         if record_count <= max_records:--> 577             return self._query(url, params)    578  C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\features\layer.py in _query(self, url, params)   1195         result = self._con.post(path=url,-> 1196                                 postdata=params, token=self._token)   1197         if 'error' in result:C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\connection.py in post(self, path, postdata, files, ssl, compress, is_retry, use_ordered_dict, add_token, verify_cert, token, try_json, out_folder, file_name, force_bytes, add_headers)   1121         else:-> 1122             resp_json = json.loads(resp_data)   1123  C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)    353             parse_constant is None and object_pairs_hook is None and not kw):--> 354         return _default_decoder.decode(s)    355     if cls is None:C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\decoder.py in decode(self, s, _w)    338         """--> 339         obj, end = self.raw_decode(s, idx=_w(s, 0).end())    340         end = _w(s, end).end()C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\decoder.py in raw_decode(self, s, idx)    354         try:--> 355             obj, end = self.scan_once(s, idx)    356         except StopIteration as err:JSONDecodeError: Expecting ',' delimiter: line 1 column 44819198 (char 44819197)  During handling of the above exception, another exception occurred: _ItemCreateException                      Traceback (most recent call last)<ipython-input-9-154c6fc082ea> in <module>()     12     start = arrow.utcnow()     13     print(start)---> 14     portal_content.clone_items([real_item]) #, copy_data=False)     15     print(arrow.utcnow() - start)     16 print('finished')C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py in clone_items(self, items, folder, item_extent, use_org_basemap, copy_data, search_existing_items, item_mapping, group_mapping)   3286             wgs84_extent = clone._wgs84_envelope(service_extent)   3287         deep_cloner = clone._DeepCloner(self._gis, items, folder, wgs84_extent, service_extent, use_org_basemap, copy_data, search_existing_items, item_mapping, group_mapping)-> 3288         return deep_cloner.clone()   3289    3290     def bulk_update(self, itemids, properties):C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py in clone(self)    570             asyncio.set_event_loop(asyncio.new_event_loop())    571             loop = asyncio.get_event_loop()--> 572             results = loop.run_until_complete(self._clone(executor))    573             loop.close()    574             return results C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\asyncio\base_events.py in run_until_complete(self, future)    466             raise RuntimeError('Event loop stopped before Future completed.')    467 --> 468         return future.result()    469     470     def stop(self):C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py in _clone(self, excecutor)    559                         if item:    560                             item.delete()--> 561                     raise result    562     563             level += 1C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\concurrent\futures\thread.py in run(self)     54      55         try:---> 56             result = self.fn(*self.args, **self.kwargs)     57         except BaseException as exc:     58             self.future.set_exception(exc)C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\_impl\common\_clone.py in clone(self)   1705             return new_item   1706         except Exception as ex:-> 1707             raise _ItemCreateException("Failed to create {0} {1}: {2}".format(original_item['type'], original_item['title'], str(ex)), new_item)   1708    1709  _ItemCreateException: ("Failed to create Feature Service AJ1770141: Expecting ',' delimiter: line 1 column 44819198 (char 44819197)", <Item title:"AJ1770141" type:Feature Layer Collection owner:slpatterson>)

[ChrisFox]

Hi Stephen,

Thanks for testing the 'attachmentGroups' error from the original post was the error I remembered fixing in 1.5. This new error I haven't seen before and is also occurring when attempting to copy the data from the original service to the new service. I am wondering if the service is public or if you could invite my user (MapFox) to a group in your online organization so that I could debug the issue by attempting to clone to a Portal on my side.

-Chris