arcpy.FeatureSet() From Feature Service

[JamesCrandall]

I'm not having luck with the GlobalID and ParentGlobalID columns when attempting to populate Feature Classes and Tables in a FGDB from a hosted feature service.  It's all fairly straight forward and seems to work just fine, except for when specifying these two fields.

Issues:

1. The GlobalId field does not seem to throw any error, however the feature classes get populated with the value: {00000000-0000-0000-0000-000000000000}

2. There's also related feature classes that contain a ParentGlobalID column as well, these throw an error when attempting to load the feature set "RuntimeError: RecordSetObject: Cannot open table for Load".  If I remove the "ParentGlobalID" column from my list of fields to download, then the process works fine without error (minus the ParentGlobalID column of course).

This is the important section:

fields = """objectid,globalid,SurveyStartTime,SurveyEndTime,CreationDate,Creator,EditDate,Editor"""
taburl =  <URLtoHostedFeatureService>/query/0
where = '1=1'
query = "?where={}&outFields={}&returnGeometry=true&f=json&token={}".format(where, '*', token)
queryfsURL = taburl + query
#try to convert the hosted service to a fgdb table
fs = arcpy.FeatureSet()
fs.load(queryfsURL)
fs.save(r'<pathToDirectory>\downloadTab.gdb\HostedFsNameHere)
#also attempted this
#arcpy.CopyRows_management(fs, r'<pathToDirectory>\downloadTab.gdb\HostedFsNameHere)‍‍‍‍‍‍‍‍‍‍‍

[JamesCrandall]

Another test attempted: compare the toolbox "JSONToFeatures" tool with python scripted "arcpy.JSONToFeatures_conversion"

Using the exact same .json file in both instances, the toolbox tool's output contains the expected GlobalID values, the arcpy method does not (again, populated with {00000000-0000-0000-0000-000000000000}).

.json file source:

{"features": [{"geometry": {"y": 3302628.348359901, "x": -9080214.856802708}, "attributes": {"GlobalID": "3d7fd4db-8fd4-4150-a6fa-7c8d9f490028", "OBJECTID": 1}}, {"geometry": {"y": 2935905.840858899, "x": -8949481.954276735}, "attributes": {"GlobalID": "17b18396-7d70-4f7f-987b-dd3c869e3d13", "OBJECTID": 2}}, {"geometry": {"y": 3182365.217025425, "x": -8958169.568138303}, "attributes": {"GlobalID": "fd287aa8-c666-4ebb-973e-af42c9210555", "OBJECTID": 3}}, {"geometry": {"y": 3116365.997722158, "x": -8944001.624695944}, "attributes": {"GlobalID": "0ef2bc1a-7ab3-4eac-936d-53c59c6ad720", "OBJECTID": 4}}, {"geometry": {"y": 3266689.3323510713, "x": -9056080.682907999}, "attributes": {"GlobalID": "c17153b0-72c6-4a71-8c9c-d08fbca4f971", "OBJECTID": 5}}, {"geometry": {"y": 3032950.2018160364, "x": -9103563.008196728}, "attributes": {"GlobalID": "7822675c-67b9-43c2-9c6d-dd3686b7684c", "OBJECTID": 6}}, {"geometry": {"y": 2987973.085353658, "x": -8925101.388748247}, "attributes": {"GlobalID": "82eb8efe-0344-424b-aed3-40f521ca0096", "OBJECTID": 7}}, {"geometry": {"y": 3149190.0434092716, "x": -8943513.028244277}, "attributes": {"GlobalID": "0ca97eef-6b5b-4f8a-8de8-43e34b8e7352", "OBJECTID": 8}}, {"geometry": {"y": 2969635.1807482317, "x": -8980605.152716484}, "attributes": {"GlobalID": "78134e5e-1e33-46f7-943c-101453e01652", "OBJECTID": 9}}, {"geometry": {"y": 3300730.1157377195, "x": -9069763.544277236}, "attributes": {"GlobalID": "6c982a39-94f5-4249-9672-db7f04e46b1b", "OBJECTID": 10}}, {"geometry": {"y": 2916452.7305216496, "x": -8963985.064438965}, "attributes": {"GlobalID": "30680b80-3ec7-492f-85e6-95d2260e7331", "OBJECTID": 11}}, {"geometry": {"y": 3295387.3267123713, "x": -9088780.378074935}, "attributes": {"GlobalID": "d8309d5a-3844-4ba9-adb9-d31793ab26f2", "OBJECTID": 12}}, {"geometry": {"y": 3083860.600086069, "x": -9123405.091868907}, "attributes": {"GlobalID": "c04be547-1c72-47f8-894f-a9c6eef4ed32", "OBJECTID": 13}}, {"geometry": {"y": 3224239.209942657, "x": -9052199.086908078}, "attributes": {"GlobalID": "5bfc2427-4189-417e-bdad-8ff71f9d76d7", "OBJECTID": 14}}, {"geometry": {"y": 2956551.1227857363, "x": -8990878.972877821}, "attributes": {"GlobalID": "9e640c17-debc-457a-b695-a2962d9cd354", "OBJECTID": 15}}, {"geometry": {"y": 2925328.7789350976, "x": -8967704.78599132}, "attributes": {"GlobalID": "7d8c6a90-1cc3-4502-a0fd-6588c0ccca47", "OBJECTID": 16}}, {"geometry": {"y": 2999472.875453083, "x": -8921906.202857886}, "attributes": {"GlobalID": "4ce0f7f6-b5fd-4387-ade9-89cbc0a9f654", "OBJECTID": 17}}, {"geometry": {"y": 3009337.3731106035, "x": -8931162.126738088}, "attributes": {"GlobalID": "6723b6c6-9094-4316-8a9c-23a229a09ea8", "OBJECTID": 18}}, {"geometry": {"y": 3040321.612889241, "x": -9097337.725443132}, "attributes": {"GlobalID": "026ec2ad-4be0-4234-a2dd-f9070fec7270", "OBJECTID": 19}}, {"geometry": {"y": 2957907.2298183567, "x": -8939240.551882569}, "attributes": {"GlobalID": "a1718bc2-f54f-4a94-a06b-d880c861498b", "OBJECTID": 20}}, {"geometry": {"y": 3164780.4135744474, "x": -9044995.89939325}, "attributes": {"GlobalID": "a981bb05-ced4-48c4-9867-c16f21bf6f9b", "OBJECTID": 21}}, {"geometry": {"y": 3007557.9593651486, "x": -9094193.111578912}, "attributes": {"GlobalID": "c8a0bf73-9f4b-4217-821f-3c22ee35ea57", "OBJECTID": 22}}, {"geometry": {"y": 3040321.612889241, "x": -9097337.725443132}, "attributes": {"GlobalID": "881b6655-92ce-4902-b650-8ab80cf5cc0c", "OBJECTID": 23}}, {"geometry": {"y": 3055864.9450819413, "x": -9065465.833404854}, "attributes": {"GlobalID": "d34c1eb3-f244-42ae-b6a2-79fb5e35eeba", "OBJECTID": 24}}, {"geometry": {"y": 3224292.2406486287, "x": -9038813.640524628}, "attributes": {"GlobalID": "41f9a259-ef37-4d0c-a621-d50955f308c7", "OBJECTID": 25}}, {"geometry": {"y": 2927263.8650856293, "x": -8953185.156013427}, "attributes": {"GlobalID": "8a860774-c19c-4a5a-a925-c52e341d91ef", "OBJECTID": 26}}, {"geometry": {"y": 3062601.9988861633, "x": -9097594.207727797}, "attributes": {"GlobalID": "536a63d6-1f2d-4607-bcbc-ae8de0796a82", "OBJECTID": 27}}, {"geometry": {"y": 3070923.263292275, "x": -9132372.515240341}, "attributes": {"GlobalID": "df893039-0ec9-4ddc-a658-d6cda417f0ab", "OBJECTID": 28}}, {"geometry": {"y": 3041218.6997780814, "x": -9103491.233161572}, "attributes": {"GlobalID": "64aa8f72-d006-4cd0-995b-701ff6022e70", "OBJECTID": 29}}, {"geometry": {"y": 2935177.955242901, "x": -8968202.019990554}, "attributes": {"GlobalID": "5dd0a695-7da8-4901-b552-976d6e18e5b0", "OBJECTID": 30}}, {"geometry": {"y": 2917410.3796297098, "x": -8966413.374766212}, "attributes": {"GlobalID": "a83bb34d-a24d-435b-b275-fd7a5514c6a6", "OBJECTID": 31}}, {"geometry": {"y": 3119756.0254174476, "x": -8916007.179496944}, "attributes": {"GlobalID": "8fe4916a-acb9-45d2-86ac-78be46956794", "OBJECTID": 32}}, {"geometry": {"y": 3280828.470373303, "x": -9053413.521350337}, "attributes": {"GlobalID": "92f09d6b-513c-42e0-925c-044f43549f11", "OBJECTID": 33}}, {"geometry": {"y": 3062601.9988861633, "x": -9097594.207727797}, "attributes": {"GlobalID": "4bcbace2-3ec2-4133-9e2b-f3f52f9e70e8", "OBJECTID": 34}}, {"geometry": {"y": 3068641.422484618, "x": -9090565.776465254}, "attributes": {"GlobalID": "e75833c9-46d1-4be4-ad0d-b83e42810d80", "OBJECTID": 35}}, {"geometry": {"y": 3156160.1965427827, "x": -8945633.634291844}, "attributes": {"GlobalID": "029a444a-ce2c-4079-9638-9c6b26548106", "OBJECTID": 36}}, {"geometry": {"y": 2975152.3453906407, "x": -8938679.301698191}, "attributes": {"GlobalID": "4e9f8c83-12a4-4348-8cdc-b07e0ddf3465", "OBJECTID": 37}}, {"geometry": {"y": 3068261.150886741, "x": -9114097.527042406}, "attributes": {"GlobalID": "a56a9893-a6eb-4a16-a0ed-3d1b46a3c115", "OBJECTID": 38}}, {"geometry": {"y": 3047310.0172173204, "x": -8919962.125129992}, "attributes": {"GlobalID": "f44736fd-7cbb-4a61-b18e-e6ace08f2802", "OBJECTID": 39}}, {"geometry": {"y": 3030133.243087964, "x": -8916829.71315664}, "attributes": {"GlobalID": "df925e89-29c0-42e1-9f5d-53a04d1e6d6e", "OBJECTID": 40}}, {"geometry": {"y": 3038496.552766487, "x": -8918397.465804476}, "attributes": {"GlobalID": "58dc4684-2657-468d-af0e-85277746984d", "OBJECTID": 41}}], "fields": [{"domain": null, "name": "OBJECTID", "defaultValue": null, "alias": "OBJECTID", "sqlType": "sqlTypeOther", "type": "esriFieldTypeOID"}, {"domain": null, "name": "GlobalID", "defaultValue": null, "alias": "GlobalID", "length": 38, "sqlType": "sqlTypeOther", "type": "esriFieldTypeGlobalID"}], "spatialReference": {"wkid": 102100, "latestWkid": 3857}, "geometryType": "esriGeometryPoint", "objectIdFieldName": "OBJECTID", "globalIdFieldName": "GlobalID"}‍

[JamesCrandall · Accepted solution]

Moving latest testing results out as reply to OP:

1. Run with the python window in ArcGIS Desktop 10.3.1 with both Background Processing checked and unchecked generates the 0's

2. Run as .bat pointing to C:\Python27\ArcGIS10.3\python.exe generates 0's

3. Run as .bat pointing to C:\Python27\ArcGISx6410.3\python.exe generates 0's

4. Run with the python window in ArcGIS Desktop 10.4.1 with both Background Processing checked and unchecked generates the expected GlobalID values.

5. Run as .bat pointing to C:\Python27\ArcGIS10.4\python.exe generates expected GlobalId values

6. Run as .bat pointing to C:\Python27\ArcGISx6410.4\python.exe generates 0's

[JamesCrandall]

Just to post my findings: the issue I was having with GlobalID/ParentGlobalID values as {00000000-0000-0000-0000-000000000000} when implementing .load() on the FeatureSet with the service url was reproduced by ESRI support.  This appears to be a problem with ArcGIS Desktop 10.3.1 (32/64 bit) and 10.4 (64 bit).

My workaround was to load a json object and parse that to generate the desired output.  Luckily, my requirement was to simply output individual csv files for the contents of the service layers/tables, so it was a matter of implementing the csv.DictWriter() to perform that task.

One other issue that came up was with date columns would be written to the csv outputs as epoch format, so I needed to run a conversion on any date column prior to writing the csv.