ArcEnterprise 10.8 Service Definition File Size Limit

[DavidColey]

Hello - I don't think that this section from the Enterprise help

Manage hosted feature layers—Portal for ArcGIS | Documentation for ArcGIS Enterprise 

at 10.8 is entirely accurate:

"Allow attachments

You can allow editors to attach images and other files to individual features in a layer in a hosted feature layer. This is useful, as it allows you to associate documentation or photos to specific, relevant features. For example, a code compliance officer might attach a photo of the code violation for a specific address point, or a building inspector might include a PDF of a permit for a building feature.

Each hosted feature layer view inherits the attachment setting of the hosted feature layer from which it was created. By default, all view users can see the attachments inherited from the hosted feature layer. To control who has access to these attachments, you can hide the attachments in the hosted feature layer view.

Each file you attach to a feature can be a maximum size of 2 GB. To attach files larger than 2 GB, you can use the Upload Part and Add Attachment operations from ArcGIS REST API to do a multipart upload. . . . . . "

What I have found is that when sharing or overwriting a hosted feature layer via the 2.5.x ArcGIS Pro sharing module to Enterprise 10..8, the maximum service definition file size appears to be 2GB.  

In my case, I have a point layer with some 3600 records and pdf attachments.  The largest attachement is about 9MB.  However, the total size of the service definition file exceeds 2GB, and the sharing module overwrite now fails with a 99999.

Specifically, the:

## arcpy.UploadServiceDefinition_server('C:/ArcProProjects/PortalUpdates/Environmental/' + lyr.name + '.sd', 'My Hosted Services', "","","","","","OVERRIDE_DEFINITION","SHARE_ONLINE","PUBLIC",
## "SHARE_ORGANIZATION","")

fails because I think that the entire sd file size cannot exceed 2GB.

Has anyone encountered this?

[DavidColey]

Sad to say this is still an issue at Enterprise 10.8.1 and ArcGIS Online.  The:

sdFile = os.path.join(sdPath, "FemaElevationCert.sd") #item[:-4]

sdItem.update(data=sdFile) 

fs = sdItem.publish(overwrite=True) 

method continues to fail as does the overwrite method from the feature layer collection manager"

portalFLC.manager.overwrite(sdFile)

whatever it is, for sd files with attachments that are larger than 2Gb can be added and published as new items through the api, but cannot be overwritten.  The json error remains:

Expecting value: line 1 column 1 (char 0) after a certain amount of time. (@ 1 minute on portal, ~ 21 minutes on agol).

Jonah Lay‌ thought you might want to know

[DavidColey]

So I gave up on this.  I began by setting up a virtual directory in my IIS and tying that to a sever directory.  I pushed all my documents into the server directory and set up a link field in arcade.  If I need to link to more than one doc for any given feature, I'll use arcade for that too, most likely in a when statement.  Sure is alot easier than trying push all these attachments as part of an sd file to the datastore.

[GayathriVijay]

Hi David,

I was able to successfully publish and overwrite an SD with attachments greater than 2GB from Pro using python window. I was also able to publish the data from scratch from Pro and also overwrite the same. I would suggest you contact Esri support and give your data, as well as the information about the errors/warning you see in the server logs.

Thanks,

Gayathri Vijay

[DavidColey]

Yeah I have done all that.  Naming convention of the pdf attachment documents becomes very important when the total size of the service definition file causes the file to upload the json in parts.  When a json part is encountered that contains some invalid character at the beginning of the part - like an ampersand, the initial upload does not necessarily fail, but all of my overwrite attempts do. 

Now that I understand that, I understand why my large service definition files with a 100s to 1000s of attachments, would fail upon overwrite, but not in the same place.