Achieving full compatibility with CDF - based services

[Francisco_R]

Is it useful to capture the fields list (and other parameters like return geometry only) when implementing the data provider functionality?

I could see that the published service (based on a full-fetch custom data provider) does already filter the response and returns the desired fields. Even though I have not explicitly captured the parameter value and implemented the filtering on the GeoJSON response.

How do I know what do we need to capture and handle at request level to produce a proper full compatible Feature Service, that behaves completely like a read-only FeatureService?

The where clause, returnCount, returnExtent and the pagination parameters are definitelly a must. But... what else? F.e. How to "build" a proper response when we query directly a feature by OBJECTID. We can query and get a list of IDs only, but the resulting hyperlink is broken. So the user cannot follow it.

[John_Hash]

Hello,

These are great questions, and the answers are based on the remote data set you are accessing, what the API in front of that data source supports, and how much control you want over what the custom data provider does.

You are correct that in a "full fetch" type of data loading pattern, the custom data framework will handle the parameters in the request and take care of filtering based on the 'outFields', 'where', etc. So if your data set is small and you can generally load all the data at once, this is a fine way to handle all those parameters.

The main reason you would want to capture those parameters yourself is if you can pass on those parameters to your data set's API and use them to filter the data at the remote data source. This will increase efficiency and allow you to interact with larger data sets that cannot be fetched all at once.

In your provider code, you will need to build in logic that inspects the incoming request parameters and decide how to query your remote data source based on those request parameters. For instance, if you want to be able to support querying your data source by OBJECTID, you would capture requests that have the 'objectIds' parameter, build an equivalent query by 'idField' (whatever your 'idField' OBJECTID equivalent is), query your remote, and return the GeoJSON for just those features that match that 'idField'.

Example 
Request:

https://yourserver.com/arcgis/rest/services/YourService/FeatureServer/0/query?objectIds=1,2,3&f=json

In the provider, have some "if" logic.

if (req.query.objectIds) // write logic for extracting oids

//convert to Mongo equivalent

db.collection.find({ myIdField: { $in: [1, 2, 3] } })

// query MongoDB

// convert to GeoJSON

// return GeoJSON

Refer to the examples here:
https://github.com/Esri/arcgis-enterprise-sdk-resources/tree/master/Samples/custom-data-feeds/mongod...

https://github.com/Esri/arcgis-enterprise-sdk-resources/tree/master/Samples/custom-data-feeds/duckdb...

Documentation on pass through provider data loading:
https://developers.arcgis.com/enterprise-sdk/guide/custom-data-feeds/pass-through-custom-data-provid...

[Francisco_R]

Thanks for a very good explanation. I see I might not have formulated my question properly, apologies for that.

I meant I have identified broken links when consuming the published service directly by feature ID, so this address: rest/services/<ServiceName>/FeatureServer/<layerID>/<ObjectID = 2> returns Not Found error.

{
 "error": {
  "code": 500,
  "message": "Feature (ID: 2) not found",
  "details": []
 }
}​

Instead for a JSON Definition of the feature with ObjectID = 2 like this example from a Hosted Service:

{
 "feature": {
  "attributes": {
   "globalid": "{4D9FC17A-661C-44F7-B688-E1AA94245F29}",
   "objectid": 2
  },
  "geometry": {
   "x": 1196029.2908000015,
   "y": 8378724.482199997
  }
 }
}

 
I would assume that I would need to define a route, and somehow parametrize the OID, so I can process the getData accordingly based on the OID (which will be basically filtering by ObjectID as you explained in your answer).
If that's the case. The route should probably look like (assumed 1 layer only):

    {
        path: `/<name>/rest/services/FeatureServer/0/:objectid`,
        methods: ['GET','POST'],
        handler: 'getDataByObjectId'
    }

And where I define a separate handler function in the Model to return single feature results. I guess the request parameters will expose that objectid for preparing the answer.

I wondered if there are more situations like these we have to handle. So we can achieve an equivalent service than a Pro-published regular FeatureService.
So far I have found (but I have just begun):

1. Capture Where clause (fundamental query)
2. Capture ObjectIDs (almost everything might leverage this)
3. Capture count and extent (metadata queries)
4. Capture spatial filter (Pro seems to leverage them by default)
5. Build a route and handler for direct OID queries

Other query parameters (returnIDs only, display fields, etc) seem to be handled at AGS level already. I Understand that I can still capture them if I need to customize the query against the datasource in advanced scenarios. But that would be more related to a query-optimization stage more than "achieving client-side compatibility".