topic Arcade - Filter Feature Set by Date in Developers Questions

Arcade - Filter Feature Set by Date

pmccord — Fri, 19 Nov 2021 05:21:50 GMT

I am trying to filter a feature set using a date field on ArcGIS Online. I need the filter to return a feature set of all records after a particular day. However, when I run the filter, I simply get back my starting feature set. In other words, it runs but it doesn't seem to be recognizing the filter expression that I've entered.

In the code below I'm creating two feature sets, one on line 4 and the other on line 5. Line 4 produces the feature set that I'd expect, but it's the filter on line 5 where I'm trying to get all records that have a date after the inspection date that doesn't seem to be working.

Is there additional formatting that I need to do to my inspectionDate variable to get it to evaluate correctly against my DATE field?

var inspectionDate = $feature.DATE var client = $feature.CLIENT var meter = $feature["SERIAL__"] var fSetByMeter = OrderBy(Filter($layer, 'CLIENT = @client AND SERIAL__ = @meter'), 'DATE ASC'); var fSetCurrentDate = OrderBy(Filter(fSetByMeter, 'DATE > @inspectionDate'), 'DATE ASC') return fSetCurrentDate

Re: Arcade - Filter Feature Set by Date

JohannesLindner — Fri, 19 Nov 2021 07:24:50 GMT

Yeah, filtering by date can be iffy, as different DBMSs handle it differently.

Try converting your date to a string:

var inspectionDate = Text($feature.DATE, "Y-M-D")

Re: Arcade - Filter Feature Set by Date

pmccord — Fri, 19 Nov 2021 13:46:07 GMT

Thanks for the response Johannes. When converting to a string, I'm still having the same problem where, on line 5 above, the filter returns all of my records from the Feature Set again. In other words, it doesn't seem like the filter that I'm applying is actually filtering anything out.

$feature.DATE is a date field, so wouldn't I need inspectionDate to also be a date rather than a string?

I've included a screen shot of some of the rows from my feature set:

inspectionDate is initially September 8, 2014, so I'm trying to filter for every record that's after September 8, 2014. As you can see from the screenshot above, all records, including September 8th, were returned.

Thanks,

Paul

Re: Arcade - Filter Feature Set by Date

jcarlson — Fri, 19 Nov 2021 14:35:05 GMT

Hard to say for sure, but I would guess that it's because DATE is a reserved SQL keyword, which can throw off a query, as the database may be expecting something else when it sees that keyword, rather than assume it to be a field in your table.

What Arcade profile are you using, and where is the featureset coming from? Is it feasible to construct a featureset manually by populating a dict of features and rename the date field there?

Re: Arcade - Filter Feature Set by Date

pmccord — Mon, 22 Nov 2021 04:26:28 GMT

Great suggestion Josh. I converted my feature set to a dictionary, and while populating the date value of my dictionary, I converted the date to Unix time. I converted to Unix time as a result of stumbling on this post:
https://community.esri.com/t5/developers-questions/arcade-dictionary-to-featureset/td-p/1047117

Once I had all the date values in Unix time, I converted my dictionary back to a feature set. I then converted the inspectionDate value to a Unix timestamp as I did above when creating the dictionary:

var inspectionDate = $feature.DATE var start = ToLocal(Date(1970, 0, 01, 0, 0, 0, 0)) var dt = DateDiff(inspectionDate, start, 'milliseconds')

After doing this I was able to successfully filter for all dates after the inspection date.

I'm sure there's a more efficient way to do this, but I'm calling it good now that I'm getting the result I want. Thanks again!

Re: Arcade - Filter Feature Set by Date

TórshavnarKommuna — Sun, 24 Sep 2023 09:43:10 GMT

Hi.

I was having the same problem. Assuming the the data are coming from a ArcGIS feature layer, when you filter a featureset, it is sending a request to REST API to a Query operation. After testing and checking some documentation, I found out that to define a "where clause" for query operation using a date field it is necessary to add DATE in front of date value. So In your case this could work:

var fSetCurrentDate = OrderBy(Filter(fSetByMeter,'DATE > DATE @inspectionDate'), 'DATE ASC')

However, I don't know if using the DATE as a field name can have some issues of its own, as it is often a reserved keyword in DMBS system. Also, I didn't got it to work by passing date in variable such as "@insperction_date". But here is an example with my code which was working for me:

var read_out_date = Text($feature.collection_date, "Y-MM-DD"); // Define an sql query and filter the data var sql = "collection_date < DATE '"+ read_out_date +"'"; var filteredDataSet = Filter(inputFeatureSet, sql);

Finally here is a link to a post about querying date fields at REST API:
https://www.esri.com/arcgis-blog/products/api-rest/data-management/querying-feature-services-date-time-queries/