Issue reading optional field from JSON-file using Arcade

[AndreasHall]

I have a scheduled data pipeline that once a day reads a JSON-file and writes it to a feature service. In the pipeline, I do numerous calculate field where I use Arcade to get the information I need from the JSON-file. In these scripts I use $record. to fetch the information I need ($record.approximateEnd in the example below). The issue I am facing is that some of this information is not always present in the input file. When this happens, the arcade script throws an Execution error - Key not found, and the pipeline does not run/fails (no data is updated). I would like to modify my script so that it first tests if $record.approximateEnd exists. If it does not exist it would create a null value. Any other solution that would prevent my pipeline from failing when data is missing would also work.

My input data is SMHI Open Data API Docs - Warnings.

(I already check if $record.approximateEnd is null but I need something to cover the case when there is no $record.approximateEnd at all.)

// Check that EndDate exist, save as Date.

if(IsEmpty($record.approximateEnd)){
  return null;
} else {
  var MyDate = Split(Left($record.approximateEnd, 10),"-");
  var MyTime = Right($record.approximateEnd, 13);
  
  MyTime = Split(Left(MyTime, 8), ":");
  
  MyDate = Date(MyDate[0], (MyDate[1]-1), MyDate[2], MyTime[0], MyTime[1], MyTime[2]);
}

// Check that EndDate is later than StartDate

if (MyDate>$record.Start_) {
  return MyDate;  
} else {
  return null;
}

[KenBuja]

This code will check if the Feature or FeatureSet contains a field.

function fieldExists(item, fieldName) { //item can be a Feature or FeatureSet
  var fields = Schema(item).fields;
  for (var i in fields) {
    if (Lower(fields[i].name) == Lower(fieldName)) return true;
  }
  return false;
}

iif(fieldExists($feature, "field_name"), "Field Exists", "Field doesn't exist");

[DuncanMackey · Accepted solution]

Hi Andreas!

There are two relevant Arcade functions that will likely be helpful here:

• DefaultValue
• HasValue

So in your example, you could use DefaultValue similarly:

// Set end date to null if the field does not exist. 
var endDate = DefaultValue($record, "approximateEnd", null)

Hopefully these functions will give your arcade script the flexibility it needs to handle JSON with varying properties. Please let me know if you have any more questions regarding this.

Thank you,

Duncan

[AndreasHall]

Great, thanks a lot!