In the February 2024 update of ArcGIS Data Pipelines, the application moved out of beta and is now available for general use. Transitions from beta to general availability are subject to breaking changes. Please be aware of the following changes in the February 2024 update:
- The results from scheduled data pipelines that were run during beta will no longer be accessible from the scheduling UIs in the Data Pipelines app. Output feature layers or tables that were generated will still be available in your content, and any active tasks will continue to run as scheduled. Results returned after the update will be available in the scheduling UI.
- The following field types from Google BigQuery are now read differently:
- BigNumeric fields are now read as type Double. Previously, fields of this type were not supported and were dropped.
- Datetime fields are now read as type Date. Previously, fields of this type were read as strings.
- The behaviour for casting numeric fields when using the Update fields tool has changed in the following ways:
- Casting a decimal value to an integer value now rounds to the nearest integer. Previously the value would have been truncated at the decimal regardless of the decimal value. For example, previously a double value of 2.9 would have been cast to an integer value of 2. Now the value will be cast to a value of 3.
- Casting a numeric value to a smaller type could now return null instead of the largest supported value for that field. For example, if you cast a big integer value of 9223372036854776000 to an integer, the result was previously 2147483647 (the largest supported value for an integer field). Now the value will be returned as null.
- Encoded URLs are no longer supported.
- For example, URLs such as the following which uses encoded values: https://opendata.org/api/geojson?query=issuedate%3A2024&timezone=America%2FLos_Angeles
- To resolve the encoding issue, you can use a publicly available decoder, such as this one. Just enter your URL with encoded values, click decode, and use the resulting URL as input to the Public URL tool in Data Pipelines.
If you have questions about changes you observe in Data Pipelines or how to remedy the changes, please reply to this thread or make a post in the Data Pipelines Questions forum.
To learn more about the February 2024 update of Data Pipelines, check out the What's new documentation and the ArcGIS Data Pipelines is Now Available blog post.
