nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

How to handle a case where there are no fields present in a JSON file?

AndreasHall

I have a scheduled data pipeline that once a day reads a JSON-file and writes it to a feature service. The file lists weather warnings (SMHI Open Data API Docs - Warnings). Some days there might be no warnings at all present. Then my pipeline fails. Is there any way to handle this use case with ArcGIS Data Pipelines? (If there is no data several days in a row my scheduled pipeline is disabled as it has failed too many times in a row.)

Find more posts tagged with

ETL

Accepted answers

DuncanMackey

Hi Andreas,

The empty JSON file is an interesting case, as Data Pipelines doesn't know the schema, hence the data pipeline failing. We'll look into some possible solutions for a future release. Thanks for bringing this up!

All comments

bharold

Andreas, I can supply an alternative workflow and have a blog on the topic which I'm publishing November 7th when ArcGIS Pro 3.4 releases - because it uses a new feature. The subject matter is weather alerts in CAP protocol, which I can see is available in your weather feed. Empty entries are OK. I'll let the pipelines team respond on what you are experiencing.

Sarah_Hanson

Hi @AndreasHall,

As you noticed, scheduling pauses when a data pipeline fails after too many failed runs (5), and this functionality was built in intentionally to prevent users from paying for the running of a data pipeline that may have a critical issue that needs to be remedied.

We will work to address this in a future release and will reach out to you if we need additional input.

Thank you!

DuncanMackey

Hi Andreas,

AndreasHall

Thanks for the feedback!

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.