Having trouble with the GeoEvent input connector "Watch a Folder for new GeoJSON Files"

203
2
01-25-2021 09:40 AM
AbigailStephens
New Contributor III

We are trying to use GeoEvent server to pull in updated data from an external system using the "Watch a Folder for new GeoJSON Files" input connector.

Right now, the set-up is that the server machine housing GeoEvent calls an external URL to download a GeoJSON each hour and saves it in a folder on the machine. Then GeoEvent watches that folder for new GeoJSONs. This process was working as expected with the initial GeoJSON pull that it was tried with. We then realized there was a different GeoJSON we should be pulling, and updated the call on the server machine to pull that GeoJSON. The only difference between these GeoJSONs is their size (due to an increase in features included) - the original pull was about 3.5 MB and the new one is about 25.5 MB.

Is there a limit for the size of a GeoJSON that GeoEvent can handle? Or is there something additional we should look at?

For reference:

  • We are using ArcGIS Enterprise 10.7.1 on Linux
  • All the data in both is in one line, so changing the settings for line batch processing hasn't had any effect
  • The Input Folder DataStore + Input Directory parameters give the full path to the folder in which the file is saved, and we have checked that the pulled and saved GeoJSON is in the folder indicated. The file filter is set to ".*\.geojson".
  • Based on preliminary review, there doesn't appear to be anything wrong with the format of the GeoJSON, and it properly converts to a feature class in ArcGIS Pro.
0 Kudos
2 Replies
EricIronside
Esri Contributor

Hey,

Seems like you've done everything right. Two things to check:

The first is the filename. GeoEvent may ignore a file it thinks it has already processed, so try to append a datetime to the end of the filename as it is saved.

For the second, please see the following blog on wathching folders for files. Sometimes the file is large enough that the file system takes some time to write the file; if GeoEvent tries to grab the file while it is still being written, it can cause issues. 

GeoEvent: Watch a folder for new files 

0 Kudos
AbigailStephens
New Contributor III

@EricIronsideThanks for the tips! I will look into both of those, I could see both items contributing to the problem.

0 Kudos