Latest Contributions by RJSunderman

Thank you @RPGIS ... unless I'm doing something wrong, though, the Geometry and Polyline functions do not accept my JSON string. It seems like the specified "spatialReference":{"wkid":4326} is not supported. Cross reference Arcade Geometry Function not working  If I remove the spatial reference from the JSON string Arcade will construct a Geometry which assumes WKID 3857. That does not interfere with pulling coordinate values out as Double. But it does feel odd specifying a Geometry without a spatial reference and allowing Arcade to assume an incorrect default. var G = Geometry({"paths":[[[-86.762566,36.180317],[-86.760232,36.184474],[-86.75721,36.186857],[-86.75467,36.188519],[-86.752679,36.194005]]],"spatialReference":{"wkid":102100}}) var Paths = G.paths for (var i=0; i < Count(Paths[0]); i++) { Console( "Longitude: " + Paths[0][i].y + ' / ' + "Latitude: " + Paths[0][i].x); }    ... View more

Thank you @AlfredBaldenweck for your quick reply.  Expressions like those below give me the access I need to the elements in the JSON. var d = fromJSON('{"paths":[[[-86.762566,36.180317],[-86.760232,36.184474],[-86.75721,36.186857],[-86.75467,36.188519],[-86.752679,36.194005]]],"spatialReference":{"wkid":4326}}') var pFirst = First(d.paths[0]) var pLast = First(Reverse(d.paths[0]))   ... View more

Hello Wadson -- A lead from the team working on supporting stream services and stream layers more broadly across the ArcGIS Enterprise reviewed the incident you opened with Esri Technical Support (Case #04081256). They've let me know that attribute-driven symbology (configuring a web map's layer to display feature records differently based on each feature record's attribute values) was introduced into the Web Map Viewer with the 11.2 release of Portal for ArcGIS. At earlier versions, the map viewer was able to consume and display a map prepared in ArcGIS Pro (where attribute symbology is configurable), but the map viewer was not able to edit, change, or reconfigure the attribute-driven symbology. Other options for configuring attribute-driven symbology she mentioned were using ArcGIS Assistant to edit the JSON and define a desired renderer. If you have deployed ArcGIS Enterprise 11.2 or later, I would definitely recommend working with Esri Technical Support to figure out how to use controls available in the Web Map Viewer to do what you are trying to do. Information on customizing Experience Builder is rather thin. The team supporting stream service and stream layer enhancements have not worked with the team supporting Experience Builder recently and are not aware of any plans to extend ExB to enhance stream layer support. My understanding, based on what I've been told, is that stream layer support in ExB is limited to displaying feature records as they are broadcast by the stream service -- there is no available option for users to configure the decay or automatic removal of "old" feature records from the layer's display. Some good news, however, is that your concept of setting a "maximum duration" property may be coming to the web map viewer. The team is working to design a way to enable users to specify a date / time so that feature records which are older than the specified value will not be displayed. Support for tracked assets and configurable properties like maxDuration might be coming out later this year for the map viewer web application -- but these are not options available to users today. Please continue working with Esri Technical Support to capture product requirements for your operational concept. The analyst can help feed those back to the development team. -- RJ ... View more

Josh Joyner's original post has been updated to show the simplified user configuration interface for Release 7 of the Waze connector ( August 25, 2023 ) mentioned by Gregory Christakos above. --- --- You will likely immediately notice, when sending traffic alert and traffic jam event records to an output to either broadcast feature records (using a stream service) or add/update feature records in a geodatabase (using a feature service), that feature records accumulate over time which end up cluttering your display. When using a stream layer in a web map to display data, you cannot configure the stream layer to periodically refresh the display. Stream layers do not query data from a geodatabase and do not offer an automatic refresh to remove feature records you might consider "old" or "stale" from the web map display and query to obtain a fresh set of feature records. Your only option, as an analyst, is to periodically clear the stream layer's display interactively when you decide the web map's display needs to be refreshed. Select the option to "Clear previous observations" from the stream layer's context menu to remove all displayed feature records for the layer and then wait for a fresh set of feature records to be processed through GeoEvent Server. Alternatively, you could use a feature layer rather than a stream layer to display your data. The only real advantage to using a stream layer in this case would be that feature records are added to the display immediately after being processed by GeoEvent Server. The inherent delay in receiving data from the Waze API -- typically no faster than every 120 seconds -- means that configuring an automatic refresh for your feature layer of every 30 seconds will reasonably display feature records as they are processed in near real-time. An advantage of using a geodatabase to store the accumulating feature record set is that you can configure a hosted feature layer view to show you only the most recent traffic alerts and traffic jams. This means that as a web map remains open its display will not become cluttered with stale alerts and jams. From your Enterprise portal, open the detail page for your hosted feature layer and select Create View Layer. Choose the option to "Create a view of this layer". With the hosted feature layer selected in the Create View Layer configuration panel, click Next to define the filters and fields you want included in the hosted feature layer view. Click the expansion arrow to add a filter to the view and click 'Add new' : You will need a date/time field which is updated every time the feature record used to model a traffic alert or traffic jam is updated. This is not part of the data schema included by the Waze API. Refer to the illustrations below for how to use GeoEvent Server to enrich event data records with the needed information. For now, you need to know that a filter expression can be configured for a hosted feature layer view which selects only feature records whose date/time is within the last several minutes : To complete your hosted feature layer view configuration, click the expansion arrows to collapse the 'Filter' and 'Layer definitions' panels then click 'Next'. Enter a title for your hosted feature layer view, something like "Traffic_Jams_Latest", enter tags and a summary if you wish, and then click Create. You can now add your hosted feature layer view to a web map. When you configure the feature layer (in the web map) to automatically refresh every 30 seconds (e.g. 0.5 minutes) the view will retrieve and display only the most recent feature records from the hosted feature layer. Note that you will still be required to periodically administer your geodatabase to remove / delete old feature records as data accumulates in the hosted feature layer. --- --- Using GeoEvent Server to add the received_dt attribute This is done using the same Field Mapper processor you need to configure to flatten a traffic alert or traffic jam event record schema before sending the simplified / flattened data record to an Update a Feature output. First, you need to recognize that integrating the Esri Sample Waze Connector into your GeoEvent Server added four GeoEvent Definitions to your deployment. There is a GeoEvent Definition named WazeAlert and one named WazeAlert-Flat. The latter omits the X/Y coordinate values nested beneath location and also drops the imageId and the imageURL attributes. New attributes longitude and latitude are added for the coordinate values. A similar pair of GeoEvent definitions are loaded into your configuration for WazeJam and WazeJam-Flat. You need to create a new pair of GeoEvent Definitions and add a new field which you can use to write a Date value each time a traffic alert or traffic jam event record is processed. You can easily do this by copying the existing WazeAlert-Flat and WazeJam-Flat GeoEvent Definitions and then adding the needed received_dt attribute to your copy of the event definition. Make sure you specify the data type for the new attribute is a Date. When configuring your Field Mapper to flatten the traffic jams schema, use the built-in function currentTime() to retrieve the local server's current time and write that, as an epoch date value, into the new attribute field received_dt. Your field mapping should be from WazeJam (the event definition used to adapt the raw data from the Waze API) to the flattened schema WazeJam-ReceivedDT (the copy of the flattened schema to which you added the received_dt attribute). Hopefully this information helps manage the traffic alerts and traffic jams you process through GeoEvent Server when adding and updating feature records in a geodatabase for display using hosted feature layers and hosted feature layer views. ... View more

The issue you are seeing with feature services needing to be republished is not something I have seen recently. I have not been able to reproduce the issue personally and I cannot say whether or not work has been put into scope for ArcGIS Server. If you want to open an incident with Esri Technical Support they can work with you to capture a reproducible case and then document the bug. If you do open a technical support incident, please make sure the incident refers to an upgrade of ArcGIS Enterprise and ArcGIS Server with emphasis on web service availability at REST after the product upgrade. ... View more

Happy to try and help Seth.  If we specify the name of an interior node for the input to use as an XML Object Name, no, there is no way to capture data above / outside of the substructure of the specified node. The XML Object Name is intended to be used to specify the node to use as the "root" when parsing the XML, so any XML above or outside of that "root" node is necessarily ignored. The data outside the "root" node won't be included in the adaptation run by the input connector, so the data won't be available for processing as part of a GeoEvent Service. That means that we have to work with <Incident> as the actual root node. The multicardinal hierarchy of the XML makes it pretty tricky to create a GeoEvent Definition. You can allow the input to create a best first-guess at what the GeoEvent Definition ought to be (mostly to save you a bunch of mouse clicks creating a GeoEvent Definition from scratch). But if you work from a managed GeoEvent Definition created for you, you will have to copy and make some specific edits to your copy of the GeoEvent Definition before you can use it. There are a couple of behaviors you will need to work around when you then reconfigure your input connector to use your copy of the GeoEvent Definition. GeoEvent Server handles JSON better than XML I'm afraid. --- --- --- Try this first: Configure your input to create a GeoEvent Definition for you, making sure to leave the XML Object Name unspecified. When looking for a root node, rather than using one you specify, the input seems to insist that it be allowed to both recognize <Incident> as the root of the XML but then create a GeoEvent Definition which ignores <Incident> as a node in the data being received. Copy whatever GeoEvent Definition the input generates for you, then edit your copy to match the illustration below. When you edit your input to not generate a GeoEvent Definition for you, you can specify your copy of the GeoEvent Definition (the one you edited) but then you must also specify the input connector use <Incident> without the brackets as the XML Object Name. You should make sure to delete the managed GeoEvent Definition created by the input, mostly so that you don't accidently reference it later when configuring your solution. You should have a GeoEvent Definition you created which matches the illustration below, with an input  configured to use that GeoEvent Definition (specifying Incident as the XML Object Name), and be able to receive and adapt the XML shown below before continuing. ( ! )  Both <Exposure> and <ExposureUser> need to be edited to specify they are multicardinal groups. Here is the input configuration : You can use any REST API you are familiar with (a lot of folks like to use Postman) to HTTP/POST the XML below to your input's endpoint shown as the URL property in the illustration above. <Incident> <Other> <UniqueIncidentIdentifier>ABC12345</UniqueIncidentIdentifier> </Other> <Exposures> <Exposure> <ExposureUnit>7894389</ExposureUnit> <ExposureUser> <ExposureUserID>10223</ExposureUserID> <ExposureUserName>Alpha</ExposureUserName> </ExposureUser> <ExposureText>First exposure unit description</ExposureText> <Exposure_Dttm>1739482095261</Exposure_Dttm> </Exposure> <Exposure> <ExposureUnit>7894395</ExposureUnit> <ExposureUser> <ExposureUserID>10628</ExposureUserID> <ExposureUserName>Bravo</ExposureUserName> </ExposureUser> <ExposureUser> <ExposureUserID>10545</ExposureUserID> <ExposureUserName>Charlie</ExposureUserName> </ExposureUser> <ExposureText>Second exposure unit description</ExposureText> <Exposure_Dttm>1739482438936</Exposure_Dttm> </Exposure> </Exposures> </Incident>   --- --- --- Now we need to work on your original question, which is how to detect when the XML received does not have any content within the <Exposures></Exposures> structure. This could get a little messy. I think we are going to have to look at attributes we expect will have null values and simply assume that when we see the null values that the reported <Incident> has no <Exposure> data. You cannot use any processor to write to an event record structure with hierarchy, but you can configure a filter to look into an event record's hierarchy using "dot notation" to peek at specific attribute values. The filter expression for each Filter is very similar. The first one toggles the pulldown to the left of the expression to specify 'Not' where the second "No Exposure" Filter uses the same expression without the logical 'Not' ... You might assume that when Exposures.Exposure isn't defined, or is null, that a reference into data beneath that -- to look at Exposures.Exposure.ExposureUnit for example -- would result in some sort of null reference  and generate an error or exception. Lucky for us error handling in the implementation of a Filter element allows us to drill down into null data without generating any such exception. --- --- --- Here, then, is a JSON representation of data allowed to pass through the upper Filter, because the incident has one or more Exposure nodes with an ExposureUnit whose data is not null : [{ "Other" : { "UniqueIncidentIdentifier" : "ABC12345" }, "Exposures" : { "Exposure" : [ { "ExposureUnit" : "7894389", "ExposureUser" : [ { "ExposureUserID" : "10223", "ExposureUserName" : "Alpha" } ], "ExposureText" : "First exposure unit description", "Exposure_Dttm" : "1739482095261" }, { "ExposureUnit" : "7894395", "ExposureUser" : [ { "ExposureUserID" : "10628", "ExposureUserName" : "Bravo" }, { "ExposureUserID" : "10545", "ExposureUserName" : "Charlie" } ], "ExposureText" : "Second exposure unit description", "Exposure_Dttm" : "1739482438936" } ] } }]   And here is a JSON representation of data allowed to pass through the lower Filter when the incident has  no data within its <Exposures></Exposures> structure. The array Exposure is empty. There are no data elements in the array, so there is no ExposureUnit string with a value which is not null. [{ "Other" : { "UniqueIncidentIdentifier" : "ABC67890" }, "Exposures" : { "Exposure" : [ ] } }]   I think as long as <Incident> has a cardinality of 1, and we can assume that a data substructure <Exposures></Exposures> will always exist -- even if there is nothing in the substructure -- the above will work for detecting incidents with no exposure. -- RJ ... View more

Hello @CoffeeforClosers -- There really isn't any logic you can configure an inbound connector to use to check raw data the connector has received and determine how many records exist -- or if no data exists -- for a particular data structure. I think we can still work with the scenario you describe though. ( Thank you for the detailed data structure. ) I tried configuring an out-of-the-box Receive XML on a REST Endpoint input to allow me to send different  blocks of XML data to test how the data is adapted. This approach should work just as well using a Poll an External Website for XML connector to periodically query a web service. The key was to specify the name of an XML node the input can look for when adapting the data. When you specify a value for the XML Object Name parameter the input will parse through the XML document looking for instances of that node and adapt each instance it finds as a separate data record. In this case I think we want to direct the input to look for <Exposure> nodes. You would enter this into the input connector's configuration without the angle brackets as illustrated here: You'll notice that I configured the input to use a GeoEvent Definition the user owns rather than allow the input to try and create and maintain a GeoEvent Definition. Avoiding the use of a "managed GeoEvent Definition" in this case is a best practice. Here is the GeoEvent Definition "CoffeeforClosers_XML" : Each <Exposure> record found (there can be zero or more in the XML document) should have at least the four attributes described in the event definition above. ExposureUnit is a simple string (cardinality: 1). The ExposureText and Exposure_Dttm are also adapted as a simple string and date respectively (both cardinality: 1). The ExposureUser has to be configured as a 'Group' because it has attributes ExposureUserID and ExposureUserName nested within it. It also has to be configured as cardinality 'Many' because each <Exposure> record is expected to have one or more <ExposureUser> nodes. I tested three different blocks of XML to verify that specifying the input look for <Exposure> would not have a problem if the XML document were <Exposures></Exposures> with no interior content. In each case we are expecting the input connector will scan through the received XML looking for blocks with a "root" node <Exposure>. It will then try to apply the GeoEvent Definition it was configured to use to adapt each <Exposure> block's interior content as a separate event record. Here is the JSON that would be written out for the second and third test (the first test has no data for the input to adapt, so no data record processing would be performed -- but the inbound adapter is able to handle this without producing an error). I hope this helps with what you were trying to do. The only piece missing would be if you need GeoEvent Server to produce some notification or alert, for example, when an XML document is received with only the nodes <Exposures></Exposures> and no interior content. I don't think we are going  to be able handle that case since there is no way to configure the input connector to process the absence of data, and there is no way to check the count of records in a received data structure as data is being adapted. -- RJ ... View more

Hello @PierreloupDucroix -- The Poll an ArcGIS Server for Features input uses token authentication, but cannot be configured to use your GP Service. This type of input uses credentials you have entered into a registered server connection to request an authentication token from the ArcGIS Server (or the Enterprise portal) and then uses that token to make authenticated requests. The token returned from the ArcGIS token service gets incorporated into requests the input sends when querying ArcGIS feature services. This is not something you specifically configure; it is inherent in the inbound connector's implementation. This is also our only input that supports paging as it is able to assume the data paging strategy used by ArcGIS Server feature services. You are correct -- the Poll an External Website for JSON input does not have a mechanism implemented to obtain an authentication token, either from an external web service or (in your case) a GP Service. Even if the "external" server were actually an internal server running ArcGIS Server, this type of inbound connector can only be configured with a URL (and optionally query parameters to pass with the URL) to send single requests to a web service for data. Basically, if you can copy/paste a URL into a web browser, this type of GeoEvent Server input can be configured to make the same query at some specified interval. It cannot query a web service to first acquire an authentication token and then incorporate that token into a second query for data. Nor can it make multiple requests to "page" through a large collection of data records. These sort of multi-step requests for authentication and/or request pagination are not configurable using any out-of-the-box input. Generally speaking an outbound connector (or "output") is able to make requests on a web service, but is not going to block or wait for any sort of response. That is why we don't have an out-of-the-box output to send requests, for example, to a GP Service. The GP Service should be allowed to run asynchronously and return a job number to the requesting server. As the termination of an event processing workflow, an output is not going to do anything with a job number returned from a GP Service. A custom processor might be developed to perform a blocking / synchronous operation and make calls to a GP Service, but this is a very bad idea. You should never have a GeoEvent Server element you've configured or developed invoke a GP Service synchronously and allow the HTTP request to block until the GP Service has completed its operation(s). GeoEvent Server was architected to process hundreds to potentially thousands of event records each second. A GeoEvent Service that performs a blocking operation with a GP Service runs orthogonal to GeoEvent Server's design. The advice is the same if you were considering developing a custom inbound transport invoke a GP Service as part of an authentication workflow. You do not want an input to make requests to a GP Service as a blocking operation as part of data ingestion and adaptation prior to event record processing. Rather than refer you to blogs on using GeoEvent Server's Java SDK (which is what you would use to develop a custom inbound / outbound transport or adapter -- or a custom processor) I'm going to recommend a different pattern you can follow to obtain authentication via a GP Service. It is not always possible to send subscription requests to a web service provider so that the web service will periodically push data to your GeoEvent Server. My recommendation for authentication workflows is to handle the authentication externally to GeoEvent Server. Consider developing a "bridge" between the data provider and your GeoEvent Server. The advantage to this approach is that you can develop the "bridge" in any language you are comfortable using -- Python script, PowerShell, ASP.NET, as an Amazon web services Lambda function ... etc. Your bridge will handle token acquisition and request authentication. The bridge can then query the web service you were thinking to configure a Poll an External Website for JSON input to perform initially. The bridge can obtain the authentication needed to authorize its queries and to pass a token (for example) along with its requests for data. You then develop a way for your bridge to relay the data to  a GeoEvent Server hosted REST endpoint as an HTTP/POST request. This allows you to configure a Receive JSON on a REST Endpoint input to receive the data. Developing this sort of bridge between a data provider and GeoEvent Server can be very useful. It enables you to send multiple requests to page through large data collections because you can develop the bridge to conform to whatever paging strategy an external web service might use. The bridge can handle  necessary authentication via whatever authorization mechanism is being used. The bridge has an opportunity to "clean" the data before relaying it to GeoEvent Server if, for example, there are characters in attribute field names like '$' that GeoEvent Server does not support. You can also use the bridge to implement a throttling mechanism and relay large amounts of data to GeoEvent Server in reasonable batches of so-many-records-per-second. -- RJ ... View more

Have you read through the examples in the thread:  Timestamps received from a sensor feed display differently in GeoEvent Sampler, ArcGIS REST Services queries, and ArcGIS Pro  I think your expression replaceAll(myField, '(\d+)[/](\d+)[/](\d+)', '$3-$1-$2T00:00:00') may be taking the value "3/7/2024 12:39:42.784 PM" and producing the value "2024-3-7T00:00:00 12:39:42.784 PM". Two problems with that are the month and day lost their leading zeros and the ISO 8601 format you're trying to create has unwanted characters from the original time appended to the tail of the string. If you are OK with GeoEvent Server assuming the value "3/7/2024 12:39:42.784 PM" is local server time, you could edit the GeoEvent Definition  your input is using adapt the date/time string to specify the value be adapted as a Date and specify the input apply MM/dd/yyyy hh:mm:ss.SSS aa as the Expected Date Format parameter. Using this formatting string to "teach" the input how to parse your date/time string, I was able to adapt the string "3/7/2024 12:39:42.784 PM" as an epoch value 1709843982784.  I could then use a Field Mapper to cast this Date value to a String "Thu Mar 07 12:39:42 PST 2024" to double-check the data was adapted properly. -- RJ ... View more

@Moi_Nccncc -- You are asking if an event record you've received, containing a polygonal area, can be enriched with the geometry of a point geofence to produce an event record with two Geometry attributes? An event record in GeoEvent Server is allowed to have more that one field whose type is Geometry, but feature records require that their geometry be referenced by name as 'geometry' -- so you will have to first transfer the event record's geometry to some other named attribute field to avoid overwriting the event record's geometry with a feature record's geometry. If you were to use a GeoTagger processor to enrich the event record (polygon geometry) with the name of a single geofence (point geometry) that shares a spatial relationship with the event record (e.g. a point geofence which intersects the event record's polygon) you could use a Field Mapper to make sure attribute fields you want to pull into the event record exist, then use a Field Enricher configured to write to 'Existing Fields' to fetch those named attribute values from whatever feature record you used to register the geofence originally. The workflow I think you're looking for would be something like: GeoTagger (to get the name of one specific geofence you can use as a key for an attribute join) Field Mapper (to map the event record's geometry to a field named something other than 'geometry'). Also to add fields to the event record which exist in the feature record schema whose data you want to import. It is a best practice to configure Field Mapper to write to 'Existing Fields' rather than allow the processor to create a managed GeoEvent Definition for you. Field Enricher (to pull attribute data from an identified feature record into the existing attribute fields) If multiple point features intersect an event record's polygon, the GeoTagger will find multiple registered geofences which satisfy its spatial relationship (e.g. Intersects Any). The resulting comma separated list of geofence names you get from the GeoTagger means that you do not have a single unique value to use as the key for the attribute join the Field Enricher needs to perform its operation. You could use a Field Splitter to split the comma delimited string and produce several individual event records, each with exactly one named geofence, then enrich each of those event records with attributes you pull from the feature record(s) you originally used to register your geofences. I would strongly recommend that you ask your Customer Service Representative about options to connect with a technical advisor or Professional Services Delivery consultant who can help you with the sort of operations you are attempting to configure GeoEvent Server to perform. -- RJ ... View more

@Moi_Nccncc -- You should submit this as a new request to Esri Technical Support so an analyst can look at the problem with you. GeoEvent Server uses Apache Kafka as an event record broker. Kafka uses on-disk topic queues to hold event records a producer needs to convey to a consumer. In this case the inbound connector (input) is the producer and the GeoEvent Service you have configured to use that input is the consumer. It is strange that you are not seeing anything in the GeoEvent Server's karaf.log system log. You would normally see ERROR messages that an event record could not be delivered within a default timeout of 60,000 milliseconds (60 seconds) and the event record was lost. There might be WARN messages logged around the error condition. I would also expect to see error messages with keywords or phrases like "timeout exception" and/or "expiring records".  There are two JVM which support a running GeoEvent Server. One is for the GeoEvent-Gateway (which manages the Kafka message broker and Zookeeper configuration store). The other is for GeoEvent Server proper. GeoEvent Server depends on the GeoEvent-Gateway. If something is wrong with the Apache Kafka on-disk topic queues, or with the GeoEvent-Gateway service or the JVM the service runs, you can see what you describe where an input is able to receive (ingest) and adapt data to create event records, but the Kafka message broker is unable to route the instantiated event record to a GeoEvent Service for processing. Problems with Kafka message brokering usually require an administrator administratively reset GeoEvent Server. Uninstalling and re-installing won't work if the Kafka and Zookeeper files left behind in the Windows %ProgramData% folder are corrupted or unusable. -- RJ ... View more