Tracing SOE crashes SOC process

[MarkTurnbull]

Using ArcGIS Enterprise 10.6.1, I have created an SOE that performs connected/upstream/downstream traces against a Geometric Network. This works in most cases. When  the start point is on an certain set of edges, the trace crashes the ArcSOC.exe process for the service. The AGS log reports the following stack trace:

java.lang.Exception: Could not service request. at com.esri.arcgis.discovery.admin.rediscovery.actors.RediscoveryAppServer.handleRequest(RediscoveryAppServer.java:539) at com.esri.discovery.DiscoveryService.handleRequest(DiscoveryService.java:112) at com.esri.client.AGSRequest.handleRequest(AGSRequest.java:216) at com.esri.client.ServiceClient.handleRequest(ServiceClient.java:110) at com.esri.client.exts.ExtensionClient.handleRequest(ExtensionClient.java:189) at com.esri.client.app.exts.ExtensionHandler$2.doOperation(ExtensionHandler.java:205) at com.esri.client.app.templates.OperationTemplate.execute(OperationTemplate.java:119) at com.esri.client.app.exts.ExtensionHandler.handleOperation(ExtensionHandler.java:268) at com.esri.client.app.exts.ExtensionHandler.extensions(ExtensionHandler.java:94) at com.esri.client.app.exts.ExtensionHandler.service(ExtensionHandler.java:83) at com.esri.rf.RServlet.service(RServlet.java:121) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) 

When I debug the source, it actually crashes on the call to perform the trace,

traceFlowSolver.FindFlowElements(pFlowMethod, esriFlowElements.esriFEJunctionsAndEdges, out _junctionIEDs, out _edgeEIDs);

This line has try catch statements around it but the ArcSOC still just crashes with no caught exceptions.

I am able to trace through these edges if I start on another edge, i.e. start on a downstream edge with an upstream trace.

Note: the tracing does work on most network edges, it just fails on some and that subset of failing edges is persistent.

I have tried to investigate for a data issue or a GN issue through ArcMap. I have found that if I display direction arrows, the edge features that cause the crash do not get a direction arrows and "The spatial references do not match" errors appear for them. My data frame is set to Web Mercator (3857), to match the base maps my client wishes to use, whilst the source data is stored in MGA Zone 56 (28356), so I tried setting my data frame to MGA Zone 56 and all the direction arrows appear with no errors.

So I published my service with the spatial reference set to MGA Zone 56, but the SOC process still crashes for the same edges.

I can't seem to debug any further. Can anyone give me any clues as to either what may be causing the SOCs to crash or how I could debug further?