FME Heap Corruption Error when using Esri ArcSDE writer

[AngelaSchirck]

Hi all,

Here's the scenerio.  I have Three tables in an Oracle database that I am combining using FME.  I then am using An SDE Writer to write to a different Oracle database (the oracle writer put the data in a format that I cannot use with Enterprise 10.9.1, so I chose the SDE writer instead).  

There are close to 700k features that need to be written, but the FME crashes and the program closes after writing about 160k features EVERY.SINGLE.TIME. 

Things I've tried: 

• Redirected the output to the data inspector.  The work flow completes without errors.
• Added a geometry validator and repaired bad geometries and ignored those that couldn't be repaired (none). 
• Added a feature keeper to collect all the features and write them after join is complete (all ~700k features are collected, still crashes in the writing phase).
• Set the writer to ignore all invalid features and continue.
• Successfully wrote ALL features to a file GDB.

There is no error message in the FME, but a general error message pops up (shown below) which ChatGPT says is a "heap corruption error" and I have no idea how to fix it.  Any suggestions?

[VinceAngelo]

Is this the FME extension to ArcGIS or FME which is crashing? Esri doesn't support FME software.

Does it work with a file geodatabase target (which you could then process with less complexity)?

- V

[AngelaSchirck]

It's FME that's crashing, but ONLY when using Esri ArcSDE writer.  I can write to a file geodatabase using the Esri geodatabase writer with no issues.  I tried writing to that then reading from the GDB and writing to the SDE.  Still the same issue.  It's clearly a problem that exists only with the SDE Writer (AGOL writer works fine, GDB writer works fine).

[VinceAngelo]

Okay, it could be the Esri DLL that's at fault, but it's more likely the FME code. The best way to explore this is with FME tech support (Safe Software -- https://support.safe.com/knowledgesubmitcase ). It's best to build as small of a test dataset as exhibits the issue (and if size impacts it, be sure to let them know where the cutoff is).

[AngelaSchirck]

Thanks, I did work with FME support and we determined that the workflow is not the issue.

[AngelaSchirck]

So, I upgraded to FME 2024 and still had the problem, but after messing around and changing the SDE Writer settings something I changed eventually got it to work.  Unfortunately I'm not sure which changed setting did the trick, but weeks of frustration are finally over LOL

[AngelaSchirck]

Yeah, scratch that, it put my parcels in the middle of the Atlantic ocean near Africa instead of south Florida.  When I deleted the coordinate system it crashed after 167k again.

[VinceAngelo]

Uh, oh. That's Null Island (wrong coordinate system issue).

Have you tried using Check Geometry and/or Repair Geometry on the source layer(s)?

- V

[AngelaSchirck]

I have done ALL those things and then some.

The EsriProjector transformer was used to project the coordinates to ESPG:3857, then I chose ESPG:3857 on the SDE writer.  When I do this, it writes 167k parcels and crashes.  If I choose a different coordinate system on the writer (Say ESPG:4326) it writes ALL 658,315 features, but puts them in the ocean. 

Weird thing is it crashes when I choose the correct coordinate system, but works fine when it's not the right coordinate system.