topic Re: FME Heap Corruption Error when using Esri ArcSDE writer in Data Management Questions

FME Heap Corruption Error when using Esri ArcSDE writer

AngelaSchirck — Mon, 08 Apr 2024 15:13:22 GMT

Hi all,

Here's the scenerio. I have Three tables in an Oracle database that I am combining using FME. I then am using An SDE Writer to write to a different Oracle database (the oracle writer put the data in a format that I cannot use with Enterprise 10.9.1, so I chose the SDE writer instead).

There are close to 700k features that need to be written, but the FME crashes and the program closes after writing about 160k features EVERY.SINGLE.TIME.

Things I've tried:

Redirected the output to the data inspector. The work flow completes without errors.
Added a geometry validator and repaired bad geometries and ignored those that couldn't be repaired (none).
Added a feature keeper to collect all the features and write them after join is complete (all ~700k features are collected, still crashes in the writing phase).
Set the writer to ignore all invalid features and continue.
Successfully wrote ALL features to a file GDB.

There is no error message in the FME, but a general error message pops up (shown below) which ChatGPT says is a "heap corruption error" and I have no idea how to fix it. Any suggestions?

Re: FME Heap Corruption Error when using Esri ArcSDE writer

VinceAngelo — Mon, 08 Apr 2024 16:41:16 GMT

Is this the FME extension to ArcGIS or FME which is crashing? Esri doesn't support FME software.

Does it work with a file geodatabase target (which you could then process with less complexity)?

- V

Re: FME Heap Corruption Error when using Esri ArcSDE writer

AngelaSchirck — Mon, 08 Apr 2024 17:27:31 GMT

It's FME that's crashing, but ONLY when using Esri ArcSDE writer. I can write to a file geodatabase using the Esri geodatabase writer with no issues. I tried writing to that then reading from the GDB and writing to the SDE. Still the same issue. It's clearly a problem that exists only with the SDE Writer (AGOL writer works fine, GDB writer works fine).

Re: FME Heap Corruption Error when using Esri ArcSDE writer

VinceAngelo — Mon, 08 Apr 2024 18:46:03 GMT

Okay, it could be the Esri DLL that's at fault, but it's more likely the FME code. The best way to explore this is with FME tech support (Safe Software -- https://support.safe.com/knowledgesubmitcase ). It's best to build as small of a test dataset as exhibits the issue (and if size impacts it, be sure to let them know where the cutoff is).

Re: FME Heap Corruption Error when using Esri ArcSDE writer

AngelaSchirck — Mon, 08 Apr 2024 19:07:06 GMT

Thanks, I did work with FME support and we determined that the workflow is not the issue.

Re: FME Heap Corruption Error when using Esri ArcSDE writer

AngelaSchirck — Thu, 30 May 2024 17:48:55 GMT

So, I upgraded to FME 2024 and still had the problem, but after messing around and changing the SDE Writer settings something I changed eventually got it to work. Unfortunately I'm not sure which changed setting did the trick, but weeks of frustration are finally over LOL

Re: FME Heap Corruption Error when using Esri ArcSDE writer

AngelaSchirck — Fri, 31 May 2024 11:01:16 GMT

Yeah, scratch that, it put my parcels in the middle of the Atlantic ocean near Africa instead of south Florida. When I deleted the coordinate system it crashed after 167k again.

Re: FME Heap Corruption Error when using Esri ArcSDE writer

VinceAngelo — Fri, 31 May 2024 14:27:52 GMT

Uh, oh. That's Null Island (wrong coordinate system issue).

Have you tried using Check Geometry and/or Repair Geometry on the source layer(s)?

- V

Re: FME Heap Corruption Error when using Esri ArcSDE writer

AngelaSchirck — Mon, 03 Jun 2024 13:25:40 GMT

I have done ALL those things and then some.

The EsriProjector transformer was used to project the coordinates to ESPG:3857, then I chose ESPG:3857 on the SDE writer. When I do this, it writes 167k parcels and crashes. If I choose a different coordinate system on the writer (Say ESPG:4326) it writes ALL 658,315 features, but puts them in the ocean.

Weird thing is it crashes when I choose the correct coordinate system, but works fine when it's not the right coordinate system.