Select to view content in your preferred language

FME Heap Corruption Error when using Esri ArcSDE writer

384
8
04-08-2024 05:01 AM
AngelaSchirck
Occasional Contributor III

Hi all,

Here's the scenerio.  I have Three tables in an Oracle database that I am combining using FME.  I then am using An SDE Writer to write to a different Oracle database (the oracle writer put the data in a format that I cannot use with Enterprise 10.9.1, so I chose the SDE writer instead).  

There are close to 700k features that need to be written, but the FME crashes and the program closes after writing about 160k features EVERY.SINGLE.TIME. 

Things I've tried: 

  • Redirected the output to the data inspector.  The work flow completes without errors.
  • Added a geometry validator and repaired bad geometries and ignored those that couldn't be repaired (none). 
  • Added a feature keeper to collect all the features and write them after join is complete (all ~700k features are collected, still crashes in the writing phase).
  • Set the writer to ignore all invalid features and continue.
  • Successfully wrote ALL features to a file GDB.

There is no error message in the FME, but a general error message pops up (shown below) which ChatGPT says is a "heap corruption error" and I have no idea how to fix it.  Any suggestions?

AngelaSchirck_0-1712577650068.png

 

Tags (3)
0 Kudos
8 Replies
VinceAngelo
Esri Esteemed Contributor

Is this the FME extension to ArcGIS or FME which is crashing? Esri doesn't support FME software.

Does it work with a file geodatabase target (which you could then process with less complexity)?

- V

0 Kudos
AngelaSchirck
Occasional Contributor III

It's FME that's crashing, but ONLY when using Esri ArcSDE writer.  I can write to a file geodatabase using the Esri geodatabase writer with no issues.  I tried writing to that then reading from the GDB and writing to the SDE.  Still the same issue.  It's clearly a problem that exists only with the SDE Writer (AGOL writer works fine, GDB writer works fine).

0 Kudos
VinceAngelo
Esri Esteemed Contributor

Okay, it could be the Esri DLL that's at fault, but it's more likely the FME code. The best way to explore this is with FME tech support (Safe Software -- https://support.safe.com/knowledgesubmitcase ). It's best to build as small of a test dataset as exhibits the issue (and if size impacts it, be sure to let them know where the cutoff is).

AngelaSchirck
Occasional Contributor III

Thanks, I did work with FME support and we determined that the workflow is not the issue.

0 Kudos
AngelaSchirck
Occasional Contributor III

So, I upgraded to FME 2024 and still had the problem, but after messing around and changing the SDE Writer settings something I changed eventually got it to work.  Unfortunately I'm not sure which changed setting did the trick, but weeks of frustration are finally over LOL

0 Kudos
AngelaSchirck
Occasional Contributor III

Yeah, scratch that, it put my parcels in the middle of the Atlantic ocean near Africa instead of south Florida.  When I deleted the coordinate system it crashed after 167k again.

0 Kudos
VinceAngelo
Esri Esteemed Contributor

Uh, oh. That's Null Island (wrong coordinate system issue).

Have you tried using Check Geometry and/or Repair Geometry on the source layer(s)?

- V

0 Kudos
AngelaSchirck
Occasional Contributor III

I have done ALL those things and then some.

The EsriProjector transformer was used to project the coordinates to ESPG:3857, then I chose ESPG:3857 on the SDE writer.  When I do this, it writes 167k parcels and crashes.  If I choose a different coordinate system on the writer (Say ESPG:4326) it writes ALL 658,315 features, but puts them in the ocean. 

Weird thing is it crashes when I choose the correct coordinate system, but works fine when it's not the right coordinate system.

0 Kudos