topic Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection in ArcObjects SDK Questions

Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Wed, 08 Nov 2017 14:26:46 GMT

Hello all,

We are using arcsde connect string to insert data from shape files into Oracle geodatabase using direct connection strings.

Everything seemed well with test data. When full data set was used to import into oracle, after inserting few thousand records application/script silently dies without throwing any error. It only happens in that environment using sde connections.

Could someone help in this scenario. It is very straight forward where we use feature cursors to read from shape files and insert data.

Regards,

Umasurya

https://community.esri.com/community/gis/managing-data?sr=search&searchId=67836751-85bf-4434-9361-c37c4b77d5ab&searchIndex=0‌ https://community.esri.com/groups/geodatabase?sr=search&searchId=bc31c01f-baa5-40d5-aaa8-ca12052c18a6&searchIndex=0‌

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

rgomes — Thu, 09 Nov 2017 11:03:00 GMT

Try to run 'Repair Geometry' before import data or split the shapefile into smaller files.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Thu, 09 Nov 2017 11:35:46 GMT

Hi, Thank you for the response.

Source data is absolutely fine. If I run any of the shape file individually, it works correctly. Application dies only after importing few 100s(shape files) if bulk number of shape files in batch put for import into oracle sde. Ironically it won't throw any error, application simply gets killed.

Regards,

Umasurya

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

rgomes — Thu, 09 Nov 2017 11:41:41 GMT

Are you using ArcGIS Desktop or ArcGIS Pro or just a Python script with command line?
If you are using ArcGIS Desktop just disable Background Geoprocessing in ArcMap Options > Georpcessing Options.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Thu, 09 Nov 2017 11:50:08 GMT

Thank you. I am using arcobjects with arcgis engine version10.1.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Thu, 09 Nov 2017 12:35:46 GMT

Hi, Could you suggest if any other options that I can try?

Regards,

Umasurya

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

rgomes — Thu, 09 Nov 2017 12:42:28 GMT

Could you show me the source code of this process to think another option?

Best Regards,
Rafael Gomes

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Thu, 09 Nov 2017 13:06:50 GMT

Code is very simple. Below are high level steps

Browse through the folders

Loop feature cursor for the shape file feature classes.

Insert features into SDE feature classes

Flush the sde feature class.

No CPU hike, no memory consumtion. It is very simple. But after few iterations, application silently dies with no exception. Possible error with arcsde with oracle.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

rgomes — Thu, 09 Nov 2017 13:20:26 GMT

Try to grant unlimited tablespace to the user and set quota unlimited. Is a common problem with Oracle with this kind of situation but i don't know why not showing any exception or error.

If doesn't work, try to trigger the import process inside a try/catch block to get us any tips about the source of error.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Thu, 09 Nov 2017 14:50:20 GMT

Hi, Thank you for suggestion. All kind of exceptions are being handled. I am sure it got plenty of tablespace assigned. It bypasses everything and silently crashes. Something really weird. Is it something to do with Oracle is being in VM? that block of code won't throw any exception, simply lets the application leaves the execution.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

rgomes — Thu, 09 Nov 2017 15:20:43 GMT

I don't know problems regarding Oracle using VM. But, I have a standalone code to import data too, created at the version 10.2 that just only works if run in compatibility mode. It's very similar problem. Have you tried to run using compatibility mode at the client side?

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Fri, 10 Nov 2017 04:49:32 GMT

In my case, it is all with 10.1. Do I still need to run using compatibility mode. It only fails after many shape files import

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

rgomes — Fri, 10 Nov 2017 10:28:34 GMT

I have another Idea .... you can call a python function inside your code like a standalone code just to import the shapefiles.
Using python may be better in this case.
See the link below (is for the most recent arcgis version but I did the same thing in the previous versions).
http://desktop.arcgis.com/en/arcobjects/latest/net/webframe.htm#aecfa7f0-ac98-4a1c-ae36-f012211efb3c.htm

So you can create a Python to import data and call like a command line to execute the import process.

Can you try to do this?

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

TinaMorgan1 — Mon, 13 Nov 2017 23:38:17 GMT

Have you taken an Oracle level 12 trace? This can help us see exactly what calls are being sent to and from the database when the failure occurs.

After you see the failure, do you see ANY data that was imported?

How many shapefiles are being imported? If it is more than one, can you add a print statement in your code so you can monitor which files the code is able to parse through?

Here is the python for Feature Class to Feature class (conversion tools)

import arcpy arcpy.env.workspace = "C:/data/GreenvalleyDB.mdb/Public Buildings" arcpy.FeatureClassToFeatureClass_conversion("buildings_point", "C:/output/output.gdb", "buildings_point")

Feature Class To Feature Class—Help | ArcGIS for Desktop

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Fri, 17 Nov 2017 14:38:24 GMT

Thank you. I am trying python option and get back to you how the result appears.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Fri, 17 Nov 2017 14:39:57 GMT

We checked on the trace and nothing unusual appears. It just logged the sequential operations.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

VinceAngelo — Fri, 17 Nov 2017 15:16:08 GMT

ArcGIS 10.1 is rather old. Have you applied the SP1 service pack and a score or more of patches?

What exact version of Oracle are you using? Is it listed as supported by 10.1sp1?

Back when it was current, I used 10.1sp1 to load ten and hundreds of millions of features at one time (680m being the most). I must admit it's easier to do so now at 10.5.1, but there's no obvious reason why you should be having this issue. I suspect it's going to come down to a bizarre IT issue, like a router or firewall limiting traffic, or an antivirus app freaking out in some way.

- V

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Fri, 17 Nov 2017 17:13:07 GMT

Thank you for the suggestion on version and reviewing the version changes or contemplating on a different source data format so we do not need to rely on any API. But I am afraid to say how it is important which version the software is for completing the simple importing of data from shape file workspace into arcsde with Oracle.

It was working in local servers. But this occurs in data center environment and no way to trace any error.

ArcGIS version is 10.1

Oracle version is 11g.

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

VinceAngelo — Fri, 17 Nov 2017 17:25:25 GMT

The *EXACT* version of Oracle goes out to five places (11.a.b.c.d).

Given the extreme age of ArcGIS 10.1, it is extremely risky to not have installed at least Service Pack 1. There are dozens of patches on top of SP1, and they ALL should be applied before reporting uncommon behavior.

- V

Re: Issue in Importing shape file data into into Oracle geodatabase using sde connection

umasuryasykam1 — Fri, 17 Nov 2017 17:36:18 GMT

Ok. I will get back to you on the exact details on the s/w spec that we have at data center. Thank you for the suggestion.