ArcSDE Java Client - Different geometries with different Service Pack versions

946
20
07-13-2011 04:38 AM
NormanMann
New Contributor
Hi,

I am new to the Spatial world, and even newer to the ArcGIS/ESRI world.

I am having an issue with reading geometries back from an Arc database (Oracle, using the LONG RAW column for geometries).

Basically, the software I have written very coarsley, goes through a layer and for each row, gets an identifier and the geometry. This seems to work fine in most cases. This is using service pack 3 of the jsde92_sdk and jpe92_sdk jars.

However a small amount (approx 10%) of the geometry come back incorrectly. In addition, the extent from the shape is similarily incorrect. I can see in the "f" tables, what the extent should be.

Upgrading to service pack 6 (anything later then service pack 3 seems to show this behaviour), the results are the opposite. ie the correct results are incorrect, and the incorrect ones are correct.

For example.


Using Service Pack 3  :

GEOMETRY 1        -5147258.777231589, -1.5492365164206002E7
GEOMETRY 2         527566.067680127, 195430.87323492393

Using Service Pack 6 :

GEOMETRY 1       385129.7801152915, 256278.68402232602
GEOMETRY 2        4.578372853940902E7, 1.2575864698354974E8

All stored as British National Grid. It seems to be that the way the LONG_RAW column is read has changed between these service packs? I am of the understanding the ArcMAP can read the data in a consistent , correct fashion.

Can someone shed some light on how I could get consistent results using 1 service pack?

Any help greatly appreciated.

Thanks
0 Kudos
20 Replies
VinceAngelo
Esri Esteemed Contributor
ArcSDE clients don't read geometries from databases -- they read geometries from ArcSDE servers.

What version of ArcSDE is installed in the database?

What version of ArcSDE is installed in the server?

Your client API should always be equal to or greater than the server.

Your server install should always be equal to the database.

If you use Direct Connect, the client is the server, and should be equal to the database.

Given that the last update to 9.2 occurred two years ago (SP3 goes back nearly four years),
if you're still using 9.2, you really should be running the terminal release (SP6 + Connection
Performance + Memory Allocation + Compress/Concurrent Editors) across the board.

Taking a step backwards, what version of Oracle are you using? [A.B.C.D notation, e.g., "10.2.0.3"]
If using Direct Connect, what version of Oracle is on your client?

The fact that 'C' API applications like ArcGIS read the data successfully indicates a possible
problem with your application (and/or the underlying Java SDK).  What are the coordinate
reference parameters of the layer being queried?  Does your code change the coordinate
reference in any way?  Was the layer upgraded to HIGH precision from a previous BASIC
precision layer at 9.1 or earlier (is there an entry in the secondary coordref property)?

- V
0 Kudos
NormanMann
New Contributor
Vince,

thanks for your reply.

To answer some of your questions :

The version of Oracle we are using 10.2.0.3.0.

I am happy to use which ever service pack version of the SDK which works.

In respect to a problem with the application, SESHape.getAllCoords returns the double values representing the coordinates. From an application level, there is nothing I can do to influence this?

The code doesn't change the coordinate reference parameters in anyway.
The coordinate reference parameters (I think you mean this) are

Coord ref system PROJCS["British_National_Grid",GEOGCS["GCS_OSGB_1936",DATUM["D_OSGB_1936",SPHEROID["Airy_1830",6377563.396,299.3249646]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Transverse_Mercator"],PARAMETER["False_Easting",400000.0],PARAMETER["False_Northing",-100000.0],PARAMETER["Central_Meridian",-2.0],PARAMETER["Scale_Factor",0.999601272],PARAMETER["Latitude_Of_Origin",49.0],UNIT["Meter",1.0]]


There is a secondary srid set in the sde.layers table for the layer in question. This is "26". Not sure what that represents.

I do also believe that the layer was upgraded to HIGH precision at some point in the past.

I do feel there is an issue with the underlying SDK, as the same set of code (using different service pack versions of the JAR files) produce different results. Service pack 3 gets SET A correct and SET B incorrect, and service pack 4 and above gets SET B correct and SET A incorrect.


I am waiting the versions of the database version of ARCSDE and the server version of ARCSDE. I will update when I have that, but if you could provide any more help in the meantime, that would be great.

Thanks again.
EDIT: Just to add I am NOT using direct connection.
0 Kudos
VinceAngelo
Esri Esteemed Contributor
There's actually a ton of things you can do to influence how getAllCoords parses the
geometry, few of them good (and all of them have to do with coordinate reference
override).

I seem to remember a bug where HYBRID coordref (BASIC upgraded to HIGH) layers
were not interpreted correctly by the Java API.  This would produce the behavior
you're seeing (where the the HIGH precision shapes are parsed with the BASIC
coordref at SP3, and vice versa at SP6). 

The easiest solution would be to use a single coordref for all shapes, but the route to
get there depends on whether the layer is versioned, participates in complex topologies,
or has CAD objects associated with any rows.  If the layer is a simple feature class, you
can backup the table, then use 'sdeexport' and then 'sdeimport' with the 'init' option
(all new shapes are always created in the HIGH coordref, and the 'init' option truncates
the table and reloads all features); Tech Support would be able to help steer you if this
is other than a simple feature class.

Client side correction is likely to be clumsy, since you'd have to:
+ Copy the shape to a temporary object
+ Get the envelope
+ Override the coordref on the temp object with the HIGH coordref (via setCoordref)
+ Get the envelope again
+ Compare the envelopes
+ If changed, use the changeCoordref request to transform the temp object into the
HIGH coordref in the original shape for use in the application.
The only good news is that the changeCoordref will just be a transform, not a true
"project", so it won't be as expensive, but it will require you to upgrade the 90%
of BASIC shapes on the fly for use (you could go the other direction, but then the
shapes are losing precision [which isn't always a bad thing]).

You might also see if the 9.3 or 9.3.1 API fixed this issue.

- V
0 Kudos
NormanMann
New Contributor
Hi VInce,

Thanks for that.
I think the client side fix is the only realistic option for me right now.

Is there anyway of easily identifying using metadata, high precision/low precistion shapes?
Are you implying that the coordinate reference system will be different?
It doesn't seem to be. Is that stored in the database somehow against each row of data?

Thankss again.
EDIT - I did try 9.3 versions  and they show the same behaviour as the later 9.2 service packs/
0 Kudos
VinceAngelo
Esri Esteemed Contributor
HYBRID coordinate references were supposed to be transparent (geometries always
interpreted as HIGH precision, even if they were originally stored as BASIC). It is for
this reason that the layer upgrade had constraints on what parameters were used
(the HIGH precision coordinate grid had to align with the BASIC one [evenly divisible]).
This obviated the need to mark shapes as being BASIC or HIGH, and eliminated the
need for such functions in the API. It was a clever design, though it appears the Java
implementation didn't take it into account. 😞

- V
0 Kudos
VinceAngelo
Esri Esteemed Contributor
I took another look at the Java API documentation and SeLayer does NOT include a
getSecondaryCoordref request the way the 'C' API does. It's possible that the request
exists and the doc is just broken, but given the nature of Javadocs, this seems unlikely
(worse, it's not in the 10.0sp1 documentation either). This will make it more difficult to
obtain the BASIC coordref without hard-coding it into your application.

Note that the coordinate reference is *not* the coordinate system which you referred to
earlier. The coordSYS is part of the coordREF, but the coordref also incorporates:
+ The PRECISION
+ X/Y offsets and scale
+ Z offset and scale
+ M offset and scale
+ An authenticated SRID
+ A description
At 9.2 and higher, X/Y, Z, and M also have optional cluster tolerances, and 9.3 introduced
vertical coordinate systems (vertcs). There's an entire White Paper on coordinate management
which should be mandatory reading for anyone who wants to use the 'C' or Java API; I strongly
recommend it to you.

- V
0 Kudos
NormanMann
New Contributor
So the client side workaround would require us to determine if EVERY geometry was "correct"?
I guess I could check that the numbers returned were sensible in terms of a bounding box.

Out of interest, how do the C APIs detemine in what wat they should be read?


I think SeShape.getCoordRef returns a SeCoordinateReference. An SeCoordinateReference has a gePrecision method.
Would this be any use in determing HIGH/BASIC geometries? Or, do I really need the secondary coordinate reference system to determine this?

Will have a read of that white paper tomorrow, but just to summarise it seems to be that there isn't a work around?
The bug in the java api (HYBRID bug) seems never to have been fixed, even in 9.3?

I am happy to use different versions of the service packs (yes, I am that desperate) but really need a way to determine which rows should be read with which version.

Thanks again.
0 Kudos
VinceAngelo
Esri Esteemed Contributor
It's not enough to check if the bounding box is sensible -- you must check if it
is the *same*.

This is not an issue with the 'C' API because all shapes from HYBRID layers
appear to be HIGH precision.

- V
0 Kudos
NormanMann
New Contributor
I meant, if the geomerty for a given row, is within the bounding box it should be.

From the example above
Using Service Pack 3 :

GEOMETRY 1 -5147258.777231589, -1.5492365164206002E7
GEOMETRY 2 527566.067680127, 195430.87323492393

Using Service Pack 6 :

GEOMETRY 1 385129.7801152915, 256278.68402232602
GEOMETRY 2 4.578372853940902E7, 1.2575864698354974E8


It's pretty clear that GEOMETRY 1 in service pack 3 isn't valid in terms of a bounding box, and GEOMETRY 2 in service pack 6 isn't valid in terms of the bounding box.

Couldn't I just discard the ones which don't work with SP3 and read them with SP6? I know this is "dirty".

But what are the options (I don't want to/can't change the data in the database)

Thanks again!
0 Kudos