ArcSDE C-API SE_stream_calculate_table_statistics() crash with Oracle

[sbrockwell]

I'm maintaing an ArcSDE client application which apparently was working with ArcSDE 9.1

Current configuration: ArcSDE 10.1 with Oracle 10.1, Win 7 x32/x64

Symptoms when calling SE_stream_calculate_table_statistics() with SE_DISTINCT_STATS as mask on a STRING property:

   1. crashes with SE_NET_FAILURE (-10)
   2. memory violation when using direct connect in sdeora11gsrvr101.dll
   3. Works fine with numerial properties (integer/double)
   4. Works fine with SqlServer with any property type (integer/double/string)

Note: 2) and 3) prove the code is correct.

Any information greatly appreciated. Thanks in advance.

[MelitaKennedy]

This may be a red herring, but when you contact tech support, you should also tell them that there was a very, very similar bug that was fixed in version 10.0 but for the SDE Java API. I looked at the fix and the same issue is not apparent to me in the C API code--which is why it might not be the same bug.

Reference NIM040332: Running SeQuery.createTableStatistics with the 'SeTable.SeTableStats.SE_DISTINCT_STATS' option hangs when processing an nvarchar field from an ArcSDE 9.2 SQL Server 2005 database and gives an error when processing an nvarchar2 field from an ArcSDE 9.2 Oracle database.

Melita

[sbrockwell]

This may be a red herring, but when you contact tech support, you should also tell them that there was a very, very similar bug that was fixed in version 10.0 but for the SDE Java API. I looked at the fix and the same issue is not apparent to me in the C API code--which is why it might not be the same bug.

Reference NIM040332: Running SeQuery.createTableStatistics with the 'SeTable.SeTableStats.SE_DISTINCT_STATS' option hangs when processing an nvarchar field from an ArcSDE 9.2 SQL Server 2005 database and gives an error when processing an nvarchar2 field from an ArcSDE 9.2 Oracle database.

Melita

Do you compile with SDE_UNICODE enabled (SE_WCHAR parameters) or not?

Yes.

That leaves you with two options -- Using SE_STRING_TYPE columns

Such change will break for sure cases which currenly work fine. Not an option. My point is the code used to work and we deal with an ArcSDE regression. 

You should certainly contact Tech Support (and have them contact me for sample code to demonstrate the problem).

Not sure I understand. You reproduced the problem, located the issue and I would expect ESRI to fix it. That is, I expect you provide me with a case # and a tentative SP so I can inform my customer. Is this not the case?

Regards.

[MelitaKennedy]

Hi,

Both Vince and I are on the development-side, not the technical support side. It is possible for me to enter a bug into the development bug-tracking software and export it to the public/technical support software, but it's frowned upon. Partially, because I don't have access to the customer information, and don't know the right questions to ask (like DBMS/OS/release/etc) that's useful to have when putting together a bug report. Technical Support will also be able to identify if the issue has been reported before; I can't always do that reliably. It also gets the bug report into the correct queue; neither Vince nor I will be directly involved in triaging, fixing, or testing it and an analyst familiar with the area will have a better idea where to put it.

Vince was able to repro it, I identified a similar bug but in a different API and stated that I couldn't identify the same issue in the C API code, so someone will still have to figure out what the real problem is. I pointed out the other bug because it might help an analyst/developer to compare working code (even in a different language).

Melita

[VinceAngelo]

No, this is not the case. I don't work in Tech Support, and I don't have access to the
tools they have for incident management.  Even if I called them to create an incident,
it would be under my user number, not yours, so you wouldn't receive notifications.
These User Forums are not a replacement for Tech Support.

I would point out that your 9.1 code did not work with SE_NSTRING_TYPE columns,
since SE_NSTRING_TYPE columns weren't supported before ArcGIS 9.2.  The default
SE_NSTRING_TYPE creation of Desktop can be overridden through the use of
the "UNICODE_STRING=FALSE" flag in the DBTUNE file (see the documentation
here and here for more info).

- V

[sbrockwell]

Hi. Our ESRI representative indicated there is no support for ArcSDE 10.0 and 10.1

Could you please confirm?

Thanks.

[MelitaKennedy]

Okay, please do not consider this a binding/legal Esri answer, but going by the ArcGIS for Server Product life cycle document, 10.0 is in mature support while 10.1 is still in extended support. Further information on what that means is available here.

Melita

[MarcoBoeringa]

Okay, please do not consider this a binding/legal Esri answer, but going by the ArcGIS for Server Product life cycle document, 10.0 is in mature support while 10.1 is still in extended support. Further information on what that means is available here.

Melita

Ergo (looking at the document Melita linked): they should help you out in Technical Support and answer your questions, as long as you have a support contract. Even for the "Mature" 10.0 that doesn't get any new service packs or patches.

[VinceAngelo]

The 'C' API is deprecated at 10.2, and won't be released with the next rev.  This adds
some quirks to the base support policy (I found documentation bugs that are no longer
being corrected), but 10.1 SP1 should be supported (as an Extended product).  10.0
is not yet Retired, but it won't be getting QFEs the way 10.1 would.  Try pointing out
the support policy doc to the person with whom you spoke.

The workaround is to write a helper function to do a GROUP BY, ORDER BY query
when the column type is SE_NSTRING_TYPE -- It's probably no more than a
dozen API calls:

• SE_queryinfo_get_tables to get the table name


• SE_table_describe to get all column types


• SE_stream_calculate_table_statistics on non-SE_NSTRING_TYPE columns


• SE_stream_prepare_sql to execute the query with the stream , in the format



SELECT count(colname) cnt,colname FROM tablename GROUP BY colname ORDER BY colname



• SE_stream_execute to start the stream


• se_stream_bind_output_column to a CHAR array long enough to hold the

column data and the CNT value (which may come back LFLOAT)

• SE_stream_fetch for up to the maximum number of distinct values


• SE_stream_close to end the query


• SE_table_free_descriptions to clean up the describe

Also in there is a malloc of enough space for the maximum number of pointers
and a buffer large enough to store all the returned strings, plus the SE_DISTINCT
and SE_VALUE elements.  You'd also need your own cleanup function (to call
SE_table_free_stats or to free your own datatype [which needs a flag to indicate
how the data is stacked]).  In this situation, I'd name the wrapper
se_stream_calculate_table_statistics and the cleanup se_table_free_stats.
If I weren't on deadline for a delivery I'd code the thing myself right now.

- V

[sbrockwell]

Excellent information about 10.0 and 10.1 life cycles.

Vince, I know how to code such query. My problem was not how but why 🙂

Thanks everyone here for the outstanding level of support.

Best regards.