Random corruption of pdf files when saved to a BLOB field in Oracle in ArcMap 10.2.1

[TylerJones3]

Summary:

I am encountering random corruptions of PDF files when they are saved to a BLOB field in ArcMap 10.2.1 and then reconciled and posted to an Oracle database. The corrupted files can be opened after the save occurs, but not after the reconcile and post. Adobe Acrobat states that there is an error and the file can't be read. The file's size is also smaller, for example a 125 KB file is only 60 KB after the reconcile and post process.

Details:

The corruption occurs at random. Initially we believed it to be tied to ESRI's standard Attachment Manager and possibly the Attachment Manager ArcObjects as well.  Our client found though that the corruption also occurs when just using an IRow.Store to save the PDFs to the BLOB field--as long as the reconcile and post occurs it does not appear to matter how the value is stored initially. The problem was discovered when a client was using a custom Attachment Manager that I had written using the IAttachmentManager and ITableAttachments ArcObjects. After further testing, we were able to identify these traits:

1. PDFs are affected, image files (JPG, PNG, etc) are not.

2. The corruption does not occur in file geodatabases (not surprising since there is no reconcile and post)

3. The corruption is apparently random. We identified 5 "problem" PDFs that would reliably corrupt after every reconcile and post. These PDFs were used for two days for testing and on the third day they no longer suffered corruption issues and could be reconciled and posted with no issue.

4. The act of storing the value and then saving the edits results in a complete PDF. I checked the file size before and after it was written to the BLOB field and the size remained the same. Only after the reconcile and post did the file size shrink.

Has anyone else encountered this or does anyone have a possible solution?

[TylerJones3]

I'll answer this one myself so that my name will not be cursed at some future point in time when someone has this exact problem, finds this post and discovers only the question and no answer.

First, I tested further and discovered it was not just PDFs, but any sufficiently large file could be corrupted (59 KB appears to be the threshold).

It turns out our client's Oracle database still used LONG RAW to store BLOB files instead of a true BLOB field. LONG RAW has since been deprecated in favor of BLOB, but given the age of this Oracle database, it was never re-configured to use the true BLOB field type.

The fun part is that ArcCatalog and ArcMap will happily report the field type is BLOB even when it is actually not. To check this you have to view the Oracle DBTUNE table to get the truth. ArcCatalog will say BLOB no matter what. Our best guess is that there is something strange with how LONG RAW stores data and there is some sort of break at 60 KB that causes the data to not store correctly if the data object is larger than that.

If you find that you have LONG RAW in your database, you can change it to BLOB, but there is a catch: Only new featureclasses and tables will make use of the newer BLOB type field; existing tables will not change.

In our case the problem was solved by accessing the DBTUNE table and changing the NETWORK_DEFAULTS keyword config string from LONG RAW to BLOB.

[VinceAngelo]

Couple of points:

1. The SE_BLOB_TYPE of ArcSDE covers all binary large object types (of all databases).  The choice of which BLOB type is left to a DBTUNE parameter (ATTRIBUTE_BINARY) set by the user.
2. There are a number of DBTUNE parameters which pertain to BLOB storage; you should make sure they're all set appropriately.
3. It's possible to migrate LONG RAW storage to BLOB, but changing the DBTUNE only impacts newly created tables (and only tables created through ArcGIS tools).
4. There have been times where BLOB types had overwrite issues, though I don't think it was as recent as 10.2.1