I am attempting to export the contents of a geodatabase table which includes four attributes/columns, and ~ 225 million records. Unfortunatly, (in addition to being very slow, ~ 8 hours), the resulting output file is incomplete/corrupted. I am able to read records in the file up to a point, beyond which the records are empty (though the file is still structured as though the total number of records is present). I have tried exporting the table in smaller pieces, but with the same result. The location in the file at which the records ends varies with each export (the maximum so far is ~64 million, the minimum has been ~28 million). Does anyone know if:
1) There is an inherent limit in the number of records being exported (note, they are being exported to a text file, and are not being loaded into memory).
2) Are there alternate ways of exporting tables of this size?
Hi All, I did not find a resolution to the corruption problem, but I changed to exporting data in smaller sets, this seems to have sped the process up, and avoids the overflow problem (assuming that is what caused the problem). Thanks!
Do you have a 32-bit OS or a 64-bit OS? I seem to recall a file size limit where corruption eventually occurs on a 32-bit system. How large is the final output once the process completes?
Eight hours isn't a particularly long time to complete the export of a quarter-billion rows.
We would need details on how you are going about the export to provide advice on other possibilities. Be sure to include the exact version of ArcGIS, the exact RDBMS source, and the code you are using for export.
Hi All, I did not find a resolution to the corruption problem, but I changed to exporting data in smaller sets, this seems to have sped the process up, and avoids the overflow problem (assuming that is what caused the problem). Thanks!
As an update (this is an old thread, but in case it's ever useful): Subsetting the process by breaking it up did seem to resolve the issue, so I am assuming at this point that the problem was in the architecture of the operating system (i.e. exceeding memory or the limits of a 64-bit system).
