I've had good experiences with leveraging an SHA-1 hash of the non-key important data*, storing the hashes with the actual primary key in a parallel table. Due to the nature of the data, I've sometimes converted the data to only a few significant digits, then hashed that (to enable "fuzzy matching", when 3.71 -> 3.72 or 3.68 is not significant, but 8.23 is). Recently I've stored the geometry and non-geometry hashes separately, and have even been able to match rows without a useful rowid key by inverting the hash dictionary spanning tens of millions of features, so that hashes could resolve feature matching at 85-97% accuracy.
Unfortunately, I can't release my code without approval. And in order to make this work , you need to
be really comfortable with endian integer and floating-pint storage representations. I can say the core
code is a set of encoder functions (one for each supported type), which is stored as a bytearray sequence, that is fed into a core Python hashing function. Use of SHA-1 can generate safety warnings, but using it
for hashing (not cryptographic) purposes is perfectly safe.
- V
---
*Unimportant data would include a URL that incorporates the primary key and another column, which could be recomputed (or changed at will)
