Large Dictionary Compression?

Discussion created by csny490 on May 23, 2012
Latest reply on Feb 22, 2013 by csny490
I have a simple dictionary like this:

exampleDict[123444556] = (1785,2234544,3545456, 165765.47654)

where all the keys are intgers and the values are either integers or floats.

My issue is that I have the need to store/access about 20 million keys at a time, and I am running out of 32-bit memory. I'd rather do this in 32-bit Python as I need (or would like) access to arcpy for its FGDB table reading/writting abilities.

Anyone know of a way to somehow "compress" keys and/or values in a dictionary? I'm looking into the binascii module, and I see lots of methods to compress strings, but not ints or floats. Maybe you can't meaningfully compress these since they are already quite numeric?

Anyone ever do something like this?