formatting numbers from an array of numbers - classbreaksrenderer

[TracySchloss]

I'm working on dynamically finding the classbreaks from a user selected field.  The data is all numeric.  Some fields contain whole numbers, some contain numbers with decimals.  For the columns that have numbers with decimal places, these are not well formatted. 

I'm executing a querytask and in the results function I'm capturing the values in the selected field in an array, which I'm using to figure the class break intervals.

Here's a sample of the values that might be returned:
0,0.09,0.13,0.25,0.31,0.43,0.94,1.22,1.34,1.74,1.81,1.91,1.91,2.1,2.15,2.39,2.5,2.93,3.14,3.21,3.44,3.65,3.72,3.77,3.84,3.91,4.14,4.19,4.25,4.29,4.3,4.44,4.46,4.61,4.61,4.62,4.73,4.79,4.87,4.92,4.99,5,5.0200000000000005,5.03,5.08,5.1000000000000005,5.21,5.22,5.24,5.3100000000000005,5.48,5.49,5.67,5.69,5.71,5.75,5.78,6.16,6.17,6.19,6.42,6.46,6.48,6.53,6.54,6.65,6.94,6.95,7.140000000000001,7.22,7.29,7.33,7.39,7.76,7.77,7.87,7.89,8.01,8.08,8.31,8.36,8.42,8.53,8.71,8.82,8.9,8.94,9.09,9.290000000000001,9.31,9.59,9.63,9.72,9.85,10.1,10.16,10.17,10.3,10.43,10.59,10.71,10.8,11.05,11.41,11.62,12.03,12.75,13.07,13.77,13.86,15.11,15.61,17.26,17.990000000000002,19.59

One, I'm not sure why some of my original values are returned with those very long decimal places.  I'm wanting to use a number format to change these few poorly formatted numbers to match the rest.

I created a number formatter as

<mx:NumberFormatter id="pctBreakFormatter" precision="2" useNegativeSign="true" />

I'm selecting the numbers from the Array as

var intervalBreak = Math.round(myArray.length / myInterval); var interval:int = 5;  for (var i:Number = 1 i < interval + 1); i++) { var dataValue:Number = myArray[(intervalBreak * i) -1]; }

So I'm defining dataValue as a number and then I ought to be able to apply the number format so that the numbers that have extra decimals match.

pctBreakFormatter.format(dataValue);

I don't see any changes to dataValue.  If the number from the array selected was 5.1000000000000000005, it still looks that way after I apply the formatter.  If I try to set a new variable:

var newDataValue:Number = pctBreakFormatter.format(dataValue);

Then I get an error 1067: Implicit coercion of a value of type String to an unrelated type Number.

I don't understand why.  At one point does it think that dataValue is a String, not a number?  It doesn't work to force it back to a number using 'as Number' either.  If I try that, then the variable ends up with the value of "0".

It might be OK to have my classbreaks have numbers with very long decimal places, but I'm also using these numbers to create a legend and I can't leave them this way.

[RobertScheitlin__GISP]

Tracy,

    The  NumberFormatter format method returns a string.

http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/mx/formatters/NumberFormatter.htm...

What you need to do is truncate the number as you would probably not want it to round up or down. You can do this by treating the number as a string and using subString on it and then turn it back to a number.

Don't forget to click the top arrow (promote) and to click the Mark as answer check as shown below:

[TracySchloss · Accepted solution]

Thanks!  It didn't even occur to me that the formatter would return a string!  I figured if it was a numberformatter, it would be a number.  I think truncating would work in this situation since I have such a string of 000000s on some of these. 

Tracy