Floating point results not happening in Python 3

560
4
Jump to solution
12-30-2018 09:18 PM
RogerAllison
New Contributor

I am trying to calculate the length of trail segments (polylines) in miles using the calculate filed tool.  my expression, !shape.geodesicLength@miles!  runs fine.  However I do not get floating point results. I have configured the field for numeric with 2 decimal places but all my results are 12.00, 3.00, 0.00 etc.  What is going on?

0 Kudos
1 Solution

Accepted Solutions
DanPatterson_Retired
MVP Emeritus

Use a double type field, specify both the precision and scale.  It looks like scale is set to 0

View solution in original post

4 Replies
DanPatterson_Retired
MVP Emeritus

Use a double type field, specify both the precision and scale.  It looks like scale is set to 0

RogerAllison
New Contributor

Thanks Dan. That worked obviously. I have forgotten some of what I learned about computing over the years.  Old dog learning new software.

0 Kudos
MichaelVolz
Esteemed Contributor

Where exactly are you making this change in Pro?  Field view of attribute table? Catalog Pane?  Catalog View?

0 Kudos
RogerAllison
New Contributor

Field view of attribute table.  I was trying to calculate trail length in a field designated as "long".  Once I changed the field to "double" the calculation gave the expected results.

0 Kudos