Significant digits

162
2
Jump to solution
05-22-2024 07:06 AM
RobertBorchert
Frequent Contributor III

I have a survey where the user is reporting inches of rain at a construction site after a rain event.

if the user puts in .75 it rounds up to .8 when the survey is sent.

In Pro the attribute is set to Double with a precision of 7 and a scale of 1

Is it the scale that is causing the rounding.

In the survey form the binding field is empty

 

 

 

0 Kudos
1 Solution

Accepted Solutions
RobertBorchert
Frequent Contributor III

I figured it out.  Pro does the rounding and it is not an issue with Pro it is a built in function that I have not utilized before.

When you create an Attribute for a Feature and make it a number (double) you have 3 fields that can control the formatting of the field.  Under "Number Format" 

RobertBorchert_0-1716404321804.png

 

You then have Precision and Scale. Precision controls how many Integers are in the number and Scale determines how many decimal points will be save. You can see by the below image that I have the Scale set to 1.  if it is changed to 2 it will round too 2 decimals. 

RobertBorchert_1-1716404429339.png

 

View solution in original post

2 Replies
abureaux
MVP Regular Contributor

This is likely an issue with Pro. S123 doesn't round like that without a parameter being set in Connect.

0 Kudos
RobertBorchert
Frequent Contributor III

I figured it out.  Pro does the rounding and it is not an issue with Pro it is a built in function that I have not utilized before.

When you create an Attribute for a Feature and make it a number (double) you have 3 fields that can control the formatting of the field.  Under "Number Format" 

RobertBorchert_0-1716404321804.png

 

You then have Precision and Scale. Precision controls how many Integers are in the number and Scale determines how many decimal points will be save. You can see by the below image that I have the Scale set to 1.  if it is changed to 2 it will round too 2 decimals. 

RobertBorchert_1-1716404429339.png