Select to view content in your preferred language

Adjusting Decimal Precision for Latitude and Longitude Calculated in DDM or DMS

2194
5
Jump to solution
07-18-2023 05:59 AM
LiamMurray1
New Contributor III

ArcGIS Pro "Calculate Geometry" returns DDM and DMS coordinates to a rather unnecessary 8 decimal places. Three is typically adequate; 4 might be required for fine detail; 8 is usually only required when mapping individual atoms.

Can the decimal precision be adjusted? There is an option to change the number of decimal places under Options > Units > Location Units; however, this seems to change only display coordinates, and does nothing to the number of decimals calculated through "Calculate Geometry".

This may seem like a minor issue; however, it causes a huge amount of problems. Certain stakeholders insist on coordinates shown to 3DP. DDM and DMS fields are strings, which makes and automated change to precision very difficult. Manually rounding every coordinate to 3DP, when there are potentially hundreds, is a non-starter. 

Any help would be appreciated.

UPDATE following resolution: this is more of a workaround than a resolution. Pro will continue to calculate to a wholly unnecessary level of precision; however, the solution below is a useful alternative to using "calculate geometry".

2 Solutions

Accepted Solutions
martineznicolasx
New Contributor

Not sure if you're still having this issue but in case you are, I figured out a workaround. I was able to perform a "Calculate Field" operation to just grab the part of the string I wanted after I ran the "Calculate Geometry" tool. My expression got the the first 14 or 15 characters of the coordinate string (equivalent to 3DP) and then added the direction back into it afterwards. It's a pretty rough way of doing things, but it's a lot faster than manually rounding everything.

For DMS Coordinates, I used the following two commands with the Python 3 expression type:

Lat= !Lat![0:14]+'" N'

Long= !Long![0:15]+'" W'

Hopefully this helps!

View solution in original post

DavidQuinonesWYO
New Contributor

Exactly what I was looking for. Thanks! 

View solution in original post

0 Kudos
5 Replies
SLouq
by MVP Regular Contributor
MVP Regular Contributor

Why can't you just make your Lat/Long fields Double instead of String? Or does it have to be String?

0 Kudos
LiamMurray1
New Contributor III

When dealing with stakeholders, we often need to provide in DMS or DDM. So they do have to be in string format.

0 Kudos
martineznicolasx
New Contributor

Not sure if you're still having this issue but in case you are, I figured out a workaround. I was able to perform a "Calculate Field" operation to just grab the part of the string I wanted after I ran the "Calculate Geometry" tool. My expression got the the first 14 or 15 characters of the coordinate string (equivalent to 3DP) and then added the direction back into it afterwards. It's a pretty rough way of doing things, but it's a lot faster than manually rounding everything.

For DMS Coordinates, I used the following two commands with the Python 3 expression type:

Lat= !Lat![0:14]+'" N'

Long= !Long![0:15]+'" W'

Hopefully this helps!

DavidQuinonesWYO
New Contributor

Exactly what I was looking for. Thanks! 

0 Kudos
Abigail_K
New Contributor

I was having a similar issue with a dataset of my own, so I wrote a Python expression for use in the Field Calculator that should work whether the cardinal direction letter is before or after the DMS values. My code was for rounding latitude seconds to two decimal places, but it can easily be adjusted to work for longitude by changing the input variable parameter and decimal places can be adjusted by changing the last number in the code block line that calculates for the variable 'return_str'.

So in the first calculation box enter:

rounddms(!LAT_DMS!)

 

and in the main code block paste:

def rounddms(lat_str):
    sec_loc = lat_str.find('"')
    min_loc = lat_str.find("'")
    rounded_val = str(round(float(lat_str[(min_loc+1):(sec_loc-1)]),2))
    return_str = lat_str[:(min_loc+1)]+" "+rounded_val+lat_str[sec_loc:]
    return return_str