topic Re: X/Y coordinates in pandas dataframe to map in ArcGIS API for Python Questions

X/Y coordinates in pandas dataframe to map

DavidRalston — Tue, 25 Jul 2017 14:20:33 GMT

Is there a way to add x/y coordinate columns in a pandas dataframe to a map using the arcgis python api in jupyter notebook?

Re: X/Y coordinates in pandas dataframe to map

RohitSingh2 — Sun, 12 Dec 2021 06:30:59 GMT

Yes, an example is below:

locations = []
path = r'C:\xc\Presentations\GeoPython\Watson\insulators'
# find locations of broken insulators
for file in listdir(path):
    filepath = path + '\\' + file
    if is_broken(filepath):
        locations.append(get_location(filepath))    

# import into ArcGIS as a layer
df = pd.DataFrame.from_records(locations)
df.columns = ['x', 'y']
broken_insulators = gis.content.import_data(df)

m.add_layer(broken_insulators)

I have defined get_locations to return longitude, latitude:

def get_location(filename):    #...         return lon, lat

Re: X/Y coordinates in pandas dataframe to map

DavidRalston — Sun, 12 Dec 2021 06:31:01 GMT

Awesome! My code looks like this:

gis = GIS()
data[['Longitude_X', 'Latitude_Y']]
layer = gis.content.import_data(x)
print (layer)
map = gis.map('UK')
map.add_layer('layer')
map‍‍‍‍‍‍‍

data is a pandas dataframe

<FeatureCollection>

The layer is a feature collection, but the lat long data doesn't show on the map. Possible reasons why?

Re: X/Y coordinates in pandas dataframe to map

RohitSingh2 — Sun, 12 Dec 2021 16:50:25 GMT

Try setting the columns of your dataframe to x and y. The following works:

from arcgis.gis import *
import pandas as pd

gis = GIS()
m = gis.map('UK')

locations = [(52.47867, -1.90848), 
 (51.50642, -0.12721),
 (53.79480, -1.54653),
 (53.47959, -2.24874)]

df = pd.DataFrame.from_records(locations)
df.columns = ['y', 'x']
cities = gis.content.import_data(df)
m.add_layer(cities)
m

I get the following output:

Re: X/Y coordinates in pandas dataframe to map

IonutAlixandroae4 — Thu, 27 Jul 2017 14:27:02 GMT

Hi David,

I also had this problem in another topic on this group. The solution was the way Rohid said, but with coordinates in DD format. I had the list of coordinates in DMS format and it wasn`t working for me.

See here: https://community.esri.com/thread/198787-display-array-of-multiple-coordinates-xy-in-the-map-object

Thanks,

Ionut

Re: X/Y coordinates in pandas dataframe to map

GregoryBrunner — Thu, 27 Jul 2017 16:30:05 GMT

rsingh-esristaff‌

How does this work it my data is in a different coordinate system (i.e. not WGS 1984)?

Re: X/Y coordinates in pandas dataframe to map

RohitSingh2 — Thu, 27 Jul 2017 17:50:14 GMT

The layer/FeatureCollection to be added should have a correct spatial reference set in that case. I don't have the code offhand to do this though. Another way would be to use arcgis.geometry.project to project the coordinates to WGS1984.

Re: X/Y coordinates in pandas dataframe to map

ManushiMajumdar — Tue, 29 Aug 2017 13:48:08 GMT

Is there a way to do this with a df that has more than 1000 rows? This script throws an error for that case..

Re: X/Y coordinates in pandas dataframe to map

IonutAlixandroae4 — Tue, 29 Aug 2017 14:15:17 GMT

If it`s something that deals with memory you should give it a try with increasing the iopub_data_rate_limit; i had this kind of problem with viewing more that a few pictures.

iopub rate limits are too low by default, for visualization-heavy projects · Issue #2287 · jupyter/notebook · GitHub

jupyter notebook --NotebookApp.iopub_data_rate_limit=10000000000

Ionut

Re: X/Y coordinates in pandas dataframe to map

ManushiMajumdar — Tue, 29 Aug 2017 14:27:43 GMT

Thanks Ionut!
It still gives me the error (Error Code: 401): UnAuthorized

Re: X/Y coordinates in pandas dataframe to map

RohitSingh2 — Tue, 29 Aug 2017 16:51:38 GMT

If there are more than 1000 rows, you should convert it to a SpatialDataFrame. You can do it like this:

df['SHAPE'] = df.apply(lambda row : arcgis.geometry.Geometry({'x': row['x'], 'y': row['y']}), axis=1 )

sdf = SpatialDataFrame(df)

layer = gis.content.import_data(sdf, title='My Data')

m.add_layer(layer) # add to map

find_nearest(layer, ...) # use for analysis

Re: X/Y coordinates in pandas dataframe to map

ManushiMajumdar — Tue, 29 Aug 2017 18:04:06 GMT

Thanks Rohit!
The exact same code snippet is giving me an error -

RuntimeError: Service name 'a' already exists for 'bkrWlSKcjUDFDtgw'

I don't know how to debug it since I don't have services with that name/id..

Re: X/Y coordinates in pandas dataframe to map

RohitSingh2 — Tue, 29 Aug 2017 18:26:54 GMT

There's a bug in import_data that creates a service by the name 'a' the first time it runs... the fix will be in the next update, but as a workaround you should be able to search for that service and delete it each time you're calling import_data with a spatial dataframe.