What is the best way to serve a layer from my REST server?

[Anonymous User]

I'm trying out ArcGIS online and after my learning, I'm able to create a map and a dashboard.
I would like to create a layer for the map that comes from our public REST server. 

I first created a 2 line Excel file that I saved to CSV format.  I chose upload from My Computer and the file created a dot on my map at the coords that I gave it from the csv file. Perfect.

I changed the file on my disk, and updated the file manually, works too.

Next I took that same file and set it up on my server.  I can connect to it fine from Chrome and download it.

When I try and get the file from ArcGIS Online it as a csv url, it came back with an error:
"The CSV, http://myserverurl.com/api/testfile, is not accessible and cannot be added to the map."

This tells me noting...

I checked the url headers, but I don't really know what the map expects.  This is a simple Excel file saved to my server with 1 record: name, latitude,longitude, and a second line with 3 values.

Can you please point me in the right direction for what I'd like to do?
The data needs to update every day automatically, so I thought a web rest service would be best.
I can also send it as a json if necessary.

What's the correct way to for ArcGIS Online to consume a layer that the data updates every day, without any manual intervention.

Thank you

[MichaelVolz]

not sure if this is the best way.

Create a mapservice on AGS with your current data saved in a point feature class that is registered with AGS.

Create a python script that will copy data from a source processing location to the location registered with AGS using a truncate and append method.

In a model that is called by the above python script, create a point layer using the XY Event Layer.

The python script will then overwrite the current data using truncate and append so the mapservice does not need to be stopped.

Have the script scheduled to run with a Windows Scheduled at whatever data refresh interval you need (daily, hourly, etc.)

[Anonymous User]

Thank you Michael for the reply...

Is a Python script the only way?  I'm not sure what to do with the script. I figured if the layer's data lived on my SQL Server, and I passed that data to the map in the form of a rest service then the layer could update itself.
This doesn't have to be a csv file.  Does that give you more information?

And to be clear, by map service, you mean upload the csv file to my Content and publish it as a service?
Then how do I refresh this information once it reaches the server.  Aside from manually uploading another file?

[MichaelVolz]

Is you data on  SQL Server just tabular data with x and y coordinates or is it truly spatially enabled? 

[Anonymous User]

It's just tabular data for now.

Here's a sample of my test map.  I picked a spot on the map in the middle of a city and wanted to place a dot.

Also I did a tutorial on snow plows that auto updated the streets and wondered how they updated the data from a server.

What I'm looking for is a way to dynamically update a table of values possibly from a rest server.  I'm not sure if I have to get an ARC Server or not.

Name	Latitude	Longitude
Needs Traffic Light	40.437788	-80.000211

[VishApte]

When you uploaded your CSV file to ArcGIS Online, ArcGIS Online created to a RESTful web service for that data in Esri's own format. This REST service will not know that the data from your http://myserverurl.com/api/testfileneeds to be re-uploaded. Something has to trigger it. Python program from your own server/workstation would be the least efforts and least cost method. The python program will: 

1. Download CSV/XLS file from http://myserverurl.com/api/testfile

2. Connect to ArcGIS Online using ArcGIS API for Python and reupload the CSV file. Key here is to update existing item in ArcGIS Online so that your Dashboard / Map etc that you have built in ArcGIS Online will continue showing updated point(s). If you upload the CSV as a new item in ArcGIS Online, existing maps/dashboards will not reflect data changes.

If you have ArcGIS Server software, you can serve the data from your SQL Server or similar DBMS. Then python or any simple program can simply update SQL server table with changes from CSV and then the maps/dashboards will reflect the changes.   

Hope this helps.

Cheers,

Vish

[Anonymous User]

Thank you so much Vish.  I'll do some research online and learn how to work with ArcGIS API for Python, and I'll post back as soon as I get something working.  Thanks for the pint in the right direction.  I'll keep this thread updated.

[Anonymous User]

For anyone that needs a "point in the right direction", here is what I found..

I didn't realize that I didn't have to use a csv file to upload the data. I can add points directly from my SQL Server DB by using the REST API.

While I was at the developer's area looking for a Python script I found that they support .NET, so I downloaded and installed the API. You are looking for the ArcGIS Runtime SDK for .NET Visual Studio Extension.

Next I opened Visual Studio and created this tutorial:

Add a layer from an item | ArcGIS for Developers 

You will learn how to build an app that loads and displays a layer from ArcGIS Online.

It didn't seem relevant at the time but I thought it would demonstrate the API.

It first asked me to do another tutorial first to create a feature layer (my points): - This was an important step because it seems like you must first create the layer in ArcGIS Online before doing any sort of edits with the API. I could be wrong about this, as I remember seeing var layer = new FeatureLayer()

Create a new dataset | ArcGIS for Developers 

With the last tutorial I was able to display a map in Visual Studio, excellent!

Next was looking to see if I could add a point outside of ArcGIS online.

Before you can access the layer if it is not public, you must have an access token.  Here's how to get one using Postman:

Get an access token | ArcGIS for Developers 

By the way, Postman gives you the C# code too, so you may add this to your REST Client.

Next was the tutorial I was waiting for:
Add, edit, and remove features - (this was a huge leap forward) I had no idea I was looking for a feature Layer.

Add, edit, and remove features | ArcGIS for Developers 

Add, edit, and remove features from ArcGIS Feature Services.

Documentation: Features and Geometries | ArcGIS for Developers 

This tutorial teaches you how to add a point to al layer, edit it and delete it.  Note that this layer is viewable in both the ArcGIS online version and the Visual Studio API version.

I'm currently working on making more then one add/edit/delete in a single call:
Apply Edits (Feature Service/Layer)—ArcGIS REST API: Services Directory | ArcGIS for Developers 

This operation adds, updates, and deletes features to the associated feature layer or table in a single call (POST only).
The applyEdits operation is performed on a feature service layer resource.

I will try and keep this thread updated, and thank you all again for all your help.

~G