nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

CSV to feature class

jaykapalczynski

I have a csv file in a folder location that gets updated every hour.

I need to take this info and update a Feature Class that is being published to a feature service.

Looking for the smartest and most effective means in which to do this.

Stored Procedure and then to FC? Any ideas would be appreciated.

Find more posts tagged with

Accepted answers

All comments

AZ911GIS

This is a script that created in Jupyter Notebook and exported as a Python Script that runs on a server once a day. It parses a website that builds a nice data frame/CSV that then appends to a point feature service. Line 134 should work for your workflow. Hope that helps.

<SPAN class="comment token">#!/usr/bin/env python</SPAN>
<SPAN class="comment token"># coding: utf-8</SPAN>

<SPAN class="comment token"># In[42]:</SPAN>


<SPAN class="keyword token">import</SPAN> bs4 <SPAN class="keyword token">as</SPAN> bs
<SPAN class="keyword token">import</SPAN> urllib<SPAN class="punctuation token">.</SPAN>request
<SPAN class="keyword token">import</SPAN> pandas <SPAN class="keyword token">as</SPAN> pd
<SPAN class="keyword token">import</SPAN> numpy <SPAN class="keyword token">as</SPAN> np
<SPAN class="keyword token">from</SPAN> arcgis<SPAN class="punctuation token">.</SPAN>gis <SPAN class="keyword token">import</SPAN> GIS
<SPAN class="keyword token">from</SPAN> arcgis <SPAN class="keyword token">import</SPAN> features
<SPAN class="keyword token">from</SPAN> copy <SPAN class="keyword token">import</SPAN> deepcopy
<SPAN class="keyword token">import</SPAN> arcpy


<SPAN class="comment token"># In[43]:</SPAN>


source <SPAN class="operator token">=</SPAN> urllib<SPAN class="punctuation token">.</SPAN>request<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'https://smoke.azdeq.gov/'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>read<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
soup <SPAN class="operator token">=</SPAN> bs<SPAN class="punctuation token">.</SPAN>BeautifulSoup<SPAN class="punctuation token">(</SPAN>source<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'lxml'</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[44]:</SPAN>


table <SPAN class="operator token">=</SPAN> soup<SPAN class="punctuation token">.</SPAN>table


<SPAN class="comment token"># In[45]:</SPAN>


table <SPAN class="operator token">=</SPAN> soup<SPAN class="punctuation token">.</SPAN>find<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'table'</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[46]:</SPAN>


table_rows <SPAN class="operator token">=</SPAN> table<SPAN class="punctuation token">.</SPAN>find_all<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'tr'</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[47]:</SPAN>


<SPAN class="keyword token">for</SPAN> tr <SPAN class="keyword token">in</SPAN> table_rows<SPAN class="punctuation token">:</SPAN>
    td <SPAN class="operator token">=</SPAN> tr<SPAN class="punctuation token">.</SPAN>find_all<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'td'</SPAN><SPAN class="punctuation token">)</SPAN>
    row <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>i<SPAN class="punctuation token">.</SPAN>text <SPAN class="keyword token">for</SPAN> i <SPAN class="keyword token">in</SPAN> td<SPAN class="punctuation token">]</SPAN>
    <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>row<SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[48]:</SPAN>


dfs <SPAN class="operator token">=</SPAN> pd<SPAN class="punctuation token">.</SPAN>read_html<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'https://smoke.azdeq.gov/'</SPAN><SPAN class="punctuation token">,</SPAN>header<SPAN class="operator token">=</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">for</SPAN> df <SPAN class="keyword token">in</SPAN> dfs<SPAN class="punctuation token">:</SPAN>
    <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>df<SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[49]:</SPAN>


df<SPAN class="punctuation token">.</SPAN>head<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[50]:</SPAN>


df<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Location'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> df<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Location'</SPAN><SPAN class="punctuation token">]</SPAN>                          <SPAN class="punctuation token">.</SPAN>astype<SPAN class="punctuation token">(</SPAN>str<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>str<SPAN class="punctuation token">.</SPAN>replace<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">"[\(\)']"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[51]:</SPAN>


lat <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
lon <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>


<SPAN class="comment token"># In[52]:</SPAN>


<SPAN class="comment token"># For each row in a varible,</SPAN>
<SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> df<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Location'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">:</SPAN>
    <SPAN class="comment token"># Try to,</SPAN>
    <SPAN class="keyword token">try</SPAN><SPAN class="punctuation token">:</SPAN>
        <SPAN class="comment token"># Split the row by comma and append</SPAN>
        <SPAN class="comment token"># everything before the comma to lat</SPAN>
        lat<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>row<SPAN class="punctuation token">.</SPAN>split<SPAN class="punctuation token">(</SPAN><SPAN class="string token">','</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
        <SPAN class="comment token"># Split the row by comma and append</SPAN>
        <SPAN class="comment token"># everything after the comma to lon</SPAN>
        lon<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>row<SPAN class="punctuation token">.</SPAN>split<SPAN class="punctuation token">(</SPAN><SPAN class="string token">','</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
    <SPAN class="comment token"># But if you get an error</SPAN>
    <SPAN class="keyword token">except</SPAN><SPAN class="punctuation token">:</SPAN>
        <SPAN class="comment token"># append a missing value to lat</SPAN>
        lat<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>np<SPAN class="punctuation token">.</SPAN>NaN<SPAN class="punctuation token">)</SPAN>
        <SPAN class="comment token"># append a missing value to lon</SPAN>
        lon<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>np<SPAN class="punctuation token">.</SPAN>NaN<SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[53]:</SPAN>


<SPAN class="comment token"># Create two new columns from lat and lon</SPAN>
df<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'latitude'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> lat
df<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'longitude'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> lon


<SPAN class="comment token"># In[54]:</SPAN>


df<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'latitude'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> df<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'latitude'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">.</SPAN>apply<SPAN class="punctuation token">(</SPAN><SPAN class="keyword token">lambda</SPAN> x<SPAN class="punctuation token">:</SPAN> <SPAN class="string token">' '</SPAN><SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>x<SPAN class="punctuation token">.</SPAN>split<SPAN class="punctuation token">(</SPAN><SPAN class="string token">' '</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[55]:</SPAN>


<SPAN class="keyword token">del</SPAN> df<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Location'</SPAN><SPAN class="punctuation token">]</SPAN>


<SPAN class="comment token"># In[56]:</SPAN>


df<SPAN class="punctuation token">.</SPAN>head<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[57]:</SPAN>


df<SPAN class="punctuation token">.</SPAN>to_csv<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">'C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\CSV\ADEQ_RX_Burns1.csv'</SPAN><SPAN class="punctuation token">,</SPAN> encoding<SPAN class="operator token">=</SPAN><SPAN class="string token">'utf-8'</SPAN><SPAN class="punctuation token">,</SPAN> index<SPAN class="operator token">=</SPAN><SPAN class="token boolean">False</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[63]:</SPAN>


gis <SPAN class="operator token">=</SPAN> GIS<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"https://www.arcgis.com"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"Your Username"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"Your Password"</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">"Logged in as "</SPAN> <SPAN class="operator token">+</SPAN> str<SPAN class="punctuation token">(</SPAN>gis<SPAN class="punctuation token">.</SPAN>properties<SPAN class="punctuation token">.</SPAN>user<SPAN class="punctuation token">.</SPAN>username<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[64]:</SPAN>


arcpy<SPAN class="punctuation token">.</SPAN>management<SPAN class="punctuation token">.</SPAN>XYTableToPoint<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">"C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\CSV\ADEQ_RX_Burns1.csv"</SPAN><SPAN class="punctuation token">,</SPAN> r<SPAN class="string token">"C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"longitude"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"latitude"</SPAN><SPAN class="punctuation token">,</SPAN> None<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]];-400 -400 1000000000;-100000 10000;-100000 10000;8.98315284119521E-09;0.001;0.001;IsHighPrecision"</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[65]:</SPAN>


arcpy<SPAN class="punctuation token">.</SPAN>management<SPAN class="punctuation token">.</SPAN>CalculateField<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">"C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"Ignition_Date"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"datetime.datetime.now()"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"PYTHON3"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"TEXT"</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[66]:</SPAN>


arcpy<SPAN class="punctuation token">.</SPAN>management<SPAN class="punctuation token">.</SPAN>Append<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">"'C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1'"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"https://services6.arcgis.com/l7uujk4hHifqabRB/arcgis/rest/services/ADEQ_RX_Burns1/FeatureServer/0"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"NO_TEST"</SPAN><SPAN class="punctuation token">,</SPAN> r'Smoke_Unit_Number <SPAN class="string token">"Smoke Unit Number"</SPAN> true true false <SPAN class="number token">2147483647</SPAN> Text <SPAN class="number token">0</SPAN> <SPAN class="number token">0</SPAN><SPAN class="punctuation token">,</SPAN>First<SPAN class="punctuation token">,</SPAN><SPAN class="comment token">#,C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1,Smoke_Unit_Number,0,5000;Burn_Number "Burn Number" true true false 2147483647 Text 0 0,First,#,C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1,Burn_Number,0,5000;Burn_Name "Burn Name" true true false 2147483647 Text 0 0,First,#,C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1,Burn_Name,0,5000;Ignition_Date "Ignition Date" true true false 8 Date 0 0,First,#,C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1,Ignition_Date,-1,-1;Approved_Acres "Approved Acres" true true false 0 Long 0 0,First,#,C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1,Approved_Acres,-1,-1;Notes "Notes" true true false 2147483647 Text 0 0,First,#,C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1,Notes,0,5000;latitude "latitude" true true false 0 Double 0 0,First,#,C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1,latitude,-1,-1;longitude "longitude" true true false 0 Double 0 0,First,#,C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1,longitude,-1,-1', '', '')</SPAN>


<SPAN class="comment token"># In[67]:</SPAN>


arcpy<SPAN class="punctuation token">.</SPAN>management<SPAN class="punctuation token">.</SPAN>Delete<SPAN class="punctuation token">(</SPAN>r<SPAN class="string token">"'C:\Code\Python\Web_Scraping\ADEQ_RX_Burns\New File Geodatabase.gdb\ADEQ_RX_Burns1'"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">)</SPAN>


<SPAN class="comment token"># In[ ]:</SPAN>




<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

JoeBorgione

Not knowing how the points are published, I would think if it is to a local on-prem portal that then is an AGOL item , updates including truncates and append should be invisible to the end users, no?

jaykapalczynski

I just did what Joe was talking about.....so now to continue with your solution

jaykapalczynski

Good point Bruce....I will have to continue your approach...but still confused....

wheelie_logo.jpg

jaykapalczynski

Alright I simply created a model

Event Layer then to Geodatabase. Will set this python script to run on a scheduled task

bharold

Nice one Joe, however it may be a requirement with this update frequency to have no downtime, in which case I would lean to the ETL approach which operates one feature at a time or to the feature service REST API so you only operate on the feature service change set in milliseconds. Sorry to be pedantic ;-).

WDMParkPlanner_0-1715112153269.png

Screenshot 2024-05-07 at 3.05.55 PM.png

Scott_Harris_0-1715112436738.png

Scott_Harris_1-1715112466631.png

JoeBorgione

What I've done in a similar situation is to turn the csv into a xy event, ( Make XY Event Layer—Data Management toolbox | Documentation ) and then create a feature class from that. All in arcpy an all in memory. Then truncate the target feature class and append the in_memory feature class to it.

bharold

This topic describes using the arcgis API:

Overwriting feature layers | ArcGIS for Developers

AndrewAidt2_1-1715110873723.png

jaykapalczynski

I have an XY field in the csv file

jaykapalczynski

I can run a stored procedure and read the data....I think I need to then read the Stored Procedure via python to create the Feature Class? Using an insert cursor?

<SPAN class="keyword token">ALTER</SPAN> <SPAN class="keyword token">PROCEDURE</SPAN> <SPAN class="punctuation token">[</SPAN>dbo<SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">[</SPAN>addData<SPAN class="punctuation token">]</SPAN> 

<SPAN class="keyword token">AS</SPAN>
<SPAN class="keyword token">BEGIN</SPAN>
	<SPAN class="comment token">-- SET NOCOUNT ON added to prevent extra result sets from</SPAN>
	<SPAN class="comment token">-- interfering with SELECT statements.</SPAN>
	<SPAN class="keyword token">SET</SPAN> NOCOUNT <SPAN class="keyword token">ON</SPAN><SPAN class="punctuation token">;</SPAN>
	
	<SPAN class="keyword token">CREATE</SPAN> <SPAN class="keyword token">TABLE</SPAN> <SPAN class="comment token">#ImportData</SPAN>
	<SPAN class="punctuation token">(</SPAN>
		CFS <SPAN class="keyword token">varchar</SPAN><SPAN class="punctuation token">(</SPAN>MAX<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> 
		Code <SPAN class="keyword token">varchar</SPAN><SPAN class="punctuation token">(</SPAN>MAX<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
	<SPAN class="punctuation token">)</SPAN>

    <SPAN class="comment token">-- Insert statements for procedure here</SPAN>
<SPAN class="keyword token">BULK</SPAN> <SPAN class="keyword token">INSERT</SPAN> <SPAN class="comment token">#ImportData from 'C:\Scripts\Service\DailyCalls.csv' WITH</SPAN>
<SPAN class="punctuation token">(</SPAN>
	FIELDTERMINATOR <SPAN class="operator token">=</SPAN> <SPAN class="string token">','</SPAN><SPAN class="punctuation token">,</SPAN>
	FIRSTROW <SPAN class="operator token">=</SPAN> <SPAN class="number token">2</SPAN>
<SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">SELECT</SPAN> <SPAN class="operator token">*</SPAN> <SPAN class="keyword token">FROM</SPAN> <SPAN class="comment token">#ImportData</SPAN>
<SPAN class="keyword token">DROP</SPAN> <SPAN class="keyword token">TABLE</SPAN> <SPAN class="comment token">#ImportData</SPAN>
<SPAN class="keyword token">END</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

bharold

An ETL approach can operate directly on both the CSV file and the feature service:

https://community.esri.com/community/open-platform-standards-and-interoperability/blog/2019/05/31/automate-your-etl-processes-on-a-schedule

You could script it entirely too a few ways. Is the geometry in the data or are you geocoding?

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.