Morning All
I want to download the Covid19 Daily Cases data from here
https://services1.arcgis.com/0MSEUqKaxRlEPj5g/ArcGIS/rest/services/Coronavirus_2019_nCoV_Cases/FeatureServer/1
I have tried this way.
1. ESRI DUMPER - Works but then I have difficulty converting the ESRI geojson to something that is usable in a DataBase.
Any thoughts on the best way to convert this output to a SQL readable format ? CSV/TAB delimited flat file for example ?
Many thanks
Regards.
Jeremy.
Morning Randy
I solved this by simply running a where statement on the ObjectID's.
Where "ObjectID>1000"
Seemed to work.
Good Afternoon Randy
Many thanks for the post. After a little bit of re-working I successfully created a script that worked.
Many many thanks. The only issue I have is that the feature layer I am querying has 3466 features in it and the Max Record Count is limited to 1000. Is there anyway to increase this ?
If you requested json instead of geojson, you could use the JSONToFeatures tool (Pro version) to import into ArcMap.
My test script using python 2.7:
<SPAN class="keyword token">import</SPAN> arcpy<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">,</SPAN> urllib2<SPAN class="punctuation token">,</SPAN> json URL <SPAN class="operator token">=</SPAN> <SPAN class="string token">"https://services1.arcgis.com/0MSEUqKaxRlEPj5g/ArcGIS/rest/services/Coronavirus_2019_nCoV_Cases/FeatureServer/1/query"</SPAN> <SPAN class="comment token"># query this feature</SPAN> <SPAN class="comment token"># using GET method</SPAN> query_dict <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">{</SPAN> <SPAN class="string token">"where"</SPAN> <SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"1=1"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"outFields"</SPAN> <SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"*"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"f"</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"json"</SPAN> <SPAN class="punctuation token">}</SPAN> <SPAN class="comment token"># results in json format</SPAN> response <SPAN class="operator token">=</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>URL<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlencode<SPAN class="punctuation token">(</SPAN>query_dict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">with</SPAN> open<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'covid_update.json'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">as</SPAN> f<SPAN class="punctuation token">:</SPAN> f<SPAN class="punctuation token">.</SPAN>write<SPAN class="punctuation token">(</SPAN>response<SPAN class="punctuation token">.</SPAN>read<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>JSONToFeatures_conversion<SPAN class="punctuation token">(</SPAN>in_json_file<SPAN class="operator token">=</SPAN><SPAN class="string token">"covid_update.json"</SPAN><SPAN class="punctuation token">,</SPAN> out_features<SPAN class="operator token">=</SPAN><SPAN class="string token">"C:/Path/to/file.gdb/covid_update"</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">print</SPAN> <SPAN class="string token">'Done.'</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
You can use the json module to process the geojson. I was testing the following using python 2.7. It makes the URL request and writes a tab delimited file.
<SPAN class="keyword token">import</SPAN> urllib<SPAN class="punctuation token">,</SPAN> urllib2<SPAN class="punctuation token">,</SPAN> json<SPAN class="punctuation token">,</SPAN> sys<SPAN class="punctuation token">,</SPAN> time<SPAN class="punctuation token">,</SPAN> collections fw <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"covidUpdate.txt"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"w"</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="comment token"># file to write</SPAN> URL <SPAN class="operator token">=</SPAN> <SPAN class="string token">"https://services1.arcgis.com/0MSEUqKaxRlEPj5g/ArcGIS/rest/services/Coronavirus_2019_nCoV_Cases/FeatureServer/1/query"</SPAN> <SPAN class="comment token"># query this feature</SPAN> <SPAN class="comment token"># using GET method</SPAN> query_dict <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">{</SPAN> <SPAN class="string token">"where"</SPAN> <SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"1=1"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="comment token"># something always true</SPAN> <SPAN class="string token">"outFields"</SPAN> <SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"*"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"f"</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"geojson"</SPAN> <SPAN class="punctuation token">}</SPAN> <SPAN class="comment token"># results in geojson format</SPAN> jsonResponse <SPAN class="operator token">=</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>URL<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlencode<SPAN class="punctuation token">(</SPAN>query_dict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> features <SPAN class="operator token">=</SPAN> json<SPAN class="punctuation token">.</SPAN>loads<SPAN class="punctuation token">(</SPAN>jsonResponse<SPAN class="punctuation token">.</SPAN>read<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> object_pairs_hook<SPAN class="operator token">=</SPAN>collections<SPAN class="punctuation token">.</SPAN>OrderedDict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN>u<SPAN class="string token">'features'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="comment token"># using the features section</SPAN> <SPAN class="comment token"># Header</SPAN> fw<SPAN class="punctuation token">.</SPAN>writelines<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN> <SPAN class="string token">'OBJECTID'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Province_State'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Country_Region'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Last_Update'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Latitude'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Longitude'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Confirmed'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Recovered'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Deaths'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="comment token"># Data rows</SPAN> <SPAN class="keyword token">for</SPAN> f <SPAN class="keyword token">in</SPAN> features<SPAN class="punctuation token">:</SPAN> fw<SPAN class="punctuation token">.</SPAN>writelines<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'OBJECTID'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Province_State'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">if</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Province_State'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None <SPAN class="keyword token">else</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Country_Region'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="comment token"># convert timestamp to local time</SPAN> <SPAN class="comment token"># time.strftime('%c', time.localtime(f['properties']['Last_Update']/1000)) if f['properties']['Last_Update'] is not None else '',</SPAN> <SPAN class="comment token"># convert timestamp to GMT</SPAN> time<SPAN class="punctuation token">.</SPAN>strftime<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%Y-%m-%d %H:%M:%S'</SPAN><SPAN class="punctuation token">,</SPAN> time<SPAN class="punctuation token">.</SPAN>gmtime<SPAN class="punctuation token">(</SPAN>f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Last_Update'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="operator token">/</SPAN><SPAN class="number token">1000</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">if</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Last_Update'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None <SPAN class="keyword token">else</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Lat'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Long_'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Confirmed'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Recovered'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Deaths'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> fw<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">print</SPAN> <SPAN class="string token">'Done.'</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
Results:
OBJECTID Province_State Country_Region Last_Update Latitude Longitude Confirmed Recovered Deaths 1 Abruzzo Italy 2020-05-22 23:32:40 42.35122196 13.39843823 3220 1647 394 2 Acre Brazil 2020-05-22 23:32:40 -9.0238 -70.812 3343 0 80 3 Aguascalientes Mexico 2020-05-22 23:32:40 21.8853 -102.2916 586 402 21 .....<SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
I noticed that Provence_State is sometimes null/None, as was the timestamp on occasion. I kept the time in GMT/UTC, but you can convert it to local time if desired. It is also possible to use an insert cursor and create a feature layer if desired.
This should give you some ideas.
Aangemelde leden kunnen berichten plaatsen, updates volgen en meer. Nieuw hier? Registreer een gratis account.
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.