Morning All
I want to download the Covid19 Daily Cases data from here
https://services1.arcgis.com/0MSEUqKaxRlEPj5g/ArcGIS/rest/services/Coronavirus_2019_nCoV_Cases/FeatureServer/1
I have tried this way.
1. ESRI DUMPER - Works but then I have difficulty converting the ESRI geojson to something that is usable in a DataBase.
Any thoughts on the best way to convert this output to a SQL readable format ? CSV/TAB delimited flat file for example ?
Many thanks
Regards.
Jeremy.
Morning Randy
I solved this by simply running a where statement on the ObjectID's.
Where "ObjectID>1000"
Seemed to work.
Good Afternoon Randy
Many thanks for the post. After a little bit of re-working I successfully created a script that worked.
Many many thanks. The only issue I have is that the feature layer I am querying has 3466 features in it and the Max Record Count is limited to 1000. Is there anyway to increase this ?
If you requested json instead of geojson, you could use the JSONToFeatures tool (Pro version) to import into ArcMap.
My test script using python 2.7:
<SPAN class="keyword token">import</SPAN> arcpy<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">,</SPAN> urllib2<SPAN class="punctuation token">,</SPAN> json URL <SPAN class="operator token">=</SPAN> <SPAN class="string token">"https://services1.arcgis.com/0MSEUqKaxRlEPj5g/ArcGIS/rest/services/Coronavirus_2019_nCoV_Cases/FeatureServer/1/query"</SPAN> <SPAN class="comment token"># query this feature</SPAN> <SPAN class="comment token"># using GET method</SPAN> query_dict <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">{</SPAN> <SPAN class="string token">"where"</SPAN> <SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"1=1"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"outFields"</SPAN> <SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"*"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"f"</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"json"</SPAN> <SPAN class="punctuation token">}</SPAN> <SPAN class="comment token"># results in json format</SPAN> response <SPAN class="operator token">=</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>URL<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlencode<SPAN class="punctuation token">(</SPAN>query_dict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">with</SPAN> open<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'covid_update.json'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">as</SPAN> f<SPAN class="punctuation token">:</SPAN> f<SPAN class="punctuation token">.</SPAN>write<SPAN class="punctuation token">(</SPAN>response<SPAN class="punctuation token">.</SPAN>read<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>JSONToFeatures_conversion<SPAN class="punctuation token">(</SPAN>in_json_file<SPAN class="operator token">=</SPAN><SPAN class="string token">"covid_update.json"</SPAN><SPAN class="punctuation token">,</SPAN> out_features<SPAN class="operator token">=</SPAN><SPAN class="string token">"C:/Path/to/file.gdb/covid_update"</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">print</SPAN> <SPAN class="string token">'Done.'</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
You can use the json module to process the geojson. I was testing the following using python 2.7. It makes the URL request and writes a tab delimited file.
<SPAN class="keyword token">import</SPAN> urllib<SPAN class="punctuation token">,</SPAN> urllib2<SPAN class="punctuation token">,</SPAN> json<SPAN class="punctuation token">,</SPAN> sys<SPAN class="punctuation token">,</SPAN> time<SPAN class="punctuation token">,</SPAN> collections fw <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"covidUpdate.txt"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"w"</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="comment token"># file to write</SPAN> URL <SPAN class="operator token">=</SPAN> <SPAN class="string token">"https://services1.arcgis.com/0MSEUqKaxRlEPj5g/ArcGIS/rest/services/Coronavirus_2019_nCoV_Cases/FeatureServer/1/query"</SPAN> <SPAN class="comment token"># query this feature</SPAN> <SPAN class="comment token"># using GET method</SPAN> query_dict <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">{</SPAN> <SPAN class="string token">"where"</SPAN> <SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"1=1"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="comment token"># something always true</SPAN> <SPAN class="string token">"outFields"</SPAN> <SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"*"</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"f"</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="string token">"geojson"</SPAN> <SPAN class="punctuation token">}</SPAN> <SPAN class="comment token"># results in geojson format</SPAN> jsonResponse <SPAN class="operator token">=</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>URL<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlencode<SPAN class="punctuation token">(</SPAN>query_dict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> features <SPAN class="operator token">=</SPAN> json<SPAN class="punctuation token">.</SPAN>loads<SPAN class="punctuation token">(</SPAN>jsonResponse<SPAN class="punctuation token">.</SPAN>read<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> object_pairs_hook<SPAN class="operator token">=</SPAN>collections<SPAN class="punctuation token">.</SPAN>OrderedDict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN>u<SPAN class="string token">'features'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="comment token"># using the features section</SPAN> <SPAN class="comment token"># Header</SPAN> fw<SPAN class="punctuation token">.</SPAN>writelines<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN> <SPAN class="string token">'OBJECTID'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Province_State'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Country_Region'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Last_Update'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Latitude'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Longitude'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Confirmed'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Recovered'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Deaths'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="comment token"># Data rows</SPAN> <SPAN class="keyword token">for</SPAN> f <SPAN class="keyword token">in</SPAN> features<SPAN class="punctuation token">:</SPAN> fw<SPAN class="punctuation token">.</SPAN>writelines<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'OBJECTID'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Province_State'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">if</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Province_State'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None <SPAN class="keyword token">else</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Country_Region'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="comment token"># convert timestamp to local time</SPAN> <SPAN class="comment token"># time.strftime('%c', time.localtime(f['properties']['Last_Update']/1000)) if f['properties']['Last_Update'] is not None else '',</SPAN> <SPAN class="comment token"># convert timestamp to GMT</SPAN> time<SPAN class="punctuation token">.</SPAN>strftime<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%Y-%m-%d %H:%M:%S'</SPAN><SPAN class="punctuation token">,</SPAN> time<SPAN class="punctuation token">.</SPAN>gmtime<SPAN class="punctuation token">(</SPAN>f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Last_Update'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="operator token">/</SPAN><SPAN class="number token">1000</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">if</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Last_Update'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None <SPAN class="keyword token">else</SPAN> <SPAN class="string token">''</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Lat'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Long_'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Confirmed'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Recovered'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN> f<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'properties'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Deaths'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> fw<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">print</SPAN> <SPAN class="string token">'Done.'</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
Results:
OBJECTID Province_State Country_Region Last_Update Latitude Longitude Confirmed Recovered Deaths 1 Abruzzo Italy 2020-05-22 23:32:40 42.35122196 13.39843823 3220 1647 394 2 Acre Brazil 2020-05-22 23:32:40 -9.0238 -70.812 3343 0 80 3 Aguascalientes Mexico 2020-05-22 23:32:40 21.8853 -102.2916 586 402 21 .....<SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
I noticed that Provence_State is sometimes null/None, as was the timestamp on occasion. I kept the time in GMT/UTC, but you can convert it to local time if desired. It is also possible to use an insert cursor and create a feature layer if desired.
This should give you some ideas.
Membros conectados podem postar, seguir atualizações e mais. Novo aqui? Registre uma conta gratuita.
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.