I have this URL: https://geohub-vadeq.hub.arcgis.com/datasets/57759688e4944bb987add68c4f0c5ada_104/explore?location=3...
or
https://geohub-vadeq.hub.arcgis.com/datasets/57759688e4944bb987add68c4f0c5ada_104/about
I want to use python/arcpy and download all its data. If I use the RestEndpoint there is a limit of 1000 records. I cant seem to get all 50k.
Any ideas on how I can programmatically ? Looking to get a GeoJSON or JSON and write it to a JSON file locally...
Solved! Go to Solution.
Forgot I posted a mirror thread... go here for the solution....
Here you go...
https://github.com/wagisdev/extractservices/blob/main/extractservice.py
Just add it to your pro project and it will pull it out.
OK I am a bit confused here.....
Questions:
This is a public hub site and NOT mine. Not sure where to specify where is downloading to and what format is it downloading?
# Project Store Location
path_Proj = r'C:\ProjectPath\project.aprx'
# Data Store Location
path_DataProcessing = r'C:\ProjectPath\Pull-Processing.sde'
path_DataFinal = r'C:\ProjectPath\Pull-Final.sde'
# Target Projection
targetSRID = 6597
# Send confirmation of rebuild to
email_target = 'xxx@yahoo.com'
email_customer = 'xxx@yahoo.com'
email_subjectSchema = '' #Whatever you want the subject to be here.
# Configure the e-mail server and other info here.
mail_server = 'smtprelay.gis.dev'
mail_from = 'Message<noreply@gis.dev>'
# Portal Configs
portalURL = 'https://geohub-vadeq.hub.arcgis.com/datasets/57759688e4944bb987add68c4f0c5ada_104'
portalUser = '' #Your User Name
portalPW = '' #Your Password encoded Base64. Remove decode if you are not practicing security by obscurity.
OK
I think I create an arcpro project and put the Services in there that I want extracted in it????
How do I specify where they are going and what DB?
# Data Store Location
path_DataProcessing = r'C:\ProjectPath\Pull-Processing.sde'
path_DataFinal = r'C:\ProjectPath\Pull-Final.sde'
Does the Data Store Location define this? if so I am not following here.... how does a .sde connection files tell me what specific Features to create or write to?
Do I have to have the ArcGIS Pro project open to run this?
It wont even auth and get me into their portal....
This is the Hub trying to get to:
https://geohub-vadeq.hub.arcgis.com/datasets/57759688e4944bb987add68c4f0c5ada_104
This is the URL set up in ArcGIS Pro
https://apps.deq.virginia.gov/arcgis/rest/services/public/EDMA/MapServer/
Forgot I posted a mirror thread... go here for the solution....
This did the same thing by getting the data from the service and put in my SDE
Couple Hundred lines of code less.... I know I can remove a few of the Imports ... they were from a lingering script
import arcpy
import requests
import json
import os
import sys
import traceback
from datetime import date
arcpy.env.overwriteOutput = True
def copyData():
# Set local variables
inFeatures = 'https://apps.deq.virginia.gov/arcgis/rest/services/public/EDMA/MapServer/102'
outLocation = r"C:\Users\xxx\Desktop\GIS_projects\TestingExportData\output.gdb"
outFeatureClass = "TEST_PetTankFac"
## Add Expression
#delimitedField = arcpy.AddFieldDelimiters(arcpy.env.workspace, "NAME")
#expression = delimitedField + " = 'Post Office'"
# Run FeatureClassToFeatureClass
arcpy.conversion.FeatureClassToFeatureClass(inFeatures, outLocation, outFeatureClass)
if __name__ == '__main__':
copyData()
Sorry for not getting back to you sooner. Looks like you got everything you needed.