How to pass username/pass to a Hosted Feature Layer in Python

DougBrowning · ‎09-17-2015

I have a simple script to grab the data from a HFS. It works great if the HFS sharing is Everyone but I can not get it to work with a username/password.

I know that arcpy.SignInToPortal_server was removed (even though it worked great). I have also tried using he script in a toolbox then run inside of ArcMap but that does not work either. I do have ArcMap open and signed in.

I use a layer file that points to the HFS.

Any help on the proper way to do this is great. I am using 10.2.2 but could use 10.3.1 also.

import arcpy

localDB = r"something.gdb"

justPlots = r"JustPlots.lyr"

# Load in from AGO HFS

arcpy.MakeFeatureLayer_management(justPlots, "plots_lyr")

arcpy.Append_management("plots_lyr", localDB + "\\Plots", "NO_TEST")

Thanks

DougBrowning · ‎09-17-2015

I now have it working in a toolbox run from within ArcMap. Turns out my lyr file went bad somehow.

Would still like any input on stand alone though.

DavinShokes1 · ‎09-17-2015

If you're primary purpose is to export to fgdb, you could bypass ArcMap altogether with ArcREST Esri/ArcREST · GitHub . This ArcREST/create_replica_fs.py will get you a a local backup of the data (no bidirectional option yet). If you want to keep the data in one location, you can follow ArcREST/query_agol_layer.py to query features since the last backup. The in-memory featureset can then be incorporated with the existing local data via arcpy cursors for fgdb or the pyodbc library for sql databases.

It is also possible to programmatically build 2 way synchronization with arcpy and arcrest. Then you can use windows task scheduler to sync your local version every 15 minutes.

Hope this helps,

Davin

DougBrowning · ‎09-18-2015

Do you use the rest to get around the feature return limit? That seems to be an issue with using the layer file - it will only return the first X records as set in the service.

The create replica script worked great.

The query script gave me this error

C:\Python27\ArcGIS10.2\lib\site-packages\requests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

InsecurePlatformWarning

C:\Python27\ArcGIS10.2\lib\site-packages\requests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

InsecurePlatformWarning

initialExtent - attribute not implmented in Feature Layer.

spatialReference - attribute not implmented in Feature Layer.

size - attribute not implmented in Feature Layer.

layers - attribute not implmented in Feature Layer.

tables - attribute not implmented in Feature Layer.

syncCapabilities - attribute not implmented in Feature Layer.

units - attribute not implmented in Feature Layer.

xssPreventionInfo - attribute not implmented in Feature Layer.

hasVersionedData - attribute not implmented in Feature Layer.

fullExtent - attribute not implmented in Feature Layer.

serviceDescription - attribute not implmented in Feature Layer.

editorTrackingInfo - attribute not implmented in Feature Layer.

supportsDisconnectedEditing - attribute not implmented in Feature Layer.

syncEnabled - attribute not implmented in Feature Layer.

Traceback (most recent call last):

File "<module1>", line 21, in <module>

File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\agol\layer.py", line 740, in query

return FeatureSet.fromJSON(json.dumps(results))

File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\common\general.py", line 590, in fromJSON

for feat in jd['features']:

KeyError: 'features'

Also is there any docs on any of this. There are many options in the Create Replica but nothing on what they mean. I was wondering if I could get a locals copy that will show as a replica and then I can sync back and forth with it. It seems like you can get a copy locally or keep the replica around but not both.

All i can find is the comments in the code

replicaName - string of replica name

layers - layer id # as comma seperated string

keep_replica - if the replica does not have returnAsFeatureClass set to true,

the feature service creates a permanent copy of the replica.

If this is just a pull, then erase the replica in order to prevent

build up of replicas.

layerQueries - In addition to the layers and geometry parameters, the layerQueries

parameter can be used to further define what is replicated. This

parameter allows you to set properties on a per layer or per table

basis. Only the properties for the layers and tables that you want

changed from the default are required.

Example:

layerQueries = {"0":{"queryOption": "useFilter", "useGeometry": true,

"where": "requires_inspection = Yes"}}

geometryFilter - Geospatial filter applied to the replica to parse down data output.

returnAttachments - If true, attachments are added to the replica and returned in the

response. Otherwise, attachments are not included.

returnAttachmentDatabyURL - If true, a reference to a URL will be provided for each

attachment returned from createReplica. Otherwise,

attachments are embedded in the response.

returnAsFeatureClass - If a local copy is desired, set this parameter to True, else

the service will return information on how to download the

json file.

out_path - Path where the FGDB will be saved. Only used with returnAsFeatureClass is

True.

thanks

DavinShokes1 · ‎09-28-2015

Do you use the rest to get around the feature return limit?

Yes I use it to sync 70k+ features with a business system. The ArcREST code checks max feature count, breaks up large queries into chunks, and then combines them before returning the result.

Traceback (most recent call last):
File "<module1>", line 21, in <module>
File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\agol\layer.py", line 740, in query
return FeatureSet.fromJSON(json.dumps(results))
File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\common\general.py", line 590, in fromJSON
for feat in jd['features']:
KeyError: 'features'

Its tough to say without seeing the featurelayer.query() line, but try

# Set Geoprocessing environments

Scratch_gdb = r'C:\Python\scratch.gdb'

arcpy.env.scratchWorkspace = Scratch_gdb

arcpy.env.workspace = Scratch_gdb

print fl.query(where="1=1",out_fields='*',returnGeometry=False, returnfeatureclass=True, out_fc='tempfc')

That may help narrow in on the issue.

Also is there any docs on any of this...
All i can find is the comments in the code
               replicaName - string of replica name
               layers - layer id # as comma seperated string
               keep_replica - if the replica does not have returnAsFeatureClass set to true,
                              the feature service creates a permanent copy of the replica.
                              If this is just a pull, then erase the replica in order to prevent
                              build up of replicas.
               layerQueries - In addition to the layers and geometry parameters, the layerQueries
                              parameter can be used to further define what is replicated. This
                              parameter allows you to set properties on a per layer or per table
                              basis. Only the properties for the layers and tables that you want
                              changed from the default are required.
                                Example:
                                  layerQueries = {"0":{"queryOption": "useFilter", "useGeometry": true,
                                                 "where": "requires_inspection = Yes"}}
               geometryFilter - Geospatial filter applied to the replica to parse down data output.
               returnAttachments - If true, attachments are added to the replica and returned in the
                                   response. Otherwise, attachments are not included.
               returnAttachmentDatabyURL - If true, a reference to a URL will be provided for each
                                            attachment returned from createReplica. Otherwise,
                                            attachments are embedded in the response.
               returnAsFeatureClass - If a local copy is desired, set this parameter to True, else
                                      the service will return information on how to download the
                                      json file.
               out_path - Path where the FGDB will be saved. Only used with returnAsFeatureClass is
                          True.

This is the only documentation currently.