How to pass username/pass to a Hosted Feature Layer in Python

[DougBrowning]

I have a simple script to grab the data from a HFS.  It works great if the HFS sharing is Everyone but I can not get it to work with a username/password.

I know that arcpy.SignInToPortal_server was removed (even though it worked great).  I have also tried using he script in a toolbox then run inside of ArcMap but that does not work either.  I do have ArcMap open and signed in.

I use a layer file that points to the HFS.

Any help on the proper way to do this is great.  I am using 10.2.2 but could use 10.3.1 also.

import arcpy

localDB = r"something.gdb"

justPlots = r"JustPlots.lyr"

# Load in from AGO HFS

arcpy.MakeFeatureLayer_management(justPlots, "plots_lyr")

arcpy.Append_management("plots_lyr", localDB + "\\Plots", "NO_TEST")

Thanks

[DougBrowning]

I now have it working in a toolbox run from within ArcMap.  Turns out my lyr file went bad somehow.

Would still like any input on stand alone though.

[DavinShokes1]

If you're primary purpose is to export to fgdb, you could bypass ArcMap altogether with ArcREST Esri/ArcREST · GitHub . This ArcREST/create_replica_fs.py will get you a a local backup of the data (no bidirectional option yet). If you want to keep the data in one location, you can follow ArcREST/query_agol_layer.py to query features since the last backup. The in-memory featureset can then be incorporated with the existing local data via arcpy cursors for fgdb or the pyodbc library for sql databases.

It is also possible to programmatically build 2 way synchronization with arcpy and arcrest. Then you can use windows task scheduler to sync your local version every 15 minutes.

Hope this helps,

Davin

[DougBrowning]

Do you use the rest to get around the feature return limit?  That seems to be an issue with using the layer file - it will only return the first X records as set in the service.

The create replica script worked great.

The query script gave me this error

C:\Python27\ArcGIS10.2\lib\site-packages\requests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

  InsecurePlatformWarning

C:\Python27\ArcGIS10.2\lib\site-packages\requests\packages\urllib3\util\ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

  InsecurePlatformWarning

initialExtent  - attribute not implmented in Feature Layer.

spatialReference  - attribute not implmented in Feature Layer.

size  - attribute not implmented in Feature Layer.

layers  - attribute not implmented in Feature Layer.

tables  - attribute not implmented in Feature Layer.

syncCapabilities  - attribute not implmented in Feature Layer.

units  - attribute not implmented in Feature Layer.

xssPreventionInfo  - attribute not implmented in Feature Layer.

hasVersionedData  - attribute not implmented in Feature Layer.

fullExtent  - attribute not implmented in Feature Layer.

serviceDescription  - attribute not implmented in Feature Layer.

editorTrackingInfo  - attribute not implmented in Feature Layer.

supportsDisconnectedEditing  - attribute not implmented in Feature Layer.

syncEnabled  - attribute not implmented in Feature Layer.

Traceback (most recent call last):

  File "<module1>", line 21, in <module>

  File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\agol\layer.py", line 740, in query

    return FeatureSet.fromJSON(json.dumps(results))

  File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\common\general.py", line 590, in fromJSON

    for feat in jd['features']:

KeyError: 'features'

Also is there any docs on any of this.  There are many options in the Create Replica but nothing on what they mean.  I was wondering if I could get a locals copy that will show as a replica and then I can sync back and forth with it.  It seems like you can get a copy locally or keep the replica around but not both.

All i can find is the comments in the code

               replicaName - string of replica name

               layers - layer id # as comma seperated string

               keep_replica - if the replica does not have returnAsFeatureClass set to true,

                              the feature service creates a permanent copy of the replica.

                              If this is just a pull, then erase the replica in order to prevent

                              build up of replicas.

               layerQueries - In addition to the layers and geometry parameters, the layerQueries

                              parameter can be used to further define what is replicated. This

                              parameter allows you to set properties on a per layer or per table

                              basis. Only the properties for the layers and tables that you want

                              changed from the default are required.

                                Example:

                                  layerQueries = {"0":{"queryOption": "useFilter", "useGeometry": true,

                                                 "where": "requires_inspection = Yes"}}

               geometryFilter - Geospatial filter applied to the replica to parse down data output.

               returnAttachments - If true, attachments are added to the replica and returned in the

                                   response. Otherwise, attachments are not included.

               returnAttachmentDatabyURL -  If true, a reference to a URL will be provided for each

                                            attachment returned from createReplica. Otherwise,

                                            attachments are embedded in the response.

               returnAsFeatureClass - If a local copy is desired, set this parameter to True, else

                                      the service will return information on how to download the

                                      json file.

               out_path - Path where the FGDB will be saved.  Only used with returnAsFeatureClass is

                          True.

thanks

[DavinShokes1]

Do you use the rest to get around the feature return limit?

Yes I use it to sync 70k+ features with a business system. The ArcREST code checks max feature count, breaks up large queries into chunks, and then combines them before returning the result.

Traceback (most recent call last):

  File "<module1>", line 21, in <module>

  File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\agol\layer.py", line 740, in query

    return FeatureSet.fromJSON(json.dumps(results))

  File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\common\general.py", line 590, in fromJSON

    for feat in jd['features']:

KeyError: 'features'

Its tough to say without seeing the featurelayer.query() line, but try

# Set Geoprocessing environments

    Scratch_gdb = r'C:\Python\scratch.gdb'

    arcpy.env.scratchWorkspace = Scratch_gdb

    arcpy.env.workspace = Scratch_gdb

print fl.query(where="1=1",out_fields='*',returnGeometry=False, returnfeatureclass=True, out_fc='tempfc')

That may help narrow in on the issue.

Also is there any docs on any of this...

All i can find is the comments in the code

               replicaName - string of replica name

               layers - layer id # as comma seperated string

               keep_replica - if the replica does not have returnAsFeatureClass set to true,

                              the feature service creates a permanent copy of the replica.

                              If this is just a pull, then erase the replica in order to prevent

                              build up of replicas.

               layerQueries - In addition to the layers and geometry parameters, the layerQueries

                              parameter can be used to further define what is replicated. This

                              parameter allows you to set properties on a per layer or per table

                              basis. Only the properties for the layers and tables that you want

                              changed from the default are required.

                                Example:

                                  layerQueries = {"0":{"queryOption": "useFilter", "useGeometry": true,

                                                 "where": "requires_inspection = Yes"}}

               geometryFilter - Geospatial filter applied to the replica to parse down data output.

               returnAttachments - If true, attachments are added to the replica and returned in the

                                   response. Otherwise, attachments are not included.

               returnAttachmentDatabyURL -  If true, a reference to a URL will be provided for each

                                            attachment returned from createReplica. Otherwise,

                                            attachments are embedded in the response.

               returnAsFeatureClass - If a local copy is desired, set this parameter to True, else

                                      the service will return information on how to download the

                                      json file.

               out_path - Path where the FGDB will be saved.  Only used with returnAsFeatureClass is

                          True.

This is the only documentation currently.