Latest Contributions by FredSpataro

‎04-03-2024

@RonnieRichards Did you figure anything out on this? I'm having about the same issue... setting sharing_draft.copyDataToServer = False and I want to be able to get the 'data source not registered' warning/error BEFORE creating the .sd file. So far it will copy all the data locally, then tell me it's not registered. Which is especially not good for large datasets. The 'analyze' button in the UI must be doing something that's not exposed to the api or gp toolset?

‎03-15-2024

Hi All, This process was working correctly just recently, only change has been the latest ArcPro update. I'm basically following the examples in this help doc. Sample of bare min code to reproduce below. My service/task list is pretty complicated so the script is a HUGE time/mistake saver. I've tried a few tweaks and modified it down to just one tool and always get the same result. I've also tried the script with just a single, simple out-of-the box tool (AddXY) rather than the custom tools/scripts -- example below. Environment: ArcPro 3.2.2 ArcGIS Server (standalone 11.1) Process Results: sddraft and sd files get created (no warnings, errors in analyze) I've opened the sd archive and extracted. Everything looks fine. The toolbox contains tools to be published as tasks as configured sd file uploads fine to arcgis server service creates without error no arcgis server manager log errors or warnings nor obvious messages at info level Two issues: service task list is empty! minor but the message level setting does not take. SOC count and other service properties get set correctly. Repro script: result = arcpy.management.AddXY( in_features=r"D:\temp\data\scratch\Default.gdb\GPXPoints" ) ^ sd = r"D:\temp\data\scratch\xy.sd" sddraft = r"D:\temp\data\scratch\xy.sddraft" serverconnectionfile = r"D:\temp\data\scratch\arcgis_on_server.com.ags" serviceName = "XYTest" analyzeMessages = arcpy.CreateGPSDDraft( result, sddraft, serviceName, server_type="FROM_CONNECTION_FILE", connection_file_path= serverconnectionfile, copy_data_to_server=True, folder_name=None, summary="XY Service", tags="gp", executionType="Synchronous", resultMapServer=False, showMessages="INFO", maximumRecords=5000, minInstances=2, maxInstances=3, maxUsageTime=100, maxWaitTime=10, maxIdleTime=180) analyzeMessages #{'errors': {}, 'warnings': {}, 'messages': {}} arcpy.server.StageService(sddraft, sd) #<Result 'D:\\temp\\data\\scratch\\xy.sd'> Attached are the output sddraft and sd files Thanks for any insights.

‎02-15-2024

No i don't think so... the layer indices definitely stayed the same but those are defined in the properties

‎02-15-2024

Hi Eugene, (sorry replied on the wrong login) 1. I don't think i've upgraded or patched my version of Pro since that last publish. I'm running 3.1.2... looks like there's a 3.2.2 available so i'm going to see what happens if i update... 2. sorry for the confusion... My screen shots are just examples of the INT field. I have multiple fields that allow NULL, if the field is an INT and value = NULL, the publish process sets the value = 0, If the fields is STRING and value = NULL, the publish process sets the value = {empty string} or "" when viewed in JSON.

‎02-15-2024

Hi All, Something happened since early December with either Pro or AGOL. We have a feature service that gets updated on monthly-ish interval using the 'Overwrite Web Layer' button in ArcPro. The polygon layer getting published has multiple INT and STRING fields where 'Allow Null=True' and many of the features have NULL values. This was all fine and dandy when we last ran the update on 12/6/23. Today when we ran the update, all the null values are converted to something explicit. INT values are now 0, STRING values are now 'empty string'. This basically broke our application code that expects NULL as well as symbols, query and everything else where NULLs were expected. WTF? Luckily we caught this in a TEST version before deploying to PROD. How do we get the data to publish as NULL? Screen shot of a REST endpoint query for the same data on the PROD (nulls correct) and TEST (nulls converted to 0): I ran a few other publish tests to verify it's not just the 'overwrite' and get the same results. Here's screen shot from Pro of the features being published (source gdb) and the gdb that's extracted from the .SD file uploaded during the publish process. The nulls are being converted sometime in the 'publishing process'

‎12-11-2023

@KenGalliher1 Thanks! That's great. Unfortunately my particular case at the moment is 'traditional' versioning.

‎12-01-2023

@Bud yeah, the SQL profile is crazy lots of dynamic sql text and cursor fetches... the As&Ds look simple enough when you see a few edits on a version or two but the live data makes me think I'll miss something unless I know it's perfectly crafted. Current strategy is: SQL sproc: cursor over version names set_current_version select OIDs to temp (use editor tracking columns if it helps) merge against base temp OIDs do some tweaks for special cases ... we'll see

‎12-01-2023

Done: https://community.esri.com/t5/data-management-ideas/add-version-changes-to-the-versions-toolset/idi-p/1355811/jump-to/first-unread-message thanks again for the replies.

‎12-01-2023

ArcPro Versioning Ribbon has a great button: Version Changes. This provides a categorized list of edits in the given version in a nice little UI window. However from there, automation, advanced QA/QC, auditing, event triggering and integrations are difficult if not impossible. The data result of this button would be tremendously useful for those operations as a GP tool.... the Versions Toolset makes sense. An advanced SQL management topic or SDE stored procedure would also be ideal but I understand the complexity there.

‎11-30-2023

@MarceloMarques thanks... i knew it wasn't in the version toolbox (hoping it might be somewhere else hidden...) I'll look into the SDK, i have a vague recollection of doing something way back with the ArcObejcts SDK. I profiled the database when the button was run, hoping for some SQL insight but it's really complicated. We've got a start with some SQL comparing on the multiversion views ... just seems like that button would be useful from arcpy or sql...

‎11-30-2023

Hi All, There's an ArcPro tool called 'Version Changes' on the Versioning ribbon (see picture below). Is there a python/gp tool or SQL template to achieve the same thing? We need to implement some automated QA/QC and auditing tasks. It would be 'waaaayyyy' better if we could grab this info without asking the user to press the button. Thanks

‎10-23-2023

A Geoprocessing Service is an ideal candidate for a webhook handler since ArcGIS server implements the necessary web server endpoint and the scripting can easily access ArcGIS data and functions. However right now a GP service requires structured form values for each parameter in the POST body and won't pass thru the JSON body from the current webhook. I've tried a few options and the webhook fires the GP Service without issue ie: https://myserver.com/arcgis/rest/S123Handlers/GPServer/MyWebHookScript/execute but the JSON data is never passed thru. A simple option 'POST as form values' and either: 1. POST the entire json to a single named parameter (eg 'eventData') 2. POST each selected event data to it's own named parameter: SurveyInfo, UserInfo, etc.. If this was an option, then the GP script can easily be created to match the correct name(s) with string/text type and then parsed in the script.

‎09-15-2023

@Scott_Tansley @A_Wyn_Jones thanks for the suggestions: 1. machine is just installed so no load or anything resource hogging at this point. Basic performance stats from Task Manager: < 10% CPU utilization & < 50% Memory usage Also yes, tried a datastore service restart. 2. Output from describedatastore.bat ... nothing seems out of the ordinary expect the tile cache validates and it's member machine is PORTAL while the relational datastore's member machine is PORTAL.{domain}.com .... the FQDN has been a recurring issue for me, especially on machine that aren't joined to the domain ie dev machine and DMZ machines ... C:\Program Files\ArcGIS\DataStore\tools>describedatastore.bat Available ArcGIS Data Stores ================================================== Relational Data Store Tile Cache Data Store Spatiotemporal Big Data Store Graph Store Object Store General information of ArcGIS data stores on PORTAL.XXXX.COM ================================================== Data store release..................11.1.0.42869 Staging location....................C:\ArcGIS\arcgisdatastore\staging Log location........................C:\ArcGIS\arcgisdatastore\logs Free disk space.....................153.00GB Threshold for READONLY mode.........1024MB Information for relational data store ds_w9mc1edt ================================================== Backup location.....................C:\ArcGIS\arcgisdatastore\backup\relational Is backup folder shared.............false Backup schedule.....................{"schedule-starttime":"00:00:00","schedule-frequency":"Every 4 DAYS"} Days backup retained................7 Data store status...................Started SSL enabled.........................true member machines.....................PORTAL.XXXX.COM Maximum connections.................150 Owning system URL...................https://portal.xxxx.com/server Portal for ArcGIS URL...............https://portal.xxxx.com/portal Number of connections...............0 connection(s) to managed database Data Store mode.....................READWRITE Is Point-in-time recovery enabled...No Query optimizer enabled.............Yes Information for tile cache data store tcs_ou0nflv3 ================================================== Data location.......................C:\ArcGIS\arcgisdatastore\nosqldata Data store status...................Started Backup location.....................C:\ArcGIS\arcgisdatastore\backup\tilecache Mode................................primaryStandby member tile cache machines..........PORTAL Owning system URL...................https://portal.xxxx.com:6443/arcgis/admin Portal for ArcGIS URL...............https://portal.xxxx.com/portal Operation completed successfully.

‎09-14-2023

Hello, I've got a simple all-in-one enterprise portal setup for development/testing but the Relational data store won't validate. Thanks for any suggestions. From the datastore config wizard: From the server manager: From the server rest admin: From the server manager logs: SEVERE Sep 14, 2023, 2:47:44 PM The connection property set was missing a required property or the property value was unrecognized. Bad login user[ ] DataValidator.GPServer WARNING Sep 14, 2023, 2:47:14 PM Validate: null Data Store WARNING Sep 14, 2023, 2:41:44 PM Validate: null Data Store WARNING Sep 14, 2023, 2:19:09 PM ArcGIS Data Store is attempting recovery. Data Store WARNING Sep 14, 2023, 2:18:27 PM ArcGIS Data Store encountered too many problems; however, the standby machine 'PORTAL.XXXXX.COM' will not proceed with failover due to validation check failures. Data Store WARNING Sep 14, 2023, 2:18:27 PM Validate: Could not connect to server on machine 'PORTAL'. ArcGIS Data Store or ArcGIS Server on that machine may not be running or the machine is not reachable at this time. Data Store WARNING Sep 14, 2023, 2:18:26 PM Validate: null Data Store WARNING Sep 14, 2023, 2:18:26 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store WARNING Sep 14, 2023, 2:18:18 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store WARNING Sep 14, 2023, 2:18:11 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store WARNING Sep 14, 2023, 2:15:16 PM ArcGIS Data Store is attempting recovery. Data Store WARNING Sep 14, 2023, 2:14:26 PM ArcGIS Data Store encountered too many problems; however, the standby machine 'PORTAL.XXXXX.COM' will not proceed with failover due to validation check failures. Data Store WARNING Sep 14, 2023, 2:14:26 PM Validate: Could not connect to server on machine 'PORTAL'. ArcGIS Data Store or ArcGIS Server on that machine may not be running or the machine is not reachable at this time. Data Store WARNING Sep 14, 2023, 2:14:24 PM Validate: null Data Store WARNING Sep 14, 2023, 2:14:24 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store WARNING Sep 14, 2023, 2:14:17 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store WARNING Sep 14, 2023, 2:14:09 PM ArcGIS Data Store has detected an issue with 'Machine: PORTAL'. Data Store WARNING Sep 14, 2023, 1:54:18 PM Failed to validate the data store machine 'PORTAL'. 'Server machine 'https://PORTAL:2443/arcgis/datastoreadmin/machines/PORTAL/validate' returned an error. 'The requested resource could not be found in the system. Please check the parameters and try again.'' Admin

‎08-15-2023

More info... got fiddler running during the exchange... see pic below... Runs a few requests against the various 'portal/info' endpoints, most fail, but then the 'sharing' one works. Then does the OAUTH sequence which successfully generates an access token Then it attaches the access token as auth header Bearer xxxxxxx on the final two info requests (which are fine without any auth) and those get the 403 result.

Online Status	Offline
Date Last Visited	a month ago

My Ideas

Latest Contributions by FredSpataro

Re: Pro - Analyze Service Draft results as python object

Publishing GP service via script fails to create Tasks

Re: Publish Web Layer is converting NULL values to 0 or empty string

Re: Publish Web Layer is converting NULL values to 0 or empty string

Publish Web Layer is converting NULL values to 0 or empty string

Re: Add 'Version Changes' to the Versions Toolset

Re: Python or SQL for 'Version Changes'?

Re: Python or SQL for 'Version Changes'?

Add 'Version Changes' to the Versions Toolset

Re: Python or SQL for 'Version Changes'?

Python or SQL for 'Version Changes'?

Allow Survey123 Webhook to POST form values for connecting a GP Service

Re: ArcGIS Data Store won't validate

ArcGIS Data Store won't validate

Re: Portal 11.1: user account works with arcpy.SignInToPortal() but not arcgis.GIS()

Publish Web Layer is converting NULL values to 0 o...

Re: Portal login hangs up on accountswitcher-callb...

Re: Pro - Analyze Service Draft results as python ...

Re: Why there is no primary key for tables

ArcGIS Data Store won't validate