Latest Contributions by MaxPayson

Nice that's great! Glad it is working

Hi @Amanda__Huber , I'd recommend opening a tech support case here to investigate the issue. In general, the error implies that the feature layer queries made by Data Pipelines are failing. Given that it is working in other apps, my hunch is that the layer server does not have enough resources to respond to the queries, especially if the layer has many features with complex geometries. This may not happen in other apps because they only need to query subsets of the data. For example, Map Viewer has optimizations to only query the data for the current map extent with features simplified for the current scale, while Data Pipelines processes the entire layer with full resolution geometries. That said, it'd be good to work with support to troubleshoot the specifics. I would also try adding a tool like Select fields - if the fields show up in the form, this verifies that Data Pipelines can at least access the layer and it is related to the underlying feature queries.

Hi @IB3 , from the Open API specification in the documentation you linked, it looks like the API key needs to be included in a header named "X-Api-Key". You can do this by following the documentation here, and specifying the following parameters for the service connection: Authentication type: API key Parameter location: Header Parameter name: X-Api-Key Then for the geometries, do you know what the data schema is, or could you provide an example? It looks like the response format is JSON and I'm wondering what specific fields and data types that JSON contains. You can get the full schema by adding the URL input, clicking "Preview", then clicking "Schema", then clicking "Expand all". There's a clip of this in the release blog under the "Enhanced schema preview" section.

‎03-18-2025

Hi @BlakeMorrison, one-to-one joins create summarized values for all "Join" dataset records that match a single "Target" dataset record, so it doesn't return the "Join" attribute values themselves. By default, the only summary statistic is the count, but more can be added with the "Summary fields" parameter. Then one-to-many joins create a new record for each matching "Join" dataset record, but they don't summarize so they don't include the count. A couple ideas to accomplish what you described: For the 0 match case, use "One to many" and "Left join", then use the Filter by attribute tool to filter to the records where the joined fields are empty (no match) or where they are not empty (match) For the generic case, add a "One to one" join to create the count, then add a "One to many" join with the original dataset and "One to one" output as inputs. This should add the count back to the original records Let us know if this helps! And thank you for the question

‎02-18-2025

Hi @RyanKelley_NA - we're still looking at this and have made some foundational improvements but don't have a timeline. What tools and processing are you using in your data pipelines? The trickiest part is defining if and how attachments propagate through integration tools such as Join and Merge. There is some nuance for the other tools as well. Any details you can share about your workflows and desired functionality would be greatly appreciated! Please feel free to reach out via email as well.

‎11-14-2024

Hi Royce, thanks for the question! We made some related updates in the release this week: The "Create" method now has an "Overwrite" option that, when selected, will recreate the entire hosted feature layer if it already exists (doc & considerations) When switching from "Create" to "Replace" after the data pipeline is run, the "Replace" layer will automatically populate with the output from "Create" (this was inspired by your post) Our recommendation is to use "Create" while authoring and "Replace" for automated workflows. "Replace" is quite optimized for automation; it preserves the schema to avoid breaking downstream apps, it rolls back on failures, and it has negligible disruption (via a table swap). That said, "Create" with "Overwrite" enabled can also work if you want the layer schema to change. I would author this all within one data pipeline because it will be easier to switch from "Create" to "Replace" for the newly created layers. Hope this is helpful and please continue to share feedback as it comes up! Thank you, Max

‎10-18-2024

Hi Royce, thanks for the question! To run a subset of the outputs in a data pipeline, you can select the desired outputs then click the "Run" button in the context menu that appears over the output element (red arrow). Multiple outputs can be selected by holding "Shift" and clicking on each one individually, or by dragging the mouse on the canvas over the elements with the "Select elements" interaction enabled (gray cursor button in the top right).

‎07-06-2024

Hi, thanks for the question! To get at this data, use "data" as the root property, then add two Unnest tools to flatten out the fields. The root property currently does not support accessing array entries (for example, "data[1]"). Here's a screenshot:

‎05-02-2024

Hi @Levon_H , thank you for the idea! Where are your file geodatabases stored? We are actively working on support for reading feature classes from file geodatabases. Like other file inputs, the files will need to be stored in an accessible and supported location (AmazonS3, Azure Storage, or uploaded locally to Online), so I'm curious if this will fit your use case? Also, if there are more details you can share about your current workflow (is there intermediary processing, is it updating multiple layers in a single service, etc), that would be very helpful as well!

‎05-01-2024

Hi @maranlk & @ShanaCrosson2 , we're starting to investigate this feature. Would you be willing to meet and share more about your use cases? If so, please email me and we can schedule a time, mpayson@esri.com, thank you!

‎01-29-2024

Hi @maranlk , thank you for the idea, and for all the context! Do you have additional ideas or requirements for managing attachments from within Data Pipelines, and the experience for this? For example, in previews, would you expect to be able to view the attachments, or would it be sufficient to have an indication that there are attachments? Any feedback here is greatly appreciated!

‎12-08-2023

Hi Romain, Thanks for the question! Can you share a link to the public data so we can investigate the structure? Is it a single file, with a top-level array of feature collection objects? Such as: [ {"type": "FeatureCollection", "features": []}, {"type": "FeatureCollection", "features": []} ] Best, Max

‎12-07-2023

Hi Jennifer, Is your Snowflake connection only accessible from an internal network, and not from an external service like ArcGIS Online? Are the other apps you are using running within your network? This is one idea. I also have a few more ideas to investigate if you'd like to follow up over email, my email is mpayson@esri.com Thank you for trying the beta and posting your issue! Max

‎04-12-2018

Hi Sarthak and Uma, Quick update, the sample is available here: https://github.com/Esri/arcgis-pro-sdk-community-samples/tree/master/Editing/SketchToolDemo Thanks, -Max

‎10-20-2017

This blog follows Acceptance & Onboarding, Access EPN Partner Benefits is next in the Orientation Series Access Your Software *Please read this closely to avoid headaches down the road & reference our Technical Alignment blog* This page describes how to structure, access, and administer the software provided to your team through the Esri Startup Program, which provides a robust software benefits package. It's important to carefully admin and track your order emails, accounts/subscriptions, and usernames. We suggest you invite key staff to My Esri as soon as possible. Overview Esri System Destinations There are three important destinations to access and administer your software: My Esri: Start here! Connect to your Esri Customer #, For your admin(s) and technical POCs. A destination for software downloads and authorization numbers to install and activate your software licenses and admin your organizations relationship with Esri. Manage user permissions for tech support, training, orders/purchasing, activation & case management reporting, and other important access points for your team. You can Link accounts to easily switch between multiple Esri ArcGIS accounts. ArcGIS Online (arcgis.com): Esri’s GIS SaaS software per-user/credits. Once activated, you need to link your subscription to your My Esri. Also, ArcGIS Online provisions licenses for per-user licensed software, such as ArcGIS Pro. Individual user type members need to be added, invited to the organizational subscription. Based on roles/privileges, members of the organization can create and share maps, apps, and other content; explore data; and publish data as hosted web layers. Configurable apps are deployed using ArcGIS Online or Enterprise vs custom apps using ArcGIS Developer SDKs/API Keys (Note, ArcGIS Enterprises also supports per-user type licensing model, if you would like to use in your deployment contact your partner manager for more information) location.arcgis.com: Sign up for a free ArcGIS Location Platform Account and enable pay-as-you-go-pricing to leverage ArcGIS Location Platform services via API Key or register your applications via your dashboard to get started. Access Esri Developer resources to build your offering. Software Setup Within 3-7 days of receiving your email, 'Welcome to the Program', the team member that applied will receive 3 critical software emails from the following senders, Service (service@esri.com), Esri, and My Esri (no-reply@esri.com): My Esri, provides instructions for the primary contact to connect to your organization and customer number within My Esri using a token. Admin needs to invite staff (users) and delegate permissions to access software or tech support. Esri Order XXXXXXX, PO XXXXXXXXXX, provides a link to activate My Esri as well as reference to all software license activation keys, which are also managed within My Esri and found there automatically linking to your customer #. Esri – Activate Your ArcGIS Online Subscription, contains an activation link for your new commercial ArcGIS Online Organization, with numerous user types and 10k credits - including add-on apps like ArcGIS Pro, Configurable Apps, and more. This should be used as your primary commercial staging for the production subscription account (unless you plan to only deploy an app using ArcGIS Location Platform pay-as-you-go services consumption pricing via API Key and the dashboard). Accounts Every team member will have at least one or multiple ArcGIS username logins from either being an added user in both, or either ArcGIS Online or My Esri- the user account should be invited or request to connect to your organization unique customer number to gain the proper admin rights. Each subscription also results in a unique set of credentials (user types/named users). We recommend using a spreadsheet and recording these credentials for reference. We also recommend organizing your individual team member’s accounts as follows: My Esri Credentials: For team members who need administrative privileges or direct access to the software. ArcGIS Online Organization: Your SaaS commercial subscription and should be used as your DEV/TESTING & STAGING ENVIRONMENT for team members who will help define production application or content/data services - (unless you only plan to use Location Platform - PaaS). Includes user types and credits. Carefully read Understand Credits docs and consider budgeting. Additional ArcGIS Online subscriptions for dev/test must be purchased via the Creator User Type. ArcGIS Location Platform Account: Provides access to a developer dashboard to register applications or utilize ArcGIS Location Platform consumption pricing for location-based services (platform as a service (PaaS)). While it can be linked to ArcGIS Online user account - it should not without a discussion with your partner manager. See FAQ Upon Request: [ArcGIS Developer Bundle for Dev&Test of ArcGIS Enterprise/Extensions]: Contact your partner manager to investigate access to the Developer Bundle to Dev & Test ArcGIS Enterprise. This is for team members who are developing, testing apps or content who need to demo noncommercial per-user licensed software, such as ArcGIS Pro, ArcGIS Enterprise. If you or your team members have a prior Esri ArcGIS Login account, make sure to create new accounts as specified below--it makes life simpler to separate concerns with separate accounts. Before each step, make sure you are logged out of all accounts to avoid confusion. If you have previous accounts with existing content, work with Esri Technical Support to migrate them to your new accounts, or call to troubleshoot these steps. 1: Activate and Configure My Esri The following assumes you have access to the Esri Order email and are setting up MyEsri for your team. 1. My Esri Account Login:Open the Esri Order email, click the link to log into My Esri, with your ArcGIS username login, you can use the same login when you signed up for the program, this associates My Esri with your customer number or create a new account. Need help? Contact Customer Support if you need assistance linking your customer number to your My Esri Account 2. Invite your Team Members & update permissions: Go to “My Organizations” -> ”Users” -> “Invite Users” and follow the instructions. Here you can add multiple users by email, configure their permissions, and annotate the invitation. Be sure to activate *Note, we recommend your team members create an ArcGIS Username login dedicated to your company to avoid confusion. Issues? Sign out of previous accounts and restart the browser. Update Permissions: Be sure to enable the proper permissions, such as access to tech support. in order for a team member to log a case with Support, they must be registered as an authorized caller. Manage Support Cases: Go to “Support” manage cases here, or access the external site. 2: Access Software The following assumes you have been provisioned a MyEsri account per Step 1 and have appropriate permissions. Access the software: Login to My Esri and go to “My Organizations”-> "Licensing". Downloads are available under the “Downloads” sub-tab and you can manage license provisions under the “Licensing” sub-tab. NOTE - DEPRECIATED (Developer Subscription Plan It is no longer part of the Startup Software Package as the plans have been deprecated for ArcGIS Developer Subscription deprecation supporting a migration process. Please refer to the following FAQs to understand the impact of the deprecation, available alternative ArcGIS Developer Bundle may be purchased or, on a case by case, basis granted; talk to your Partner Manager. 4: Activate and Configure ArcGIS Online Subscription The following assumes you have access to the ArcGIS Online email and are setting up the commercial staging/production Organization for your team. We recommend you read and reference "Get Started with Administration" documentation. Activate the Organization: In the ArcGIS Online email, first, click the link then create a NEW account and follow the instructions. This creates the ArcGIS Online account. Configure the Organization: Fill out the forms to configure your Organization’s preliminary settings. Click “Save and Continue.” You will be taken to the administrative landing page. Invite Team Members: Click “Invite Members” in the “Organization” administrative console and follow the steps provided. Enable Esri access. This ArcGIS Online documentation has instructions for the administrator on how to change this setting. Organization administrator can Enable Esri Access for all org members. This blog topic has some good information in the comments section about this setting. Use ArcGIS Online: There are no additional software configurations for ArcGIS Online. ArcGIS Online is a credit-based model and the credits provided through the Program will be consumed as you use Online services. It is important to read through Understand Credits documentation and consider enabling credit budgeting. View ArcGIS Online Subscription Info in My Esri: "Organization"-> "Liscensing"-> ArcGIS Online" but Manage ArcGIS Online via www.arcgis.com. *Remember to track and save your usernames, subscription IDs, and accounts* If you have questions, we suggest you first troubleshoot the issues with Esri Customer Service or Tech Support, you have access to Standard Tech Support via the program. They have back end views into the systems. For more in-depth inquires, email your partner manager, local distributor, or contact startups@esri.com. Blog Return: Acceptance & Onboarding. Next: Access EPN Partner Benefits. Program Benefits Event Benefits Acceptance & Onboarding Access Your Software Access EPN Partner Benefits Patterns for Success ArcGIS Technical Alignment

Online Status	Offline
Date Last Visited	a week ago

My Ideas

Latest Contributions by MaxPayson

Re: ArcGIS Data Pipelines- Stored Credential Layer Error Out

Re: ArcGIS Data Pipelines- Stored Credential Layer Error Out

Re: Help Needed: Accessing API

Re: Join Count attribute

Re: Preserve hosted feature layer attachments

Re: Create Vs Replace

Re: Can I control which parts of a pipeline get run?

Re: Correct root property path for input JSON

Re: Include Feature Classes as source and output options

Re: Preserve hosted feature layer attachments

Re: Preserve hosted feature layer attachments

Re: GeoJson with multiple feature collections

Re: Snowflake Data Pipeline in ArcGIS Online

Re: Create a tool to split rectangles

Esri Startup Program Orientation: Access Your Software

Re: ArcGIS Data Pipelines- Stored Credential Layer...

Re: Create Vs Replace

Re: Can I control which parts of a pipeline get ru...

Re: Correct root property path for input JSON

Esri Startup Program Orientation: Access Your Soft...