Data Loading Tools - Version 2 Preview

3444
14
08-21-2020 12:47 PM

Data Loading Tools - Version 2 Preview

The Data Loading Tools streamline loading data from a source to a target dataset with the ability to perform in-flight data transformation. If you are not already familiar with the tools check out The Data Loading Tools - Video Series.  

Since the initial release of the tools, we have added predictive mapping capabilities. These new capabilities will help automate the creation of the Data Loading Workspace, saving you time.  This preview includes an update to the Create Data Loading Workspace tool and the new Generate Custom Mapping tool.

Create Data Loading Workspace

The Create Data Loading Workspace geoprocessing tool user interface.This tool creates a Data Loading Workspace. The workspace contains folders, Excel workbooks, and other items to assist in mapping data from a source to a target schema.

What’s New: 

Dataset Matching  The Source to Target mapping can now match similar datasets in a geodatabase or feature dataset automatically. You can also manually specify each source and target dataset or explore the custom mapping capability.

Match Options (optional) – These options match fields and values from a source dataset to the target schema based on their similarity.

Custom Mapping (optional) – Matches datasets, values, and fields based on a user-defined input (table or CSV). This is best suited when the source and target schema are not similar.

The Generate Custom Mapping tool user interface.Generate Custom Mapping

Export a configured Data Loading Workspace using the Generate Custom Mapping tool. This tool generates a Custom Mapping file that can be used in the Create Data Loading Workspace tool to map datasets, fields, and values from the source and target that are not alike. This saves you from manually migrating your configuration when creating a new Data Loading Workspace and can also be shared with others to create a Data Loading Workspace that is comparable.

Install the preview
Before you begin installing the Data Loading Tools toolbox, the following is required:

 

To install the toolbox using the Python Package Manager, complete the following steps:

  1. In the Windows Start menu, browse to the ArcGIS folder.
  2. Click the drop-down arrow next to ArcGIS and click Python Command Prompt to open the window.
  3. Verify that the first line of code displays the name of your cloned environment before continuing.
  4. Once confirmed, type conda install -c solutionsdev/label/preview dltsolutions and press Enter.
  5. When the installation is complete, close the Python Command Prompt and start ArcGIS Pro to confirm that the Data Loading Tools toolbox has been added.

       Note: If you already have the Data Loading Tools, installing the preview will overwrite them.

Change Log 

To get the latest release of the preview, follow the steps for installing the toolbox again. 

DateDescription
8/21/2020The first release of the Data Loading Tools Version 2 preview
9/01/2020This release includes bug fixes and documentation for the tool user interface.

Let us know what you think by commenting below.

Comments

Hello. I think the new updates are great. Very useful. Although, I'm not sure if I'm doing something wrong, but while using the updated Create Data Loading Workspace tool, when I map something like Plan_Structure (polygon) to StorageTank (polygon), or Trans_Road_Center (polyline) to RoadCenterline (polyline) I can't generate the DataLoadingWorkspace folder, but Plan_Fence (polyline) to Fence (polyline) works fine. I don't get any errors or warnings indicating that there's an issue. These worked fine in the previous version of the tool. I'm wondering if I'm missing a step. Any help with this is appreciated. Thanks.

Hello Evaristo, 

I touched base with the lead developer and we are making updates to fix this issue. I will let you know when the tools have been updated. 

Thanks,

Gelisa 

Evaristo,

Your issue has been fixed please run through the installation steps again to get the latest tool updates. Additionally, we plan to release updates to this toolbox at the end of the month. Check out the documentation here for more information.

Thanks,

Gelisa

Gelisa,

I appreciate the quick response. I will update the tool and test it out.

Thank you,

Evaristo

Hello again. I tested the previous scenario with the updated tool, and it worked great, but I'm now running into an issue with using the custom mapping option. When selecting a custom mapping table (which I generated from a previous data loading workspace), I don't see a way to select the source gdb and target gdb, I can only select a specific feature class. If I click Run without select these options, I get the message "Parameter is missing or invalid". I'd also like to create a custom mapping table from scratch in excel and use it as input to generate the data loading workspace. Is that possible without first having a dataReference document to generate the custom mapping?

I look forward to seeing the next updates to this tool.

Thanks for all your help and have a nice day,

Evaristo

I'm sorry. A quick correction. I am able to select a geodatabase target, but not a geodatabase source.

Evaristo,

When selecting a custom mapping table (which I generated from a previous data loading workspace), I don't see a way to select the source gdb and target gdb, I can only select a specific feature class.

Do you mean that you can not select a container (geodatabase or feature dataset) when doing the Source to Target mapping? Currently, if you would like to add a database you will need to drag it into the source/target from the catalog or select a layer and remove the layer from the path. We currently have an issue logged to resolve not being able to select a container from the browse dialog. 

You can create a CSV with the schema as type,phrase1,phrase2 

type can be dataset, field, or value 

I do want to let you know that with the release of this toolbox at the end of the month there will be some big changes. We are going to make updates to the user interface/naming of parameters but the toolbox will still provide the same functionality as you are currently testing.

Please let me know if you have any other questions.

Thanks again for the quick response. This is very helpful.

Evaristo

I wasn't sure where to ask this question, but has there ever been a similar tool (data loading tools) for ArcGIS Pro version 2.2? I'm asking for a friend. @GelisaJoseph 

@EvaristoRamos As far as I am aware, both the Data Loading Tools and the Data Assitant Add-In are only available starting with ArcGIS Pro 2.4.

I have Data Loading Tools (untools version 2.7), I just upgraded to ArcGIS PRO 2.8.2.  the Data Loading Tools disappeared.  When is the untools new version compatible with Pro 2.8 coming up?

@ELIZABETHF_T_ 

When you upgrade ArcGIS Pro, you need to recreate your python clone environment and re-install python packages. Please follow the steps to Install the Data Loading Tools—Get Started | ArcGIS Solutions .

The untools 2.8.x should be available now: Install the Utility Network Package Tools toolbox—Get Started | ArcGIS Solutions

After upgrade to ArcGIS Pro 2.8.2, I tried to recreate the python clone env.  I get a proxy Error, Any idea how to fix Proxy Error?

ProxyError Conda.png

@ELIZABETHF_T_  I'm not too familiar with how to resolve this issue, but this related post may help: Installing untools on ArcGIS Pro 2.4 behind a corp... - Esri Community

Version history
Last update:
‎08-21-2020 12:47 PM
Updated by:
Contributors