[MasterClass] Prepare and Transform Your Data with ArcGIS Pro

[#1]

The management and conversion of geographic data is one of the fundamental skills of ArcGIS Pro, this software has data conversion tools that allow you to centralize information from other sources within a geodatabase. This is an advantage that not only facilitates the integration of information in one place, but also optimizes the handling and management of data within GIS, improving efficiency and accuracy in your projects.

Our upcoming ArcGIS Pro Data Preparation and Transformation MasterClass is designed to introduce you to the workflows for transforming Excel, CSV, CAD, KML, and Shapefile data to Geodatabase and start optimizing your information management and conversion. Join and strengthen these skills in ArcGIS Pro!

Learning Objective

• Transform data of different formats into Geodatabase feature class and validate its topology.

MasterClass Topics

1. Geodatabase y feature dataset.
2. Preparation, cleaning and conversion of Excel, CSV, CAD, KML and Shapefile files.
3. Topology creation and validation.

PDF presentation, data and recording of the session:

• PDF Presentation
• Data
• Recording

LEARNING RESOURCES

Instructor-led training

• Introduction to GIS with ArcGIS
• ArcGIS Pro: Essential Workflows
• Create and edit data with ArcGIS Pro
• Introduction of these ArcGIS Indoors

Esri Academy

• Learning Plan - ArcGIS Pro Fundamentals

FREQUENTLY ASKED QUESTIONS TO TAKE INTO ACCOUNT BEFORE TAKING THE MASTERCLASS

Do I need to register in advance to participate?

Yes, you must register at the following link to participate in the MasterClass.

What is the date, time and duration of the session?

Thursday, August 22, 2024 from 10:00 am to 12:00 pm.

Will a certificate of attendance be given?

Yes, a certificate of attendance will be given to those who participate in the session.

I already have an ArcGIS account. Can I use it to take the MasterClass?

Depends on. Free ArcGIS personal accounts are not compatible with some tools used in the MasterClass. Also, your ArcGIS account administrator may not have enabled permissions required to take this MasterClass.

[MariaFerrigno]

Hello, why can't I see the recording??

[WillNavichoque]

Nice day... upload the recording???

[IngJuanMaSuarez]

@MariaFerrigno and @WillNavichoque the recording is already available. The post was edited to add the video. Thank you!

[WillNavichoque]

@IngJuanMaSuarez Thanks a lot!!!

[IngJuanMaSuarez]

Questions & Answers MasterClass

Questions about Geodatabase and Feature Class.

Can you share only one Feature Class from one team to another (similar to a shape that was to pass the 6 to 8 files)?

No, a Feature Class is within a Geodatabase and its model is completely different from the 6 files of a SHP. If you want to do this, you need to convert it. In ArcGIS Pro, you can share a Feature Class as a separate file by exporting it to a format such as SHP.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/sharing/overview/sharing-data.htm

Is there compatibility of feature classes with other GIS?

Natively not. GDB is the native format of ArcGIS and therefore the type of data that is there is native to ArcGIS as well. However, there are plugins that allow GDB to be read in other GIS. It is not something native, but that integration can be done. You can also export ArcGIS Pro Feature Classes to formats that support other GIS, such as Shapefiles, GeoJSON, and GeoPackage, ensuring interoperability between different GIS platforms.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/data-interoperability/what-is-the-data-interoperability-extension-.htm

In Colombia, how can I get these types of data?

In Colombia, you can get geospatial data inputs through the Agustín Codazzi Geographic Institute (IGAC), the Colombian Spatial Data Infrastructure (ICDE), and other government institutions that provide open, country-specific geospatial data.

Reference link: https://www.icde.gov.co/

How can I assign edit or read-only permissions to specific Feature Classes per user?

In ArcGIS Pro, you can assign edit or read-only permissions to specific Feature Classes through the role and privilege settings in an Enterprise Geodatabase.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/manage-sqlserver/geodatabase-administration-sql-server.htm#ESRI_SECTION1_EE7A3AD3D06642178ADDEB2DC65E1679

In a GDB I have 5 Feature Classes of the same structure and type, but separated because they are created weekly, could I add them all to a single Feature Class?

Yes, you can handle multiple Feature Classes in a Geodatabase and aggregate them into one using the "Append" tool to combine them if they have the same field structure and data types.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/append.htm

Questions about Excel and CSV data.

Can I upload a CSV or Excel file type with the domains defined?

It is not possible to define domains directly in CSV or Excel files. You must import these files into a Geodatabase and then create and apply the domains using ArcGIS Pro.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/tool-reference/conversion/excel-to-table.htm

Is it possible to bring an Excel table and convert it to a domain?

Yes, you can import an Excel table into ArcGIS Pro and then use it to create a domain. This is accomplished by importing the table into a Geodatabase and using the "Table To Domain" tool.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/table-to-domain.htm

Questions about CAD data.

What transformation methods does ArcGIS Pro use to georeference a DWS file?

In ArcGIS Pro, you can use several transformation methods to georeference a CAD file as a DWS. The most common methods include first-, second-order, and third-order transformations, which rely on polynomials to fit control points. You can also use the spline fit transformation, which is better suited to data with irregular deformations.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/cad/georeferencing-a-cad-dataset.htm

Questions about KML data.

What is the difference between a KML and a KMZ?

A KML is an XML-based file that stores spatial data such as points, lines, polygons, and attributes, commonly used in Google Earth. A KMZ is simply a compressed version (in ZIP format) of a KML file, making it easy to distribute and store by including not only the KML file, but also other resources such as images.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/mapping/layer-properties/kml-keyhole-markup-language.htm

If the KML or KMZ file is created from a folder in Google Earth is capable of storing more than one geometry, does this generate any inconsistency when transforming the data in ArcGIS Pro?

A KML or KMZ file can contain multiple geometries and should not cause inconsistencies when importing it into ArcGIS Pro. However, it is crucial to ensure that the file is well-structured and that the geometries do not overlap or contain topology errors to avoid problems in subsequent analysis.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/mapping/layer-properties/kml-keyhole-markup-language.htm

How can I export KMZ pop-up information?

The information from the pop-ups in a KMZ file can be exported by extracting the attribute data of the entities contained in the KMZ. This can be done by converting the KMZ to a feature class and then exporting the attribute table to a CSV or Excel file.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/tool-reference/conversion/kml-to-layer.htm

Do I need to define SRC when migrating to GDB?

Yes, it is important to define the Coordinate Reference System (SRC) when migrating data to a Geodatabase to ensure that all spatial data is aligned correctly. It is advisable to define the SRC using the "Define Projection" tool.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/define-projection.htm

How to generate a KML/KMZ file that allows you to view the legend in Google Earth?

When generating a KML/KMZ file in ArcGIS Pro, the legend is not included by default. However, you can manually create a caption as an image and then add it to the KML/KMZ as a screen overlay.

Link de referencia: https://developers.google.com/kml/documentation/kmlreference#screenoverlay

How to get a KML file from a public entity Geodatabase?

All Geodatabases use feature class, therefore, it is not possible to get a KML that is inside a Geodatabase directly. However, you can convert a layer to KML in ArcGIS Pro, or you can also publish the information to your website, viewer, or server to be downloaded directly to KML.

Questions about Shapefile data and domains.

Does the use of tilde affect the creation of domains or their subsequent viewing?

No, using accents does not affect domain creation or subsequent viewing in ArcGIS Pro. You can use special characters and accents in domain names and descriptions without issue.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/overview/domains-in-geodatabases.htm

Does it affect anything to use a number in the description as an encoder or is it better to use values such as 0, 1, 2?

The use of numbers or text in domain encoding does not affect functionality, but it is common practice to use numbers to encode domain values because of their ease of later analysis and clarity of display.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/overview/domains-in-geodatabases.htm

Is it possible to assign a domain in bulk, depending on the number of elements it has?

Yes, you can create a domain and set a default value. You can also use the Calculate Field tool or add Python code to bulk assign attributive information.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/overview/field-data-types-and-domains.htm

Does it limit domains from entering information into the attribute data calculator?

Domains do not limit the ability to enter information using the attribute calculator, but they do restrict the values to those defined in the domain, ensuring data consistency.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/overview/attribute-rules.htm

When there are thousands of pieces of data, how can attributes be assigned in the new column?

You can assign values to this field using the field calculator or by adding Python code.

How to cross-reference the information in a Shape file with one of ours?

To cross-reference (join) information from one SHP to another, you can use the "Spatial Join" tool if the data shares a spatial relationship, or "Join Field" if they have a common field.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/tool-reference/analysis/spatial-join.htm

Does GDB in ArcGIS Pro work the same as in ArcGIS Desktop?

Yes, Geodatabases (GDB) work similarly in ArcGIS Pro and ArcGIS Desktop. However, ArcGIS Pro offers additional tools and capabilities, such as the use of new topology rules and integration with other web services.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/overview/what-is-a-geodatabase.htm

Questions about Topology.

How does having topology rules affect my analysis and when does it not interfere with my analysis?

Topology rules help maintain spatial integrity and ensure that data meets certain geometric constraints. This can affect your analysis if the data doesn't meet the defined rules, as it might need correction. If the rules are correct, they help improve data quality without interfering with analysis.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/topologies/topology-rules-and-fixes.htm

Where is the topology tool activated from?

The topology tool in ArcGIS Pro is activated from the "Topology" panel under the "Edit" tab. From here you can create and validate topology rules on your data.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/topologies/overview-of-topology.htm

Does this topological validation only consider aspects of geographical consistency, could attribute rules be validated?

Topology rules in ArcGIS Pro focus on spatial integrity, but you can combine this with attribute rules using Attribute Rules that allow you to validate and automate behaviors in your data based on attributes.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/geodatabases/overview/attribute-rules.htm

What does "Number XY Rank" mean in topology configuration?

The "XY Rank" in the topology configuration determines the priority of a Feature Class in resolving geometry conflicts during topology validations. Entities with a higher rank prevail over those with a lower rank in overlapping conflicts.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/topologies/topology-ranks.htm

How do I modify topology rules?

You can modify topology rules at any time from the Geodatabase. You don't need to create a new topology; You can simply edit the existing one through the topology panel.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/topologies/topology-rules-and-fixes.htm

How to define the area to which you want to apply topology validation?

You can define the topology validation area in ArcGIS Pro using a "Subset Area" that delimits the specific area where topology validation will be performed, which is useful for local analysis. It is also possible to run or filter topological errors based on the map extend.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/topologies/validate-a-topology.htm

What is the difference between topologies and Data Reviewer?

Topologies in ArcGIS Pro focus on the spatial integrity of geometries and their relationship to other entities, ensuring that they comply with specific geometric rules. Data Reviewer, on the other hand, provides a broader set of tools for data review and validation, including reviewing attributes, null values, and other types of non-spatial errors.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/arcgis-data-reviewer/what-is-data-reviewer.htm

Questions about other topics.

How to disaggregate polygon databases to load into a BigQuery?

To disaggregate polygonal databases in ArcGIS Pro and upload them to BigQuery, you first need to export the data to a supported format, such as CSV or GeoJSON. These files can then be imported into BigQuery, where you can use SQL to perform spatial analysis.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/sharing/overview/sharing-data-with-google-bigquery.htm

How to georeference a raster with the Tools option directly?

In ArcGIS Pro, you can georeference raster data directly using the Georeferencing tool in the Georeference panel. Here you can adjust the control points and apply the appropriate transformation to align your dataset.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/imagery/georeferencing-a-raster-dataset.htm

What is the difference between a GeoJson, GeoPackage, and a Feature Class?

A GeoJSON is a JSON-based data format that represents spatial objects. GeoPackage is a more comprehensive spatial storage format based on SQLite, which can contain multiple spatial and non-spatial datasets. A Feature Class is a data structure used in ArcGIS to store geometries and attributes in a Geodatabase.

Link de referencia: https://pro.arcgis.com/en/pro-app/latest/help/data/arcgis-data-types/what-is-arcgis-data-types.htm

[MonicaGrajales]

Questions & Answers MasterClass

ArcGIS Pro General Questions

If we don't have ArcGIS pro? Is there a free version to learn how to use it?
There is a trial version where you will have 1 Professional Plus user type license and 400 credits for 21 days. All the information can be found at the following link: ArcGIS Pro free trial.

Questions about Geodatabase

What is the maximum capacity of the geodatabase?
The capacity of a geodatabase changes depending on its type:

• File Geodatabase: It can store up to 1 TB of data in total and supports up to 1,000 feature classes per geodatabase.
• Enterprise Geodatabase: It doesn't have a practical specific size limit, as its capacity depends on the underlying database management system (such as PostgreSQL, SQL Server, Oracle, etc.). However, the limits depend on the configuration and restrictions of the database system used.
• Personal Geodatabase: It has a limit of 2 GB in total size and can only be used with Access as a backend.

What files can be uploaded to the geodatabase?
The files that a geodatabase supports are Feature class or Feature Dataset, to enter other types of files such as KML, CAD or shapefile it is necessary to use conversion tools that transform them into files supported by the geodatabase. You can rewatch the MasterClass video where it is explained how to prepare and convert this data.

Can geodatabase personnel data be viewed in ArcGIS Pro?
To visualize data in ArcGIS Pro from a personal geodatabase, you must export the schema of the personal geodatabaase from ArcMap and import the resulting xml file into a geodatabaase file in ArcGIS Pro.

You can see this workflow in the MasterClass 1 session at minute 34, where the procedure is explained in detail. [Masterclass] Learn and refresh your GIS—from classic ArcMap to modern ArcGIS Pro. (esri.com)

Can a raster file be included within a feature dataset?
No, raster can't be included directly in a feature dataset. However, you can create a separate raster dataset that allows you to add your raster files.

How necessary is it to add domains and subtypes to a geodatabase?
It will depend on the needs of your data management, it is mainly used to avoid manual data digitization errors to create quality and consistent information. Therefore, adding domains and subtypes to a geodatabase in ArcGIS Pro is recommended, domains help avoid input errors, while subtypes allow you to organize and classify data effectively

How do I forward a geodatabase file via email?
The geodatabase file is located in a folder and has a .gdb extension. You can compress this folder into a ZIP file and send it freely via email.

When do I need to create an index in ArcGIS Pro?
Indexing is necessary when working with large datasets to improve the performance of queries and joins. This helps speed up searches and filters in attribute tables.

What options do I have to limit actions on a geodatabase file for external users who can connect, such as editing and updating data?
Multiple user editing is not possible with a geodatabase file. This function is performed through a corporate geodatabase, so editing and updating data depends on the role that the administrator assigns to the user. Learn more in the ArcGIS Enterprise https://enterprise.arcgis.com/es/server/latest/manage-data/windows/enterprise-geodatabases-and-arcgis-enterprise.htm documentation

If a piece of data in the related table (such as in a geodatabase) is modified, is it automatically updated in the feature class that was joined??
If you are working on the join directly from a geodatabase, if a piece of data in the related table is modified, it will be automatically updated in the joined feature class. If you are working from the Content Panel, the data will not be updated.

Questions about Space Reference Systems

How to change the reference system?
In the following link you will find a step-by-step guide on how to change the reference system of a map or a layer in ArcGIS Pro: https://pro.arcgis.com/es/pro-app/latest/help/mapping/properties/specify-a-coordinate-system.htm

How does the mapping of the reference system for a geodatabase and for a feature class work?
A geodatabase can include files that have different reference systems. Whereas in a Feature Dataset, only files that have the assigned reference system are supported.

What are the coordinate systems for Colombia?
According to IGAC Resolution 471 of May 14, 2020, in paragraph i of article 4 is the cartographic projection system for Colombia, called MAGNA-SIRGAS National Origin, which has a single origin and consists of a Transverse Mercator cartographic projection.

More detailed information on the official IGAC website: https://origen.igac.gov.co/index.html

How to make a projection to a feature?
To project a feature, use the "Project" tool in ArcGIS Pro. This tool allows you to specify the output coordinate system and perform the projection of your data efficiently. Note the tool parameters in the following Esri documentation: https://pro.arcgis.com/es/pro-app/latest/tool-reference/data-management/project.htm

Questions about Excel

How do I upload an Excel sheet to ArcGIS Pro?]
To upload an Excel sheet to ArcGIS Pro, follow these steps:

1. In the "Catalog" panel, right-click on "Folder Connections" and select "Add Folder Connection" to connect the folder containing your Excel file.
2. Navigate to the Excel file and double-click on it. A list of sheets will open within the file.
3. Right-click on the sheet you want to add and select "Add to Current Map" or "Add to New Map" depending on your preference.

When I upload an Excel to ArcGIS Pro can it only have 1 sheet?
No, ArcGIS Pro can import multiple sheets from an Excel file. However, it is recommended that you work with well-structured sheets to facilitate data import and handling.

The first row of the Excel file should be the name of the columns, could you clarify this point?
Yes, in ArcGIS Pro, the first row of the Excel file must contain the column headers, as these are treated as field names when imported into ArcGIS Pro, similar to how fields and records are displayed in the attribute table.

Does ArcGIS Pro recognize all versions of Excel?
ArcGIS supports both Excel files from .xls 2003 and earlier and Excel files from .xlsx 2007. An advantage of Excel 2007 is that it allows for much larger worksheets (1,048,576 rows by 16,384 columns) than you can have in Excel 2003 (65,536 rows by 256 columns).

If you have coordinates in an Excel file, how do you upload them to ArcGIS Pro? Will there be any conflicts with the decimal symbol (semicolon)?
You can use the XY Table To Point tool in ArcGIS Pro to create point features from Excel coordinates. It's critical to make sure the decimal separator in the file matches the system locale (semicolon), or adjust the settings if necessary.

Questions about unions and relationships

In what situations is it appropriate to use union or relationship?
A join combines attributes from a table to a feature class based on a common field. You can use it when you have data in a table that you want to analyze or visualize along with the attributes of a feature class and the relationship is one-to-one or one-to-many.

A relationship connects two tables based on a common field, allowing both tables to remain separate. You can use it when you're working with data that has a one-to-many relationship, and you want to keep the tables separate, allowing flexible access to related records without merging them.

Do both datasets need to be in Excel format to join?
No, both datasets do not need to be in Excel format. You can join an Excel table to a feature class or other formats, as long as they share a common field.

What happens if there is a discrepancy in a row or data?
If there is a discrepancy in a row, it might not affect the overall joint. However, discrepancies in key fields (such as unique IDs) can cause problems during the joining process.

After joining, does the information brought in from Excel remain permanently in the entity class or does it disappear when you close the project?
The join is temporary, unless the entity class with the joined data is exported. If it is not exported, the union will disappear when the project is closed.

How stories are displayed in ArcGIS Pro and how do you select from the table of many?
To view a relative, open the "Attribute Table" and use the "Related Data" option to view the records. It is possible to select related records using the "Select Related Records" tool.

Questions on other topics

How can you display points if they do not have coordinates?
If the points do not have coordinates, they must be geocoded or the coordinates added manually. It is also possible to use tools such as "Add X/Y Coordinates" if the location information is based on attributes such as addresses.

How do I link photos to a point layer in ArcGIS Pro?
You can link photos to a point layer using "Attachments" in ArcGIS Pro. In the "Data" tab, you can use the "Manage Attachments" tool or set up a hyperlink field that points to the path of the images. In the Esri documentation, you can follow this step-by-step process https://pro.arcgis.com/es/pro-app/latest/help/editing/edit-file-attachments.htm

When I work on a topic in Power BI and create the map in ArcGIS, what do I need to do to publish that information in ArcGIS?
You can use the ArcGIS add-in for Power BI to import the map directly or share the map through an ArcGIS Online link or service. All related information can be found at: https://doc.arcgis.com/en/microsoft-365/latest/power-bi/publish-to-web.htm