Suzanne-Boden
Esri Regular Contributor
since ‎07-01-2014
41 Badges Earned
Expert Discoverer
View All Badges Earned
Friday
496 Posts created
162 Kudos given
55 Solutions
553 Kudos received
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

My Ideas

(0 Idea Submissions)

Latest Contributions by Suzanne-Boden

BLOG
1
6
21422
BLOG
0
0
1362
BLOG


Lacking the ability to publish a map service myself, I opted to go the minimalist route—that is, use the ArcGIS Online map viewer. If you have access to an ArcGIS Online organizational account, you can map your data right inside Excel using Esri Maps for Office.

A viable alternative for those without an organizational account is to add a .csv file (text file of comma-separated values) to the map viewer and save the map to their ArcGIS Online public account. An Excel spreadsheet is easily saved as a .csv file. To be mapped, the spreadsheet must have fields that store location data—latitude and longitude values, GPX coordinates, or addresses.

I wanted to visualize the locations of individuals in the U.S. who attended the Creating Hosted Map Services with ArcGIS Online live training seminar. Understanding the geographic distribution of our viewers is useful to evaluate seminar broadcast times. All I had to do to create a web map was follow these easy steps.Step-by-Step Example
  1. Open the Excel spreadsheet and save it as a .csv file.
  2. Go to www.arcgis.com and sign in to your public or organizational account.
  3. Click the Map link at the top (or bottom) of the page.
  4. In the map viewer, zoom to the data's extent.
  5. Keep the default Topographic basemap or click the Basemap Gallery button and choose a different basemap.
  6. To the left of the Basemap Gallery button, click Add > Add Layer from File, browse to the .csv file, then click Import Layer. (If you're using Firefox or Chrome, you can just drag and drop a .csv file into the map viewer.)
  • Note: You may get prompted to specify which fields contain location information (e.g., your address or lat-long field names). There is also a 250-feature limit when adding .csv files to the map viewer.
PointsOnMap-300x198.png

Point features can be drawn on the map with a single symbol, unique symbols based on an attribute, or as a heat map. To quickly see the point distribution, I'll choose to use a single symbol for all the features. Clicking Select under Location (single symbol), then Done displays the points on the map.

In less than two minutes, I'm visualizing my data as points on a pretty map. I can click each point and see the associated data from the Excel spreadsheet in a pop-up. Easy. Except...

I only want to show certain columns, and I don't like the names of those columns. I also decide I'd like to symbolize the points by self-reported industry.

I can address all of these issues right in the viewer.
  1. In the Contents pane to the left of the map, mouse over the layer name and click the arrow next to it, then click Change Style.
  2. Under Choose an attribute to show, choose the field that stores the values you want to show.
  3. For the drawing style, select Types (Unique symbols).

In the symbol preview, I notice an industry category named Other. For my purposes, Other is the same as no data. It's easy to change the Other symbol label.
  1. Click Options, click the label next to the Other symbol, then type a new label (No Information is more understandable for this map). If I wanted, I could click the symbol and assign a different symbol to distinguish this category.
  2. Click OK, then Done.
  • UniqueValuesMap.pngTakeaway #1: Explore your data and decide how you want to present it before putting it on a map. In the map viewer, it's easy to change the display properties, but you can't modify the data itself.

So, in just a few more minutes, I've mapped viewer distribution and self-reported industry. Next, I'll configure the pop-ups.
  1. In the Contents pane, click the arrow next to the layer name again and choose Configure Pop-up.
  2. Enter a descriptive pop-up title or select an attribute to use as the title.
  3. Click Configure Attributes and uncheck fields you don't want to display, then make sure the fields you do want to display are checked. You can also use the arrow buttons to reorder the fields. Be sure to type a friendly alias for each field that will display, then click OK.
  4. At the bottom of the Configure Pop-up pane, click Save Pop-up. PopupInfo.png

Now when I click points on the map, I see only the information of interest. I also notice something.

When I zoom in to a large scale, the Topographic basemap gets very detailed. Building footprints and local streets display. Some of my symbol colors blend into the basemap features.

Switching to the Light Gray Canvas basemap is a quick solution. This basemap, with its subdued colors and less detail at large scales, is a better backdrop for my data. LightGrayCanvasMap.png
  • Takeaway #2: Before deciding on a basemap, explore it at large (and small) scales to understand the level of detail and assess its appropriateness for your mapping purpose. It's important to choose symbol colors that will stand out against the basemap at various scales.

Finally, I want to show time zones on the map. I don't have a layer of time zones but I'll search ArcGIS Online content to see if anyone has shared this data.
  1. Click Add > Search for Layers.
  2. In the Find text box, type time zones, choose to search in ArcGIS Online, then click Go.
  3. The search results include several time zone feature services. After adding one of them to the map, I change its symbology to unique symbols so each zone will have a different color and I increase the layer transparency value so the time zone features don't obscure the other data. TimeZonesOnMap.png

In less than 20 minutes, I've "mappified" my Excel data and added context to it. To generate a link to the map that I can embed in a web page, e-mail, or social media post, I just need to save the map to my public account, add a title, some tags, and a brief description, then share the map.
  • Takeaway #3: There are lots of ways to create, save, and share a web map these days. If you don't have access to software for creating GIS services, you're not out of luck. Creating a high-performing web map with the ArcGIS Online map viewer really is an easy—and useful—method.

Related posts: -->
0
0
1841
IDEA
-->
0
0
3251
BLOG
0
0
2415
BLOG
0
2
1433
IDEA
-->
0
0
1507
BLOG
0
0
837
BLOG
0
0
667
BLOG
Exploring Data

The first part of step 2 flows directly from the analysis question. Spend time upfront identifying all the data, including attributes, needed to answer the question. Don't get halfway through a project and discover you're missing a key dataset.

The point of exploring data is to understand what valid things you can do with it. You should:
  • Examine the metadata. Note the spatial resolution and accuracy, coordinate system, when the data was collected and by whom, data use constraints, and other important information.
  • Explore all the spatial datasets you plan to use in ArcMap. Do the layers align properly together? Are some layers more generalized than others? Do some layers have a larger (or smaller) extent than needed?
  • Explore the attribute table for each layer and note the number of records and the attributes. Sort fields and look at field statistics to understand the values. Note obvious data entry errors and inconsistencies.
Preparing Data

Preparing data means ensuring datasets can be validly analyzed together and reducing processing time as much as possible. Data preparation tasks often include projecting data, reducing the spatial extent to the area of interest, deleting unneeded attributes, creating new attributes, cleaning up attribute values, and more.

Get the most out of ArcGIS geoprocessing tools. Many can be run in batch mode and some have options you can use to combine data preparation tasks. For example, inside the Feature Class to Feature Class tool, you can create a SQL expression to import only features located in your area of interest, you can exclude source attributes you don't need, and you can define new attribute fields that you do need. With one tool, you accomplish three preparation steps.

You can also create a model to automate data preparation—just drag the geoprocessing tools you need into a model window and define their parameters. Creating a model is an excellent way to visualize and order tasks as you go.

The same data may be used for multiple analyses. It's good practice to make a copy of the original data before you make changes to prepare for a specific project. This way, the original data is preserved for subsequent analyses—and you have something to go back to just in case.Example: Analyze Piracy Incidents

Let's look at a simple example to illustrate the importance of exploring and preparing data. The analysis question has been framed as:
  • In the years 2009 through 2011, what types of vessels were the most frequent victims of piracy in and around the Gulf of Aden and Arabian Sea?

The U.S. National Geospatial-Intelligence Agency (NGA) distributes Anti-Shipping Activity Messages (ASAM) reports, which include locations and descriptions of hostile acts against ships and mariners worldwide. You can download ASAM data in shapefile format from the NGA website.Explore the Data

After downloading the ASAM shapefile, add it to ArcMap. For geographic context, it helps to add a basemap; in this case, the World Topograpic Map from ArcGIS Online works well.

The map graphic on the right shows the global extent of the ASAM data (symbolized with red dots). A quick exploration of the attribute table reveals 6,158 records; an attribute that stores a world subregion code; and an attribute that stores the date each incident occurred—the date range is 5/1/1978 through 1/9/2012.  ASAM_fullextent1.png

To efficiently answer the analysis question, you need to reduce the data to incidents that occurred between 1/1/2009 and 12/31/2011 in subregion 62 (the NGA website lists subregion codes). You also need to do some other preparation work to resolve issues with this data.Issue 1: The data has no spatial reference.

When the ASAM shapefile was added to ArcMap, a message that the data was missing a spatial reference displayed. For shapefiles, spatial reference information is stored in the projection (.PRJ) file and it's not unusual for the .PRJ file to be missing, as it is here. So how do you know which coordinate system to use?

A good place to start is the Layer Properties dialog box. In the Source tab, when you look at the extent coordinates, the number of digits to the left of the decimals tells you it's a geographic coordinate system. ASAM_coordinates1.png
  • With geographic coordinate systems, the Left and Right extent values will have one to three digits to the left of the decimal, while the Top and Bottom extent values will have one or two digits to the left of the decimal.

Since the data has a global extent, it is reasonable to assign the WGS 1984 geographic coordinate system. If your analysis involved precise measurements, you would want to assign a suitable projected coordinate system for the area of interest as well.Issue 2: The data is more extensive than you need.

The ASAM data has a larger spatial extent and a longer temporal extent than you need. SQL queries will resolve these issues.

For this analysis, the initial data preparation tasks are:

1. Create a file geodatabase to organize the project data.

2. Import the ASAM shapefile into the file geodatabase.
  • Import only features within subregion 62 that fall within the date range 1/1/2009-12/31/2011.

3. Assign the WGS 1984 coordinate system to the geodatabase feature class. DataPrepModel1-300x174.png

The model graphic on the right shows the geoprocessing workflow.

A SQL expression was defined in the Feature Class to Feature Class tool dialog so only the incidents meeting the analysis criteria will be imported.
After running the model, you have a new feature class with 643 features. Now that the data is a more manageable size, before moving on to step 3 of the analysis process, you need to verify that all 643 features represent piracy incidents that occurred during the analysis timeframe.

In ASAM data, two fields contain date information (Reference and DateOfOcc). DateOfOcc stores mm/dd/yyyy, while Reference includes the year followed by a unique ID number. The Reference field was used in the Feature Class to Feature Class dialog's SQL expression for convenience, but it means there's cleanup to be done now. Sorting the DateOfOcc field reveals that four records have 2009 in the Reference field (reflecting the year the report was submitted), but their DateOfOcc values tell you the incidents actually occurred during the last week of 2008. These records can be deleted. Now there are 639 records to analyze.Issue 3: Attribute values are inconsistent.

Next, you need to make sure all records are incidents of piracy. The Aggressor field stores this information. Sorting the Aggressor field reveals multiple values, the vast majority of which contain some variation of "pirate."  UniqueValuesLegend1.png
  • Tip: A quick visual method to understand how the Aggressor values vary is to open the Layer Properties dialog box and symbolize the layer by Unique Values based on the Aggressor field. Each unique aggressor value displays and the Count field tells you how many of each.

After exploring the Aggressor values, you can create an attribute query to select all records that have one of the "pirate" variations in the Aggressor field, then switch the selection to see how many records don't directly reference pirate aggressors. In this case, only 16 incidents have Aggressor values that don't reference pirates.

You need to explore the incident descriptions to determine whether any of these 16 likely were attacks by pirates. If they clearly did not involve pirate aggressors, delete the records. In this case, the incident descriptions don't clearly rule out pirate aggressors, so you choose to keep the records but categorize them to account for the ambiguity. You use the Field Calculator to change those 16 Aggressor values to Possible Pirates, and you clean up the remaining records so they all have an Aggressor value of Pirates.Issue 4: The table has duplicate records.

With the DateOfOcc field sorted, a look at the Reference field shows something alarming. There seem to be many duplicate records—records with the same Reference value and identical data in the other attribute fields. How could this be? Did something go wrong when you converted the shapefile to a geodatabase feature class?  DupValuesTable.png

This is the time to go back to the original data and determine if the duplicates existed there. Because of the number of features in the shapefile, it's efficient to summarize the Reference field, which outputs a table with a count of each unique reference value in the shapefile.

After creating the summary table and sorting the Count_Reference field, you see that many Reference numbers have a duplicate. To find out exactly how many, select the records whose Count_Reference value is equal to 2. It turns out that 493 Reference numbers have a duplicate in the source data you started with. Determining how many of those 493 are in your geodatabase feature class requires more work.

Just like you did for the shapefile, in the Incidents to Analyze layer, summarize the Reference field to output a table with a count of reference values, then select the records whose Count_Reference value is equal to 2. It turns out that 170 records have a duplicate. Is there a way to easily remove 170 duplicate records? Yes, fairly easily. Here's how:
  1. Join the summary table of reference values to the Incidents to Analyze table.
  2. Select records whose Count_Reference value is equal to 2—as expected, there are 340 selected records.
  3. Start an edit session, then use the Delete Identical tool to delete the selected records that have the same Reference value as another record (note that before running the Delete Identical tool, you must remove the table join—the selected records remain selected after the join is removed).

Final result: 469 incidents ready to be used as input in step 3 of the GIS analysis process.

All this data exploring and preparing really only required a couple of hours of focused work. Of course, the incident victim values likely need cleanup as well—but we'll end the example here as the boundary line between steps 2 and 3 of the GIS analysis process is not a solid fence.Takeaway: When preparing data for analysis, you have to make decisions and live with some uncertainty. Your specific analysis criteria and how well you know the topic determine how far you go to clean up the data. To avoid creating or propagating errors, carefully explore the data and document any data preparation you do and why you chose to do it. Remember that a model is a valuable tool to document your workflow and help others better understand your analysis results.

The data you use for a GIS analysis project may not be perfect and it may not fit your needs exactly. But planning and preparation go a long way toward making sure the data generates reliable results you can confidently share with others.

Want to learn more best practices for GIS analysis? Here are courses that can help. -->
0
0
4323
IDEA
-->
0
0
1385
BLOG
0
0
1177
IDEA
-->
0
0
877
IDEA
-->
0
0
1564
IDEA
-->
0
0
1564
Activity Feed