topic Re: Misaligned Land Parcel Update in ArcGIS Enterprise Portal Questions

Misaligned Land Parcel Update

EricRoot — Fri, 01 Jun 2012 11:22:13 GMT

I have been given an updated version of the land parcels layer for a geodatabase that covers the company's service area (From Hunter GIS). This update is in fact a new shape file that I am to replace the old shapefile with. This would not usually cause any problems but when I uploaded the new shapefile and placed it over the existing one, the road edges and property lines did not line up. Some of the parcel's edges were misaligned by up to 20 metres in some places.

Biggest issue - not a uniform misalignment. Some places line up well but others are skewed, either to the north east, south east, you name it.

Both shapefiles are projected on NAD 1983 in zone 18N, and when comparing some coordinates on the original map to coordinates on google earth (plotted on WGS 84)they were a close match - 0.5m difference. When comparing the same google earth coordinates to the same point on the new land parcels layer they differed by 3m and 8m for Eastings and Northings respectively.

If I have left any information out that might help solve the problem please let me know and I will do my best to answer.

Question: how do I re-align these so they match?
Can anyone explain why the 'correct version''s coordinates do not match those of Google Earth?

Any help is appreciated!

Re: Misaligned Land Parcel Update

HardolphWasteneys — Mon, 04 Jun 2012 06:21:40 GMT

Eric,

The magnitude of the displacement is typical of an incorrect GCS transformation, but to confuse that you mention that the displacement is variable.
Also, I'm not sure what you mean by "Both shapefiles are projected on NAD 1983 in zone 18N": do you refer to the Data Frame projection or the CRS of the files as shown in layers Properties > Source or in ArcCatalog under the shapefile Properties XY Coordinate system?

If they are the same CRS then the GIS source has made some error in the new file or corrected some error in the old.
If they have a different Geographic Coordinate System then look at Transformations in the Data Frame Properties and make sure the correct one is set for transforming the GCS of the new layer to that of the Data Frame and presumably of the original layer.

The other question is: how do the shapefile line up against some reference like one of the "Add Data from Resource Center" maps in the main ArcMap menu under "File"? This is usually a good test especially in high precision areas of the eastern seabord and southe Canada and more convenient than importing the shapefiles into Google Earth or indirectly comparing a few spot coordinates.

If you can post the CRSs of the two files and a screenshot of their alignment relative to the Resource Center maps that would help clarify the problem.

Hardolph

Re: Misaligned Land Parcel Update

EricRoot — Mon, 04 Jun 2012 11:37:00 GMT

If they are the same CRS then the GIS source has made some error in the new file or corrected some error in the old.
If they have a different Geographic Coordinate System then look at Transformations in the Data Frame Properties and make sure the correct one is set for transforming the GCS of the new layer to that of the Data Frame and presumably of the original layer.

The other question is: how do the shapefile line up against some reference like one of the "Add Data from Resource Center" maps in the main ArcMap menu under "File"? This is usually a good test especially in high precision areas of the eastern seabord and southe Canada and more convenient than importing the shapefiles into Google Earth or indirectly comparing a few spot coordinates.

If you can post the CRSs of the two files and a screenshot of their alignment relative to the Resource Center maps that would help clarify the problem.

Hardolph

What I mean by "Both shapefiles are projected on NAD 1983 in zone 18N" is that the Coordinate System tab in The data frame properties is set to "NAD_1983_UTM_Zone_18N" (see screenshots)

When I check the CRS os the files in layers properties > Source the projection is stated as the same for all layers (see screenshots), with the exception of the orthophotos that make up the background.

Unfortunately, the area I am looking at is Pembroke, ON and the ArcGIS online only seems to provide US maps. The orthophotos were submitted with the 'new' parcels layer and so the new parcels line up quite well.
**The yellow parcels are the original ones and the pink parcels are the new and 'improved' bunch from the supplier.

I will attach a few pictures of each step. One more detail - the original parcels and new parcels are not in the same data base, but their stats seem quite similar.

Let me know if I did not answer all of your questions. I believe that it would also be possible to upload the shapefiles. However, I am interested in learning to solve the problem with assistance, as opposed to handing it off.

Thanks,

ORPC

Re: Misaligned Land Parcel Update

HardolphWasteneys — Mon, 04 Jun 2012 13:13:40 GMT

Eric,

there are quite a few choices in the Resource Center for different types of imagery, topo etc that provide good coverage outside the US. I'd start there again and see if you can come to some resolution about the 2 files.

Hardolph

Re: Misaligned Land Parcel Update

HardolphWasteneys — Mon, 04 Jun 2012 14:48:49 GMT

Hi Eric,

had a closer look at the clips of the shapefile alignment on the orthophotos. Looks to me like there is no problem with the coordinate systems, but more of disagreement between the old digitizing and the new orthophotos and shapefile. The new shapefile appears to be redrawn completely from the orthos (I assume that is what is displayed below)so various small shifts may have something to do with the mosaicing of the photos relative to the old ones. May be a case where the old shapefile was drawn from less accurately aligned photos.

Do you have the old photos to compare with the new? How about the alignment with Resource Center imagery?

Hardolph

Re: Misaligned Land Parcel Update

EricRoot — Mon, 04 Jun 2012 16:36:05 GMT

Hardolph,

The images that I was able to find are much larger than the area in which I am working, and as a result drawing time is greatly increased: I will try to look at a street corner after posting here. However, the resolution is significantly lower than the orthophotos I am currently using and accuracy may be an issue. I searched through the online database and was only able to find a map of Ontario roads.

Unfortunately, I do not have the imagery used to create the original land parcels layer.

I was able to measure the distance between land parcels, and in some places it is upwards of 20 m. Most places are around 3m difference but I did not think this would be expected. The original map is from around the year 2000 and the new parcels are from 2012. I have attached the shapefiles in a ZIP folder, but one is added to my map as a .CPG file. Please let me know if they work. I have attached them incase you would like to take a look.

Side note: I have encountered another issue: when plotting street poles around the city ArcMap closed unexpectedly and I can no longer access the information on my poles attribute table. The number of entities is still there and I can select poles by clicking on a row in the table, but all cells are blank. I receive an error message each time I try to open it, even after refreshing. I will attach a screenshot as well.

Re: Misaligned Land Parcel Update

HardolphWasteneys — Mon, 04 Jun 2012 18:42:12 GMT

Eric,

I'll have a look at the data later today when I get back to my GIS machine.

If the Resource Center imagery is clear enough, make the parcels show just outlines and you might be able to get a better feel for which shapefile is more accurate. One presumes that more recent photo coverage and mosaicing should be more accurate than older stuff that might not have been set up for the world of GIS parcel fabrics.

Given the precision you need for this, and the available resolution on the Resource Center you could go back to overlaying both shapefiles in Google Earth, but make the parcel polygons clear so you can see alignment better. The May 2010 GE image looks high enough resolution to draw from itself as you can see sidewalks and fence lines.
Another essential place to check would be with Pembroke city engineering for CAD drawings based on surveys. You'd have to georeference these dxfs in ArcMAP by finding two unambiguous points for which you have independent coordinates.

The photomosaic/orthophotos will have ground control points with precise coordinates and you could also cross check those both against the display in ArcMap and in Google Earth to see if there is any glaring discrepancy assuming the GE imagery is correctly georeferenced. Overall you are trying to check the veracity of the new orthophoto mosaics; from what I can see the new digitizing is good relative to them. So the 3 to 5 m shift you measured relative to GE spot locations in the new shapefile (and presumably imagery) is cause for concern. GE imagery might be off by that much, but the city engineering drawings will not. Another independent check would be some GPS coordinates of distinct features on the new orthophotos, but those would have to be taken with a sub meter unit (e.g. Trimble GeoXT) not just a recreational one (e.g. Garmin Map60CSx). If it came to that you could also have the control points checked, but I would not hold much hope that they were inaccurately measured. Maybe a wrong transformation from WGS84 to NAD83, for the control point file, but that would not account for much.

Meanwhile one thing I noticed that might help with the connection problem is that you seem to have a space in the name of the personal geodatabase in C:\Drawings\Pembroke2\Pembroke DB.mdb, which is not advisable. If so, good idea to either rename or copy contents to a new one. Also, I generally no longer use .mdbs in favour of file geodatabases after finding problems with them moving from an XP platform to Win7, but I don't think that is a cause of the connection problem.

Hardolph

Re: Misaligned Land Parcel Update

EricRoot — Tue, 05 Jun 2012 10:48:10 GMT

Hardolph,

I spoke with my employer and it turns out that the original parcel fabric was based off of the original lotting plans for the city. I was thinking that issues may have arose when converting the paper documents to a digital format but I would not expect such a difference between the two layers.

The main reason why I am concerned about the alignment of the layers is because all of the hydro information (poles, lines, buildings etc.) is locked into other layers that fit onto the original land parcels layer. Due to construction in the past few years, more lots have been placed in the city that need to be updated and drawn in the layer (see southeast corner of layers), as powerlines have been installed in those subdivisions. When plotting the original information onto the new land parcels, most poles are now located in the middle of people's front lawns or backyards, as opposed to being on the property line/easement.

I will take a look at the unfilled parcels but as it stands on the basemap provided by ESRI, it looks as it the original layer fits the shoreline more accurately. The image is too low resolution to check roadways.

ORPC

Re: Misaligned Land Parcel Update

EricRoot — Tue, 05 Jun 2012 14:21:40 GMT

I was hoping that by figuring out the problem I would be able to move/alter all of the other information to line up with the new parcels, that way the parcel information would be as accurate as possible. Another option might be to draw the parcels onto the old layer by hand but seeing as the orthophotos do not line up this may cause some issues..

Re: Misaligned Land Parcel Update

HardolphWasteneys — Tue, 05 Jun 2012 21:18:16 GMT

Eric,

If you have confirmed that the old parcel shapefile is accurate then you should stay with it, period. AND to boot, it is also important since you've noted that all sorts of utilities layers are based on the same datum. I wasn't able to do anything with the xml file you posted, but it sounds like you've resolved things anyway with the news that the original was drawn from survey plans.

If there is new information in the New parcel shapefile pertaining to existing parcels then you can use the attribute transfer tool http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=Transferring_attributes_between_features to copy it over one parcel at a time or use a spatial join and add attribute fields to your existing shapefile. Another alternative would be to use the spatial adjustment tool http://webhelp.esri.com/arcgisdesktop/9.3/toc.cfm?Action=1&LID=702&rand=865#702 and shift the whole thing to match your old shapefile, but this would apparently involve some rubbersheeting so I'd be hesitant to waste my time fixing a contractors error. You might also consider regeoreferencing the new photomosaic and then spatially adjusting the new shapefile to match, but again same deal ... why fix the contractors mistake.
Parcel fabrics may not be legal lot surveys, but they should be as accurate as possible.

However, it seems just a bit disturbing that the new orthophotos and the shapefile drawn from them ARE inaccurately georeferenced. This would be a case for going back to the contractor and getting them to fix it. It seems especially glaring in view of the obvious variable displacement from Google Earth imagery which shows lot lines (sidewalks, fences etc.) in Pembroke so clearly I could see drawing them straight from that.

Anyway, it would be educational if you have the time to go through the steps of checking the ground point surveys in the new photomosaic, look at how the photos were orthorectified and mosaiced and see if you can spot the error. Once you contact the contractor you could also go through the steps they took to see if someone did a transformation from WGS84 to NAD 1983 and used a wierd algorithm. Maybe they just mixed up data from a recreational GPS rather than an RTK or survey GPS. Clearly, they did not do any ground truthing.

good luck, I'll look forward to seeing how this mystery turns out.

Hardolph

Re: Misaligned Land Parcel Update

EricRoot — Thu, 07 Jun 2012 13:25:32 GMT

The contractor has simply stated that the original shapefile is incorrectly laid out, they did not offer any type of solution so I thank you for all of your help.

I have decided to use the Spatial Adjustment tool, using the rubbersheet option. I spent the last day working on moving each corner of the land parcels and after adjusting there are still some errors. I noticed that if I move one land parcel the parcel across the street may be affected. What I am doing now is aligning each corner of each land parcel - a very tedious process.

Would you have any tips or suggestions on how to decrease my work time? Maybe even some sort of keyboard shortcuts?

Re: Misaligned Land Parcel Update

HardolphWasteneys — Thu, 07 Jun 2012 17:33:47 GMT

Eric,
frustrating business ...

offhand suggestions regarding a wholesale spatial adjustment would be:
1. work from copies of the original
2. review the help sections on spatial adjustment thoroughly
3. review the use of snapping

As you observed, rubbersheeting the whole thing can lead to frustrating iterative effects and you may be better off shifting sections of parcels and snapping them to your old ones.

All that being technically feasible, the contractor's statement that the original shapefile was not correctly laid out seems ambiguous in light of the incorrect georeferencing of the photomosaic they used, unless they were given the photomosaic as is.   I'd get clarification of that from them.   I don't know how much is at stake in the new shapefile or what the intent was, so it is hard to recommend a course of action, but I'd resolve the georeferencing of the photomosaic first. If the new shapefile covers the same parcels as the old one only and with the same attributes, I'd be tempted to abandon it, but I have not seen either except in glimpses from the screen shots.   If you adjust the new file the photomosaic will no longer be useful as an underlay for new development areas. At least it is just Pembroke and not all of Ottawa or Toronto.

So to reiterate: I'd not recommend spatially adjusting the new shapefile until you have resolved the georeferencing of the photomosaic. Get the ground control points and double check their coordinates and that they were used correctly to georeference the photos. If there was an error in that find out who was responsible. If the GIS contractor was only responsible for drawing the shapefile they might be off the hook, but it seems byzantine to me.

If you want to upload the two shapefiles, copy them into a new folder in ArcCatalog and see if you can upload that here as a unit or send them to me directly at my e-mail and I'll have a look. As I said I could not do anything with the xml file alone.   The photo raster is probably too large.

looks like you've done a thorough job so far.

Hardolph

PS. as an afterthought on spatial adjustment in case you really get stuck into it: avoid rubbersheeting as much as possible and minimize the offset you use if you do by first finding an absolute offset that comes close as possible by using the editor MOVE function to shift all the polygons by some measured and recorded amount (e.g. 3m west and 5m south), or attempt to use a Projective transformation. Again work from copies and if it screws up ditch it and start again. Rubbersheeting should be your last resort.
H

Re: Misaligned Land Parcel Update

EricRoot — Thu, 14 Jun 2012 11:25:27 GMT

I have successfully oriented all parcels to be aligned with the previous (older) layer, using the rubbersheet method. I ensured that the other layers were being transformed as well.

The limited adjustment tool allows the user to draw a polygon around a specific area to set boundaries for rubbersheeting. Anything outside of the polygon will not be affected. This is how I was able to line up the layers precisely.

Now I have to transfer the attributes and set the symbology in my new layers to match the utilities symbology.

I am using ArcMap10.

Re: Misaligned Land Parcel Update

EricRoot — Thu, 14 Jun 2012 16:08:31 GMT

Hey,

I was able to get through the entire city, any new parcels or shapes I did not join to but would use the edge snapping tool to get close. If there was a curve vs a point I would rubbersheet the edges until the beginning of the shape so that it would best fit. There area few small areas of interest, such as hydro lines getting a little close together, but as I use the map I will work out the kinks. I will be using the shapefile you have created in place of the PembrokeParcels_ORC to maintain data, and I thank you for providing me with this.

If I simply used the rubbersheet on the corners a complex area the polygon gets deformed. In the example I attached (as I did for the rest of the city) I placed displacement links on all corners of all polygons if they had a corresponding parcel on the other layer.

I have showed my employer that some buildings are somewhat deformed - skewed building edges on some - and they are happy with the results. For the most part the building polygons maintained their shape, reason being: I placed links on all corners and so the shape of the polygons did not change, just the orientation.

Thanks again for all of your help.

Eric