When I generate a raster with Point To Raster, I am seeing odd shifts in the raster cell locations. I overlayed the original points with the raster. Across the northern and southern regions of the extent, the original points are centered north/south in the raster cells. They become less and less centered as one moves toward the middle of the extent relative to north/south. East/west there is no variation across the extent. They are centered east/west across the extent. There is a one cell wide band across the middle of the raster that has no cells with NoData values. It looks like the cells "drift" out of alignment as you approach the middle band of the extent.
What is also odd that I get the behavior when I start and end my processing (steps below) with a 1km resolution DEM but do not get the behavior with a 30m DEM. Images in the attached PDF for reference. Am I doing something off here? Any idea what is off in my data or work flow?
I have looked at the thread here: point to raster point is not in the center of the created cell
Not sure how to apply the other advice given there as I am seeing a different shift across the extent of this problem raster. This is a regular grid of points, so it seems confusing that the raster cells shift.
1) Clip DEMs to study area. Roughly 100km east/west by 200km north/south.
2) Use Raster To Point to create a regular grid of points from the raster. Examining these points show them centered on the cells.
3) Use Add XY Coordinates on point feature. (Need lat/long for climate model processing.)
4) Use Table To Table to generate a CSV file.
5) CSV file is used as input into the ClimateWNA program.
6) ClimateWNA outputs a CSV file with ~23 climate attributes for each point supplied. Output includes the original lat/long/elev. The program does not change these three values.
7) Use XY Table To Point to create a feature layer. Examining these points, they are centered on the cells in the original clipped DEM.
8) Use Point To Raster to create a raster for each of the needed climate attributes. One raster per attribute. The environment parameters for GCS, snap raster, cell size, and extent were all set to match the original DEM in step 1.
Step 8 is where the odd behavior described shows up. The work sequence using the 30m DEM has no issues with the offset. I have scanned across the same regions in the extent. Everything aligns: extent, cell size, points centered in cell. Even all the features and rasters with the 1km DEM align until the P2R step.
All GIS processing is with ArcGIS Pro 2.2.4 on a HP Windows 10 Pro desktop. ClimateWNA is a desktop program that is the source for the climate data attributes.
All features and rasters verified as being NAD83 GCS. ClimateWNA outputs in WGS84, but points projected to NAD83 show no delta in location. Raster issues appears whether the points are NAD83 or WGS84.
The north/south dynamics make me suspect some kind of projection issue. I tried combinations leaving various environment settings at default (other than cell size) but get the same odd behavior. ClimateWNA output was also suspect, but the points generated from that data align fine with the original DEM.
I am massaging data for eventual export into a MaxEnt environmental niche model. The two DEMs are a USGS 3DEP 1arc-sec (~30m) acquired from the USGS site and PRISM's 30arc-sec (aka 800m aka 1km) DEM downloaded from their site. The PRISM data source is a USGS NED 3arc-sec DEM interpolated to resolution per their literature.
I need all the rasters generated at each resolution to align exactly in extent and cell size for the MaxEnt processing. They eventually get output as ASCII files and moved into an R workflow. I have done this workflow before with other USGS DEM datasets and had no issues. First time using the PRISM DEM however.
Solved! Go to Solution.
Got some great guidance and advice from ESRI via email. Updated work flow below avoids the issues with the banding and cell offsets. Changes from original work flow noted in brackets.
1) Clipped DEM to study area.
2) Raster To Multipoint to generate point feature class of clipped DEM [Instead of the Raster To Point tool]
3) Multipart To Singlepart on point feature class [new step]
4) Add Geometry tool [instead of Add XY Coordinates] to add longitude and latitude attributes to the point features
5) Table To Table to create CSV file
6) Import CSV file into ClimateWNA
7) cWNA generates an updated CSV file with lat, long, elev from original file associated with climate attributes for every point
8) XY Table To Points to convert CSV output file to a feature class.
9) Point To Raster to create a raster for each needed climate variable. Used the clipped DEM as set point for extent, snap raster, and cell size in the Environment parameters for the tool.
10) Success! Points overlayed onto new climate raster align with the cell centers. Raster contains climate attribute.
Possible points of failure in the original work flow:
* The original polygon used to clip the DEM was found to be projected to Albers PCS. Should not have caused the problems seen subsequently as the clipped DEM, not the polygon, was used for all subsequent processing. Still, it could have introduced something. However, it seems very strange that if it did, why did it not manifest in one of the early steps in the processing instead of the last step? The early points still aligned with the clipped raster as did the latter point features.
* Raster To Point. Used the two tools listed in 2 & 3 to bypass this tool.
* Add XY Coordinates. Used Add Geometry instead.
* When I ran the XY Table to Points (step 8) I got a warning message about there being no vertical coordinate system for the Z value. Do not recall seeing this in the prior work flow. Either I missed seeing the warning in the prior runs or a default was applied that I did not catch, resulting in no error message. I suspect the latter.
I would have to do trail-and-error on each of these to narrow it down to the specific cause. Will have to wait until after the holiday at least. Time to kick off the model run then off to bed.
Thank you all for the help and insights. Took a group effort to sort out a solution.
on a lark, could you try this. You have covered all the usual bases (extent, snap raster, cell size, projection issues)
I have a vague recollection of a similar issue not covered by the aforementioned and feature to raster popped into mind (even though they are points, I know)
Thanks Dan for the quick response. Tried the Feature To Raster. Same shift in the cell centers and same banding across the middle.
What is really odd about this is the divergence from the center of the raster out, both north and south. I would have expected some east-west shift in cell size going north-south given the GCS projection, but a north-south shift in centers is really strange. Especially when it does not show up on the 30m data.
I vaguely remember something about usgs dem issues for a particular area in the US. I can't remember where nor can I put my hand on the thread. Perhaps if you search for links on geonet for which contain 'dem' and your area (state etc) it may pop up. banding or bands was mentioned. it is rest for a few hours for me
one cell gap, striping, banding...
hopefully it is one of these
Thank you. That looks like a good candidate for the issue. It is a level 1 DEM as described in the USGS article. The banding and shift show up after processing.
I will have to rethink my approach to creating the climate rasters. Resampling is not a good option in this case. The climate data is interpolated output as it is. Further processing would distort the numbers going into the model.
Options I will explore:
* Skip the development of the climate rasters in ArcGIS Pro. Use R and the CSV data and the ASCII files and matrix needed for the modeling. The rasters in Pro were an intermediate step to get to ASCII outputs and for visualization of the data. It is more work to do this in R and I loose some of the visualization benefits in Pro but it would get me the input data for the model.
* Suggestion from my thesis adviser: Create a null grid copy from the original DEM. Assign the climate values to the resulting raster.
Thank you again for the help.
Pro works with numpy without any extra requirements. Perhaps, you can articulate what is needed in R and beyond the visualization portion. I have many blog posts on doing 'stuff' with raster/array/ascii data that may be of use.
Not familiar with numpy. I have been working with r-bridge, arcgisbinding/arc, raster, dismo, and other R packages so more comfortable with that language currently. Some Python exposure but to be honest, my experience has been limited to learning just what I need to, the focus being getting the tasks done for class projects and the recent thesis work. A prior background in IT/tech, with some coding experience in C, C++, java, Pascal, FORTRAN, COBOL/Telon (dating myself here!), etc. helps with the jumps between languages.
I will post the R code I come up with once I sort it out. Should not be too complex as I have the point set and the data values. That code will not provide a solution for the original question about the raster offset but may provide a path for anyone that is trying to do similar data prep. My wife and I head out for the holiday on Wednesday, so it may be early next week before I post back the code.
Attached is a Rnotebook created during a class project earlier this year. The thesis work is more involved, but this gives an idea of what the target is for the raster data. After the workflow I outlined in the thread above, the rasters would get output as ASCII files. The R work picks up from there, taking the ASCII files into raster stacks in R. I found it easier to work in Pro for the DEM manipulations (clip, projections, climate attributes, etc.), map presentations, and creating the ASCII files, but any of the Pro steps can be done in R or Python.
Honestly Cass, I think the issue is with the data you are trying to overlay your point onto and not the workflow you are using.
I would shift the dem by half a cell so the points are within the raster. There was enough doubt that the source data was 'correct', especially when it was fine in parts of the country and not others. But that is an opinion and confirmation would entail dealing with those people directly
The problem with shifting is that the offset is not consistent across the extent. The points and cell centers align on the north edge of the extent and on the south side of the extent. They start gradually shifting alignment as you move toward the center of the extent, eventually ending up at the edge of each cell instead of the center. No east/west shift discernible. That is what really baffles me. If I saw a increasing drift north/south or a consistent shift across the extent I would think I had something related to the GCS.
Some other odd tidbits.
* DEM extent is 210x245
* Point features align with that grid
* Raster generated from those point features is 192x240