topic Re: Problem running Region Group tool in ArcGIS Spatial Analyst Questions

Problem running Region Group tool

ClintCabanero — Fri, 12 Nov 2010 23:13:32 GMT

Hello all,

I'm simply trying to run the RegionGroup Spatial Analyst extension tool in ArcToolbox. For some reason, when running this against a large raster (northern California), the tools runs 'successfully' - meaning it completes without error - except that the output has a data anomaly - whereby there are striped bands across the output. The cells in these striped bands also have large integer values (e.g. 1991 while all other values are 1, 2, 3, 4, etc. for each unique group). This tool problem has been repeated by different people on my team.

I'm seeing this same RegionGroup tool problem in ArcGIS Desktop 10 AND ArcGIS Desktop 10 Service Pack 1.

Can anyone confirm this same problem - or recommend a work around?

Thank you

Re: Problem running Region Group tool

DanPatterson_Retired — Sat, 13 Nov 2010 09:41:00 GMT

do you have a screen grab to illustrate this?
did you try filtering the grid a bit before processing using one of the options in the help files? "Simplify your data by filtering it. This can be done with tools such as Majority Filter, Aggregate, Shrink, and Focal Statistics". The choice would depend upon the nature of your input data.

Re: Problem running Region Group tool

FelixStracke — Thu, 18 Nov 2010 12:02:45 GMT

Hi,

I am running into the same problem. I run the region group toll on whole North African Countrys and it runs without failure. But if the country is big (e.g. Algeria, Egypt) it creates the stipes without data. On smaller countrys it runs quite well (e.g. Morocco, Tunisia). Has anybody found a solution - I am getting crazy ...

Best regards

Felix

Re: Problem running Region Group tool

AntonioNatale — Tue, 22 Feb 2011 08:46:01 GMT

I have the same problem but with data not so big (a second level of italian's administrative unit). One horizontal strip is created in the output raster. I confirm this problem and I'm not able yet to find some solution on the web. I tryed also to repeat the Region Group tool changing some parameters and environment of processing (ArcMap, ArcCatalog e Arcpy) but without positive effects.
I attach one image of the result!
I think the best way to create the region raster is to create it out from a geodatabase. If you create a simple GRID you will not have any kind of mistake.
IT'S A BUG

Re: Problem running Region Group tool

FelixStracke — Tue, 22 Feb 2011 10:24:05 GMT

Hi,

thanks for your reply. I figured out that it is seems to be a size problem. The bug only appears when the dataset reaches specific size, e.g. Tunisia works quite well, Algeria was a mess.... Thus, my solution was to write a script that divides big image into smaller parts and than mosaic these parts together after region group.

Best regards

Felix

Re: Problem running Region Group tool

AmyKeister — Thu, 14 Apr 2011 18:31:20 GMT

I have the same problem. I am re-running a model that I built in 9.3 and the region group results with the same bands of errors that are described above. It ran fine on the same raster in 9.3 (the only difference is that I'm doing 8 neighbor instead of 4 neighbor this time). Dividing the big image into smaller pieces won�??t work for me because I know I have at least one very large clump that would be split in the process. Has ERSI created a fix for this problem yet?

Re: Problem running Region Group tool

AnaGonzalez — Wed, 08 Jun 2011 07:18:25 GMT

Hi,

I am doing research on Tree Canopy Cover and I am using the Supervised Classification option. I have been trying to run the Region Group tool but I always get a raster with a horizontal line on the center of the image. Is there someone who knows how to create the Region Group raster without the line?

Re: Problem running Region Group tool

MichaelSutherland — Thu, 30 Jun 2011 14:38:06 GMT

I have run into the same problem while trying to use the tool. It appears as if the data of the original raster is maintained but simply shifted horizontally.

I have tried running from within a file gdb, per team04's post, to no avail. Striping persists.

Re: Problem running Region Group tool

ChrisSnyder — Thu, 30 Jun 2011 19:15:20 GMT

If you create a simple GRID you will not have any kind of mistake.

I think team04 meant that by using good ole' grid format as an input/output format (not a FGDB raster), the problem was avoided.

Just an FYI too, but generally, reading writting grid format in SA tools is way faster than FGDB raster format. If you use FGDB format stuff, behind the scenes it is formats to grid, runs the SA tool algorithm, and then, writes the grid output back to FGDB raster fromat. There is a lot of time that is potentially lost in all that reformating...

Re: Problem running Region Group tool

PavanYadav — Fri, 01 Jul 2011 18:05:52 GMT

Hi All,

We are currently investigating this potential bug. Once, our testing is done and we have a bug ID, I will contact you back. This issue might be related to two aspects:
1. Large input raster in terms of number of rows and columns; and/or
2. Unique value range and complexity of unique pixel values distribution.

Comments on this forum indicate that if the spatial extents (coverage) of the input data is fairly large, the output will have horizontal lines. So maybe the second condition is not related to this issue.

I will greatly appreciate if you could contact Esri Support and share your test data with us. Testing with different datasets will help us narrow down the problem.

Re: Problem running Region Group tool

curtvprice — Fri, 01 Jul 2011 21:48:08 GMT

generally, reading writing grid format in SA tools is way faster than FGDB raster format. If you use FGDB format stuff, behind the scenes it is formats to grid, runs the SA tool algorithm, and then, writes the grid output back to FGDB raster fromat. There is a lot of time that is potentially lost in all that reformating...

This is true at 9.3 but not at 10.0. Most raster formats are now supported natively by the tools. ESRI Team Raster people have told me that they have committed themselves GRID and FGDB will probably be the fastest (and probably will have less bit-depth weirdness problems, IMHO). I have heard good things from users, some of which who have moved to FGDB for their raster processing at 10.

What's" rel="nofollow" target="_blank">http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/What_s_new_in_ArcGIS_Spatial_Analyst/00qp0000001n000000/]What's new in ArcGIS Spatial Analyst 10
- look for the heading: Native Read/Write

Re: Problem running Region Group tool

PavanYadav — Tue, 19 Jul 2011 17:55:04 GMT

Greetings,
This issue has been logged as a bug in ArcGIS 10. We will do our best to get a resolution to this issue in an upcoming service pack. Please feel free to let us know about any additional information you feel is relevant to this bug.

Here is the bug ID and synopsis:

NIM070461: The Region Group tool may produce unexpected output - the data may have an anomaly whereby there are striped bands across the output. This happens when the input is fairly large.

Thanks
Pavan

Re: Problem running Region Group tool

ChrisSnyder — Tue, 19 Jul 2011 18:40:42 GMT

Off topic, but...

This is true at 9.3 but not at 10.0

ESRI never said all their SA tools support native read/wriite, and of course never indicated specifically which tools still rely on native grid. In my experience, many of the v10 hydrologic tools (fill, flowdir, etc.) are still much faster in GRID format. Unless someday ESRI says - "now in ArcGIS v23.1 all the SA tools are native FGDB read/write and are now all faster when used in FGDB format than GRID format" I will keep using good ole' GRID format as my default raster format of choice.

Re: Problem running Region Group tool

EricRice — Wed, 20 Jul 2011 15:45:30 GMT

Hi Chris,

All Spatial Analyst tools support native I/O. We enhanced the underlying Spatial Analyst Engine to support native read/write, which every SA tool relies on.

Our development teams did substantial work to make FGDB comparable and sometimes even faster. Our efforts continue into 10.1 and beyond. The real benefit is for people who wanted to use FGDB or other formats all along. i.e. Using FGDB in 10.0 is way faster than using FGDB in 9.3.1.

Performance is gained simply by us not creating scratch GRID's behind the scenes. Also, performance is not the only benefit to native I/O. See below.

Any field name and path length limitations imposed by the ESRI GRID format are overcome. (Unless you still choose to use GRID.)

The 2.1 GB shapefile size limit is avoided by writing outputs to file GDB or SDE.

If supported by the specified input and output formats, time values in the date fields are preserved, and nulls are treated as such and no longer being converted to zeros.

Our intention with the native I/O project was not to drive people away from using GRID. If the field and file name limits never bothered you or hindered you in accomplishing something, please continue to use the GRID format. It is still definitely one of the top two fastest formats to work with.

Re: Problem running Region Group tool

ChrisSnyder — Wed, 20 Jul 2011 22:16:19 GMT

I will plead ignorance here, but why is it that for me a standard 90m USGS DEM for WA State consistently takes 30 seconds to run a Fill command in GRID format while the same raster in FGDB format takes 90 seconds? This users conclusion: FGDB is still, for whatever reason, sometimes significantly slower than GRID. I am left to theorize that for some tools there is still some extra processing overhead happening when the input/output is in a FGDB (or other non-GRID) format.

However, granted, noted, and appreciated that most of the SA tools perform at par or sometimes faster when used with FGDB format (when compared with GRID format). Just curious here, but why aren't all SA the tools faster with FGDB format? Is the algorithm code somehow tailored to specific data formats?

Re: Problem running Region Group tool

SamanthaSifleet1 — Fri, 22 Jul 2011 18:32:17 GMT

Greetings,
This issue has been logged as a bug in ArcGIS 10. We will do our best to get a resolution to this issue in an upcoming service pack. Please feel free to let us know about any additional information you feel is relevant to this bug.

Here is the bug ID and synopsis:

NIM070461: The Region Group tool may produce unexpected output - the data may have an anomaly whereby there are striped bands across the output. This happens when the input is fairly large.

I have spent the day dealing with this Bug NIM070461. I am working on grids of the MW US 30mx30m, one state at a time. Today I play with Minnesota. The tool ran fine this morning out of FGDB (plain 'ole grid). I had no success running the tool from/to FGDB. I got the stripes. Now I am running the tool again outside of the FGDB. Same thing but different. The tool completes to 100% but never finishes. In the results window I see an empty output raster. I have opened that raster and seen the regions as complete, however the attribute table won't display. So no stripes but no attribute table either. I am on my second replication of this, tool started running at 15:12:00 said it completed at 15:14:00 it is now 15:28:00 and I see no signs of completion. Just killed it same thing...no stripes but no attribute table either. - Maybe this will help in fixing the bug.

Side note: Anyone able to get Patch Grids from Patch Analyst 4 up and running in 10?

Re: Problem running Region Group tool

EricRice — Mon, 25 Jul 2011 15:47:36 GMT

Hi Samantha,

We don't always automatically build raster attribute tables. Try running Build Raster Attribute Table on the result to get the table.

By default, the size of a raster attribute table is limited to 65,535 unique values. You can increase this number on the Options dialog box by clicking the Raster Attribute Table tab on the Raster tab.

Regards,
Eric

Re: Problem running Region Group tool

FelixStracke — Wed, 28 Sep 2011 12:45:27 GMT

After trying to fix this region group issue with workarounds using scripting blablabla - my suggestion is straight from my heart: Use the good old ESRI GRID format. It is indeed a lot of faster (e.g. 1 minute using ESRI Grid as compared to ~ 15min using tif or FGDB raster) and produces a correct output!!!!!

Regards

Felix

Re: Problem running Region Group tool

ChrisSnyder — Sun, 12 Dec 2021 04:26:08 GMT

To add to this issue, in v10.0 SP3, I found that Eulcidean Allocation also suffers from the same sort of buggy behavior as RegionGroup (weird linear artifacts) when the euclidean allocation output grid is placed in FGDB format. When run entirely using grid format, the output is correct.

In my Python code I was able to rectify the problem very simply: By changing the arcpy.env.Workspace variable to a folder (thus forcing the Grid format) instead of using my FGDB as the workspace.

Somewhat related, I find it a bit frustrating in the new v10.0 map algebra that you cannot seem to force the "temp" output rasters to be a particular name, and have to rely on the .save method to, in effect, rename it.

I wish you could do this:

r"C:\Temp\euc_aloc" = arcpy.sa.EucAllocation(roadGrd, "", "", "", "VALUE", "", "")

But it seems you have to do this:

arcpy.env.workspace = r"C:\Temp"
junk = arcpy.sa.EucAllocation(roadGrd, "", "", "", "VALUE", "", "")
eucAlocGrd = r"C:\Temp\euc_aloc"
junk.save(eucAlocGrd)

If I do this, the euclidean allocation output (in FGDB format) gets wonkey:

arcpy.env.workspace = r"C:\Temp\test.gdb"
junk = arcpy.sa.EucAllocation(roadGrd, "", "", "", "VALUE", "", "")
eucAlocGrd = r"C:\Temp\test.gdb\euc_aloc"
junk.save(eucAlocGrd)

#roadGrd is in Grid format BTW...

Re: Problem running Region Group tool

curtvprice — Sun, 12 Dec 2021 04:26:11 GMT

Chris, did you try:

arcpy.env.scratchWorkspace = r"C:\Temp"
junk = arcpy.sa.EucAllocation(roadGrd, "", "", "", "VALUE", "", "")
eucAlocGrd = r"C:\Temp\test.gdb\euc_aloc"
junk.save(eucAlocGrd) # copies "C:\Temp\raster3" to "C:\Temp\test.gdb\euc_aloc"

This would (in theory) do the processing using grids and then copy the output over to file gdb format (that is, how it works in 9.3 when you specify fgdb output) .

I have been told that (in 9.3, probably same as 10.0) if you set the scratch and current workspace to the same location, the temp grid (say, "raster3") will be created and then renamed (rather than copied) when the tool completes. Not quite what you were looking for, but just as fast:

arcpy.env.scratchWorkspace = r"C:\Temp"
arcpy.env.workspace = r"C:\Temp"
junk = arcpy.sa.EucAllocation(roadGrd, "", "", "", "VALUE", "", "")
eucAlocGrd = r"outgrid"
junk.save(eucAlocGrd) # renames "raster3" to "outgrid"

If you can, *please* submit test data that reproduces these problems with file gdb rasters to ESRI support! Sometimes it is very difficult for them to fix things unless they have a dataset that reproduces the issue.