topic Re: Count the percentage of positive and Negative pixel within the zone. in Python Questions

Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Thu, 03 Aug 2017 09:56:17 GMT

Hi All,

I have a raster which contains value range from -20 to +20. I am trying to calculate the zone wise percentage of negative and positive pixel count within the zone.

Somewhere I read TABULAR option can work, but it supported only the Vector data and Zonal statistics also can't that.

So how to calculate the percentage of pixel count with in the zone. I have zone's shape file.

Re: Count the percentage of positive and Negative pixel within the zone.

DanPatterson_Retired — Thu, 03 Aug 2017 12:33:22 GMT

You could use a Con statement and subdivide your raster into values < 0 and > 0

like Con('raster' <= 0, 0, 1) which would result in a raster with values <= 0 being classified as 0 and those > 0 being classified as 1. Then you can use the conventional zonal statistics to get the counts for each class within the zone.

The rest is just conversion to percentage.

Re: Count the percentage of positive and Negative pixel within the zone.

JayantaPoddar — Thu, 03 Aug 2017 12:44:47 GMT

Adding to Dan, after executing the Con Tool, if you don't need the final output as a Raster, you could use Zonal Statistics as Table.

Re: Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Sat, 05 Aug 2017 12:02:00 GMT

Hi Dan Patterson‌ , Jayanta Poddar‌, Thanks.

I followed the suggestion given by you and I stuck here, I have attached an image,

I want to count the pixel for Negative and positive classes for each zones. After classification < 0 and >0, I got the 0 and 1 value raster, after that i used Zonal statistics to get the pixel number for each zones. but its shows the total number of pixel in each zones, but I want it to count of the positive and Negative pixel for each zones.

You can see above screenshot, For zone4 shows 571922 numbers of the pixel, but how to get the number of the pixel under to category of positive and negative for each zones.

How to do it .

Thank you

Regards

Shouvik Jha

Re: Count the percentage of positive and Negative pixel within the zone.

JayantaPoddar — Sat, 05 Aug 2017 17:16:57 GMT

Hi Shouvik,

You should have used Con Tool or Set Null (or Raster Calculator) tool.

I will explain the use of Set Null function in Raster Calculator, since Dan has already mentioned the Con tool.

Use the following expressions in Raster Calculator.

1. First, use the following expression to get Raster having all positive or 0 values.

SetNull("Raster1", "Raster1", "VALUE < 0")

2. Then, use the following expression to get Raster having the negative values.

SetNull("Raster1", "Raster1", "VALUE >= 0")

You could use Zonal Statistics (as Table) using the respective Rasters, and append the tables, if desired.

Re: Count the percentage of positive and Negative pixel within the zone.

XanderBakker — Sat, 11 Dec 2021 16:47:53 GMT

Hi shouvik jha ,

The easiest way would be to follow the instruction provided by Dan Patterson and Jayanta Poddar . You will have two zonas statistics tables as result, one for the positive values and one for the negative values both with the respective count you're after.

Since in recent occasions we have done some things with using rasters converted to numpy arrays, below the an example of doing it the "hard way". This will only work if both the zone raster and the value raster are aligned (same extent and cell size and coordinate system).

The value raster with the zone:

The resulting statistics:

The code used:

#-------------------------------------------------------------------------------
# Name:        count_pos_neg_pixels.py
# Purpose:
#
# Author:      Xander
#
# Created:     06-08-2017
#-------------------------------------------------------------------------------

def main():
    print "load modules..."
    import arcpy
    arcpy.env.overwriteOutput = True

    import numpy as np
    zones_ras = r'C:\GeoNet\CountPosNegPixels\data2.gdb\test'
    values_ras = r'C:\GeoNet\CountPosNegPixels\data2.gdb\corr'
    nodata = -3.4028235e+38

    print "create numpy rasters..."
    np_zones = arcpy.RasterToNumPyArray(zones_ras)
    np_values = arcpy.RasterToNumPyArray(values_ras)

    print "read props numpy raster..."
    rows = np_zones.shape[0]
    cols = np_zones.shape[1]
    print " - rows:", rows
    print " - cols:", cols

    print "loop through pixels..."
    pix_cnt = 0
    dct = {}
    for row in range(rows):
        for col in range(cols):
            pix_cnt += 1
            if pix_cnt % 5000 == 0:
                print " - row:", row, "  col:", col, "  pixel:", pix_cnt
            zone = np_zones[row, col]
            value = np_values[row, col]
            if zone < 255:
                if zone in dct:
                    stats = dct[zone]
                    stats = UpdateStats(stats, value, nodata)
                else:
                    stats = CreateStats(value, nodata)
                dct[zone] = stats

    print "\nZone\tsum\tcnt\tmean\tmin\tmax\tcnt_neg\tcnt_pos\tcnt_nodata"
    for zone, stats in sorted(dct.items()):
        print zone, '\t', '\t'.join(str(a).replace('.', ',') for a in stats)


def UpdateStats(stats, value, nodata):
    # sum, cnt, mean, min, max, cnt_neg, cnt_pos, cnt_nodata
    if value > nodata / 2.0:
        stats[0] += value
        stats[1] += 1
        stats[2] = float(stats[0]) / float(stats[1])
        if value < stats[3]:
            stats[3] = value
        if value > stats[4]:
            stats[4] = value
        if value < 0:
            stats[5] += 1
        if value > 0:
            stats[6] += 1
    else:
        stats[7] += 1
    return stats

def CreateStats(value, nodata):
    # sum, cnt, mean, min, max, cnt_neg, cnt_pos, cnt_nodata
    if value > nodata / 2.0:
        return [value, 1, value, value, value, 1 if value < 0 else 0, 1 if value > 0 else 0,  0]
    else:
        return [0, 0, None, 99999999, -9999999, 0, 0, 1]

if __name__ == '__main__':
    main()
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

But once again, it is much easier following the steps as described by Dan and Jayanta.

Re: Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Sat, 11 Dec 2021 16:47:55 GMT

Hi Xander Bakker‌, Thank you very much. Really very interesting script. And yes the step described by Dan Patterson and Jayanta Poddar is very easier to understand.

And This script is one more step ahead rather than producing two different rasters for Negative and positive and then count the pixel numbers by doing Zonal Statistics. So this script may save the time.

Instead of GDB file extension, can it be just file path (e.g r'D:\Raster_Statisctics\Count_Pixel\Zones). And it can be possible to use direct Shapefile instead of raster for extracting the zone information.

I ran the script, but I am getting an error. I think the problem with my Zone's raster. I had a zone vector file (Attached below), which I converted it to a raster format in same spatial resolution as values_ras.

I have attached my file which I am using for counting the negative and positive pixel for each zones. Please have a look.

And the error message is :

Input path:

import numpy as np
    zones_ras = r'D:\Raster_Statistics\Count_Pixel.gdb\Zones'
    values_ras = r'D:\Raster_Statistics\Count_Pixel.gdb\NPP'‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Error message:

load modules...
create numpy rasters...
Traceback (most recent call last):
  File "<module1>", line 70, in <module>
  File "<module1>", line 12, in main
  File "C:\Program Files\ArcGIS\Desktop10.3\ArcPy\arcpy\__init__.py", line 2244, in RasterToNumPyArray
    return _RasterToNumPyArray(*args, **kwargs)
RuntimeError: ERROR 000732: Input Raster: Dataset D:\Raster_Statistics\Count_Pixel.gdb\Zones does not exist or is not supported‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Mon, 07 Aug 2017 07:10:11 GMT

Hi Jayanta Poddar‌, Thank you very much. Your command on the topic is very helpful.

Re: Count the percentage of positive and Negative pixel within the zone.

JayantaPoddar — Mon, 07 Aug 2017 08:55:22 GMT

I am glad it helped.

Re: Count the percentage of positive and Negative pixel within the zone.

XanderBakker — Mon, 07 Aug 2017 19:41:27 GMT

In order to use a zone featureclass the script should convert the shapefile to raster, using the same extent and cell size and coordinate system in order for the numpy arrays to align and to read out the zone and raster value that correspond. Using a TIF raster in stead of a GDB raster might work, but maybe the nodata value is different and the code should be adapted for that.

When you do a Zonal Statistics as Table on the negative and positive values rasters and create all the statistics, you will have many relevant statistics. In my code the mean is calculated on negative and positive values together, when you might be interested in knowing those statistics for only the positive and negative values in a zone.

Re: Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Sat, 11 Dec 2021 16:47:58 GMT

Hi Xander Bakker‌ Thank you.

I ran the code but the same error I am getting which I mentioned above. Please, could you check, where is the problem.

I have made the gdb file using below code,

import os
import sys
from arcpy import env
import arcpy

env.workspace = "D:\Raster_Statistics\Count_Pixel"

# Set local variables
out_folder_path = "D:\Raster_Statistics\Count_Pixel"
out_name = "Count.gdb"

# Execute CreateFileGDB
arcpy.CreateFileGDB_management(out_folder_path, out_name)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

While I using the Count_Pixel.gdb as an input path, that time it's producing the error. I think somewhere name of Zone is not matching.

if you can attach the data (Zone_ras and value_ras) which you used for generating the result, so I can compare and check with my data.

Thank you

Re: Count the percentage of positive and Negative pixel within the zone.

JayantaPoddar — Tue, 08 Aug 2017 05:34:02 GMT

Shouvik,

In Python, the path of a Windows Directory should be mentioned as

r"D:\Raster_Statistics\Count_Pixel"

"D:/Raster_Statistics/Count_Pixel"

Re: Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Tue, 08 Aug 2017 06:30:10 GMT

Hi jayanta.poddar‌, Thanks. I tried the same, but still same error. i think problem with Name of zone field in feature class.

Re: Count the percentage of positive and Negative pixel within the zone.

XanderBakker — Tue, 08 Aug 2017 12:26:45 GMT

Can you indicate what the exact error message is and on which line the error occurs? Apart from the missing 'r' (raw) in the path, I don't see what could be the problem.

Does the path "D:\Raster_Statistics\Count_Pixel" exist and does the script has write access to that location? Does a gdb with the output name already exist at that location?

You are also setting your workspace to the folder and not the gdb. Is there a specific reason for that?

Re: Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Sat, 11 Dec 2021 16:48:00 GMT

Hi Xander Bakker, thanks.

I am mentioning below the input path which I am using and error message also. I tried both with gdb and without gdb in the input path but the error message is same.

However, I have attached my input file both raster and Zone vector file with the previous thread. location Just after the script which you posted.

Input path name:

import numpy as np
    zones_ras = r'D:\Raster_Statistics\Count_Pixel\Zones'
    values_ras = r'D:\Raster_Statistics\Count_Pixel\NPP'
 ‍‍‍‍

Error massage:

load modules...
create numpy rasters...
Traceback (most recent call last):
  File "<module1>", line 70, in <module>
  File "<module1>", line 12, in main
  File "C:\Program Files\ArcGIS\Desktop10.3\ArcPy\arcpy\__init__.py", line 2244, in RasterToNumPyArray
    return _RasterToNumPyArray(*args, **kwargs)
RuntimeError: ERROR 000732: Input Raster: Dataset D:\Raster_Statistics\Count_Pi
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Count the percentage of positive and Negative pixel within the zone.

DanPatterson_Retired — Tue, 08 Aug 2017 18:10:21 GMT

change the filename and saving location to something like..

r"D:\gis\cnt\x

Whenever you are getting filename errors something is amiss. and given your input raster's name is truncated halfway through a folder name, then it is time to quit trying the same thing, move the raster to a new folder and give it a new name... then try again... if it works, then it was something about the filename, if not... then it is something else.

Re: Count the percentage of positive and Negative pixel within the zone.

XanderBakker — Sat, 11 Dec 2021 16:48:04 GMT

I noticed something strange in the data that you attached:

The folder Count_Pixel.gdb is a folder and not a File Geodatabase as I suspected. The folder contains two subfolders, one with a TIF for the NPP slope values and another with an Esri Grid file with the rasterized zones.

To avoid confusion I placed them in a separate folder and adjusted the code to run with the data. See code below:

#-------------------------------------------------------------------------------
# Name:        count_pos_neg_pixels.py
# Purpose:
#
# Author:      Xander
#
# Created:     06-08-2017
#-------------------------------------------------------------------------------

def main():
    print "load modules..."
    import arcpy
    import numpy as np
    arcpy.env.overwriteOutput = True

    values_ras = r'C:\GeoNet\CountPixels\NPP_slope.TIF'
    zones_ras = r'C:\GeoNet\CountPixels\acz_raster'
    nodata = -3.4028235e+38

    print "create numpy rasters..."
    np_zones = arcpy.RasterToNumPyArray(zones_ras)
    np_values = arcpy.RasterToNumPyArray(values_ras)

    print "read props numpy raster..."
    rows = np_zones.shape[0]
    cols = np_zones.shape[1]
    print " - rows:", rows
    print " - cols:", cols

    print "loop through pixels..."
    pix_cnt = 0
    dct = {}
    for row in range(rows):
        for col in range(cols):
            pix_cnt += 1
            if pix_cnt % 5000 == 0:
                print " - row:", row, "  col:", col, "  pixel:", pix_cnt
            zone = np_zones[row, col]
            value = np_values[row, col]
            if zone > 1:
                if zone in dct:
                    stats = dct[zone]
                    stats = UpdateStats(stats, value, nodata)
                else:
                    stats = CreateStats(value, nodata)
                dct[zone] = stats

    print "\nZone\tsum\tcnt\tmean\tmin\tmax\tcnt_neg\tcnt_pos\tcnt_nodata"
    for zone, stats in sorted(dct.items()):
        print zone, '\t', '\t'.join(str(a).replace('.', ',') for a in stats)


def UpdateStats(stats, value, nodata):
    # sum, cnt, mean, min, max, cnt_neg, cnt_pos, cnt_nodata
    if value > nodata / 2.0:
        stats[0] += value
        stats[1] += 1
        stats[2] = float(stats[0]) / float(stats[1])
        if value < stats[3]:
            stats[3] = value
        if value > stats[4]:
            stats[4] = value
        if value < 0:
            stats[5] += 1
        if value > 0:
            stats[6] += 1
    else:
        stats[7] += 1
    return stats

def CreateStats(value, nodata):
    # sum, cnt, mean, min, max, cnt_neg, cnt_pos, cnt_nodata
    if value > nodata / 2.0:
        return [value, 1, value, value, value, 1 if value < 0 else 0, 1 if value > 0 else 0,  0]
    else:
        return [0, 0, None, 99999999, -9999999, 0, 0, 1]

if __name__ == '__main__':
    main()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The result was loaded into Excel:

When you do it the way Dan and Jayanta explained, you will create a raster with only the positive values:

Create a raster with only the negative values:

Do a Zonal Statistics as Table on the raster with the positive values:

.... and another one on the negative values:

Which will produce this result for the positive values:

and this one for the negative values:

When I compare the count for both methods, they correspond.

Re: Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Sat, 11 Dec 2021 16:48:07 GMT

Hi Xander Bakker‌, Thanks.

I ran the script using same data sets which you used, after running the script I am getting such output which is not matching with your output file which you attached and the Zone column 1 is not appearing. the last column acz_raster also not appearing in my output. I copied the output from PyScripter after running and paste it into Excel.

Below I have attached the output which I received after running the script.

I am running the script on PyScripter and ArcGIS 10.3.

Zone     sum     cnt     mean     min     max     cnt_neg     cnt_pos     cnt_nodata
2      568906,785936     800059     0,711081040194     -5,48433862166     9,41357835423     91714     626752     774
3      304342,163442     389143     0,782083099122     -6,14446017525     4,65285074061     32275     327596     407
4      75973,486289     254708     0,298276796524     -9,89411033284     16,0140284267     64581     146669     1436
5      138842,463549     182611     0,760318182087     -6,15788269043     11,1494489421     26080     147168     157
6      505731,281974     691091     0,731786815303     -8,45667485757     9,71765823364     3174     273474     16118
7      60633,5277998     113542     0,534018493595     -16,3086192868     20,4710965796     18460     85396     495
255      10,6494542035     62     0,171765390379     0,0     2,51514136574     0     8     2371928‍‍‍‍‍‍‍‍

Re: Count the percentage of positive and Negative pixel within the zone.

XanderBakker — Tue, 08 Aug 2017 19:14:51 GMT

My bad. Change the code "if zone > 1:" on line 38 into "if zone < 255:".

In my case the Nodata value of the raster is -128 and therefore I should have used "if zone > 0:". However, in your case the nodata value is 255, so you could check for < 255.

Re: Count the percentage of positive and Negative pixel within the zone.

ShouvikJha — Tue, 08 Aug 2017 19:32:07 GMT

Hi Xander Bakker Yes, now it's appearing perfectly and got exact correct pixel calculation. thanks for such wonderful script and it will save the time and also reduce the chances of human introduce error.

And the last column (acz_raster) did you type it manually into excel, based on the zone sequence in the zone raster or it's automatically come through script?

Because for me it's not appearing. it looks like (below attached) after paste into excel, after "cnt_nodata", there is no other column.

The table which you attached there is complete information. but when i am running not getting the same, is there anything need to change?

Thank you

regards

Shouvik Jha