topic Re: How can search and insert cursors be used to duplicate points? in Python Questions

How can search and insert cursors be used to duplicate points?

ChristopherBevilacqua — Sat, 11 Dec 2021 13:36:04 GMT

I'm trying to use Python to duplicate points in a file geodatabase feature class using an insert cursor in combination with a search cursor. The points represent wells, and I want to create a duplicate point for each geologic reservoir with which the well is associated. A where clause in the search cursor is used to identify the points to be duplicated, and then the insert cursor replicates the rows in the search cursor, with some minor modifications. This appears to work fine when I run the code with the first reservoir. So I copy and past that bit the code, make some minor modifications to reflect the next reservoir for which I want to create duplicates, and run it again. The problem is that each time I run the block of code, I get additional records inserted into the feature class. In other words, there are 63 wells that meet the criteria for duplication. When I duplicate these with the cursors for ATOKA, 63 rows is what I get. However, I get 68 CLFKL records and 78 CLFKU records. I want to do this for a total of 10 reservoirs. By time I get to the 8th or 9th one, the code appears to hang but is actually creating rows. I let it run overnight and had a feature class with almost a GB of points the next morning.

I'm using ArcGIS 10.1 and PyScripter.

Here's a sample of the code:

import arcpy

allPoints = r'\\GIS\allPoints'

# Duplicate features for Clear Fork, Lower Wolfcamp, Strawn, Wolfberry vertical wells in allPoints fc and calculate fcEUR values
# Create search cursor to identify Clear Fork, Strawn, and Wolfberry values in allPoints fc
try:
    print "Duplicating vertical wells - ATOKA"
    whereClause = "\"RESERVOIR\" = 'CLEAR FORK' OR \"RESERVOIR\" = 'STRAWN' OR \"RESERVOIR\" = 'WOLFBERRY' AND \"EUR\" IS NOT NULL AND \"RESERVOIR2\" IS NULL"

    sCur = arcpy.da.SearchCursor(allPoints,["SHAPE@XY","PROPNUM","WellID","RESV_CAT","RESERVOIR","HORIZONTAL","EUR","VOOIPratio","fcEUR","RESERVOIR2"],whereClause)

    #Create insert cursor that will be used to create duplicate points that represent each vertical well within the Atoka formation.
    iCur = arcpy.da.InsertCursor(allPoints,["SHAPE@XY","PROPNUM","WellID","RESV_CAT","RESERVOIR","HORIZONTAL","EUR","VOOIPratio","fcEUR","RESERVOIR2"])

    for sRow in sCur:
        XY = sRow[0]
        PROP = sRow[1]
        WellID = sRow[2]
        RESV_CAT = sRow[3]
        RES = sRow[4]
        HZ = sRow[5]
        EUR = sRow[6]
        VOOIPratio = 0.00425532
        fcEUR = sRow[6] * 0.00425532
        RES2 = "ATOKA"

        iCur.insertRow((XY,PROP,WellID,RESV_CAT,RES,HZ,EUR,VOOIPratio,fcEUR,RES2))

    del sCur, iCur
    del sRow

except Exception as e:
    print (e.message)

###########
try:
    print "Duplicating vertical wells - CLFKL"
    whereClause = "\"RESERVOIR\" = 'CLEAR FORK' OR \"RESERVOIR\" = 'STRAWN' OR \"RESERVOIR\" = 'WOLFBERRY' AND \"EUR\" IS NOT NULL AND \"RESERVOIR2\" IS NULL"

    sCur = arcpy.da.SearchCursor(allPoints,["SHAPE@XY","PROPNUM","WellID","RESV_CAT","RESERVOIR","HORIZONTAL","EUR","VOOIPratio","fcEUR","RESERVOIR2"],whereClause)

    #Create insert cursor that will be used to create duplicate points that represent each vertical well within the CLFKL formation.
    iCur = arcpy.da.InsertCursor(allPoints,["SHAPE@XY","PROPNUM","WellID","RESV_CAT","RESERVOIR","HORIZONTAL","EUR","VOOIPratio","fcEUR","RESERVOIR2"])

    for sRow in sCur:
        XY = sRow[0]
        PROP = sRow[1]
        WellID = sRow[2]
        RESV_CAT = sRow[3]
        RES = sRow[4]
        HZ = sRow[5]
        EUR = sRow[6]
        VOOIPratio = .1617021
        fcEUR = sRow[6] * .1617021
        RES2 = "CLFKL"

        iCur.insertRow((XY,PROP,WellID,RESV_CAT,RES,HZ,EUR,VOOIPratio,fcEUR,RES2))

    del sCur, iCur
    del sRow

except Exception as e:
    print (e.message)
##############‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Any suggestions would be greatly appreciated.

Re: How can search and insert cursors be used to duplicate points?

BlakeTerhune — Sat, 11 Dec 2021 13:36:07 GMT

Here's my take on what I think you're trying to accomplish. It looks like your whereClause and field names are the same for both ATOKA and CLFKL so I condensed the cursors together. I also reformatted your whereClause because it looked strange. However, I'm not sure your allPoints variable is correctly pointing to a feature class.

try:
    allPoints = r'\\GIS\allPoints'

    whereClause = (
        "RESERVOIR IN('CLEAR FORK', 'STRAWN', 'WOLFBERRY')"
        " AND EUR IS NOT NULL"
        " AND RESERVOIR2 IS NULL"
    )

    fields = [
        "SHAPE@XY",
        "PROPNUM",
        "WellID",
        "RESV_CAT",
        "RESERVOIR",
        "HORIZONTAL",
        "EUR",
        "VOOIPratio",
        "fcEUR",
        "RESERVOIR2"
    ]

    with arcpy.da.SearchCursor(allPoints, fields, whereClause) as sCur:
        with arcpy.da.InsertCursor(allPoints, fields) as iCur:
            for row in sCur:
                # Duplicate row modifications for ATOKA
                row[7] = 0.00425532  ## VOOIPratio
                row[8] = row[6] * 0.00425532  ## fcEUR
                row[9] = "ATOKA"  ## RESERVOIR2
                iCur.insertRow(row)

                # Duplicate row modifications for CLFKL
                row[7] = 0.1617021  ## VOOIPratio
                row[8] = row[6] * 0.1617021  ## fcEUR
                row[9] = "CLFKL"  ## RESERVOIR2
                iCur.insertRow(row)

except Exception as e:
    print e‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: How can search and insert cursors be used to duplicate points?

LukeWebb — Sat, 11 Dec 2021 13:36:10 GMT

This is caused because you are using the same input data, and output data, and where clause.

As you dont change the value for duplicated points field:

row[4]  #Reservoir

When you create the points the first time round the script properly works.

Next time, when run the where clause picks up the original input points, plus these newly created ones! So its working correctly.

You could add a field say 'Source' and populate with 'Input Data', and then for the duplicate rows populate it with a different value 'Dupe point', and include this in the where clause.

e.g.

    whereClause = (
        "RESERVOIR IN('CLEAR FORK', 'STRAWN', 'WOLFBERRY')"
        " AND EUR IS NOT NULL"
        " AND RESERVOIR2 IS NULL   and   SOURCE = 'Input Data'"    
    )

Re: How can search and insert cursors be used to duplicate points?

BlakeTerhune — Fri, 06 Jan 2017 15:38:01 GMT

Are you referring to cbevilacqua‌'s original script or the one I posted? I only create the cursor once before any inserts so shouldn't it miss any of the new, duplicate records it adds?

Re: How can search and insert cursors be used to duplicate points?

LukeWebb — Fri, 06 Jan 2017 15:41:15 GMT

You are correct I didnt see you had fixed that part, by processing the values simulataneously in one pass of the cursor.

Both solutions may be helpful as he may not know in advance all combinations required to be created maybe

Re: How can search and insert cursors be used to duplicate points?

ChristopherBevilacqua — Mon, 09 Jan 2017 15:17:45 GMT

Thanks Blake Terhune. I'll try this out today and will reply with results.

Re: How can search and insert cursors be used to duplicate points?

ChristopherBevilacqua — Mon, 09 Jan 2017 15:23:37 GMT

Thanks @Luke Webb. I believe I was avoiding re-duplication with

AND \"RESERVOIR2\" IS NULL‍

in the where clause, since this field is blank for all of the original rows. When I added this text to the where clause it did cut down on the number of rows that were being inserted, but I was still getting some extras.

Re: How can search and insert cursors be used to duplicate points?

ChristopherBevilacqua — Sat, 11 Dec 2021 13:36:13 GMT

I tried out the code you suggested. This looks like a much more efficient way to do what I want to do. However, I get the following error:

'tuple' object does not support item assignment

I thought this might be because of the where clause, since that's basically the only tuple that isn't holding cursor parameters. I tried using my original where clause, but got the same error. So, I inserted a few print statements and found that the code executes until line 27 in @Blake Terhune's code, where the field values for the insert cursor are defined, before giving the error. When I comment out lines 27-29 and 33-35 in @Blake Terhune's code, the insert cursor adds the correct number of rows to the feature class, but with null values in the RESERVOIR2, VOOIPratio, and fcEUR fields.

I tried rearranging the way the cursors were nested, like this:

    with arcpy.da.SearchCursor(allPoints, fields, whereClause) as sCur:
        for row in sCur:
            # Duplicate row modifications for ATOKA

            with arcpy.da.InsertCursor(allPoints, fields) as iCur:
                row[7] = 0.00425532  ## VOOIPratio
                row[8] = row[6] * 0.00425532  ## fcEUR
                row[9] = "ATOKA"  ## RESERVOIR2
                iCur.insertRow(row)

        for row in sCur:
            # Duplicate row modifications for CLFKL

            with arcpy.da.InsertCursor(allPoints, fields) as iCur:
                row[7] = 0.1617021  ## VOOIPratio
                row[8] = row[6] * 0.1617021  ## fcEUR
                row[9] = "CLFKL"  ## RESERVOIR2
                iCur.insertRow(row)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

...which did not work and gave the same error as above.

Any suggestions would be greatly appreciated.

Re: How can search and insert cursors be used to duplicate points?

BlakeTerhune — Sat, 11 Dec 2021 13:36:15 GMT

Ah, I forgot tuples are immutable so you can't change them; sorry! You have to convert the row tuple to a list to change it. Here's the updated code:

try:
    allPoints = r'\\GIS\allPoints'

    whereClause = (
        "RESERVOIR IN('CLEAR FORK', 'STRAWN', 'WOLFBERRY')"
        " AND EUR IS NOT NULL"
        " AND RESERVOIR2 IS NULL"
    )

    fields = [
        "SHAPE@XY",
        "PROPNUM",
        "WellID",
        "RESV_CAT",
        "RESERVOIR",
        "HORIZONTAL",
        "EUR",
        "VOOIPratio",
        "fcEUR",
        "RESERVOIR2"
    ]

    with arcpy.da.SearchCursor(allPoints, fields, whereClause) as sCur:
        with arcpy.da.InsertCursor(allPoints, fields) as iCur:
            for row in sCur:
                # Convert row tuple from search cursor to list for modifications
                new_row = list(row)

                # Duplicate row modifications for ATOKA
                new_row[7] = 0.00425532  ## VOOIPratio
                new_row[8] = row[6] * 0.00425532  ## fcEUR
                new_row[9] = "ATOKA"  ## RESERVOIR2
                iCur.insertRow(new_row)

                # Duplicate row modifications for CLFKL
                new_row[7] = 0.1617021  ## VOOIPratio
                new_row[8] = row[6] * 0.1617021  ## fcEUR
                new_row[9] = "CLFKL"  ## RESERVOIR2
                iCur.insertRow(new_row)

except Exception as e:
    print e‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: How can search and insert cursors be used to duplicate points?

ChristopherBevilacqua — Mon, 09 Jan 2017 18:49:45 GMT

Thanks Blake. That works, and I learned something new!