topic Re: How to copy new rows from a shapefile to a feature class in Python Questions

How to copy new rows from a shapefile to a feature class

MaryamPanji — Mon, 25 Jan 2021 01:16:18 GMT

Hi, I am trying to add new features from a shapefile to a feature class sitting in database in sql server. Both have common fields but name of a few fields are different.

My question is that how can I automate this process using a python script.

I am new with Python and I've created this one but I am getting this error:

Runtime error
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "<string>", line 4, in getPrimaryFieldValues
TypeError: 'field_names' must be string or non empty sequence of strings.

My script looks like this:

ID is the field that the script should check it if the values don't exist then insert the rows.

import arcpy

def getPrimaryFieldValues(fc, field😞

values = []

with arcpy.da.SearchCursor(fc, [field]) as cursor:

for row in cursor:

values.append(row[0])

return values

def getSelectCursor(fc, whereClause😞

return arcpy.da.SearchCursor(fc, ["*"], whereClause)

def diff(a, b😞

b = set(b)

return [aa for aa in a if aa not in b]

source = r"original"

destination = r"test"

fieldName =["ID"]

sourceValues = getPrimaryFieldValues(source, fieldName)

destinationValues = getPrimaryFieldValues(destination, fieldName)

add = diff(sourceValues,destinationValues)

with arcpy.da.InsertCursor(destination, ["*"]) as insertCursor:

for a in add:

insertRows = getSelectCursor(source, fieldName + " = " + str(a))

for r in insertRows:

insertCursor.insertRow(r)

Re: How to copy new rows from a shapefile to a feature class

DanPatterson — Mon, 25 Jan 2021 01:54:42 GMT

Code formatting ... the Community Version - GeoNet, The Esri Community

will get rid of all you sad faces in your code and make it more readable 😉

On a quick look, I am not sure if you set up field mapping

Mapping input fields to output fields—ArcGIS Pro | Documentation

Re: How to copy new rows from a shapefile to a feature class

Anonymous User — Mon, 25 Jan 2021 03:32:49 GMT

I am not sure where you are getting your source and destination field list, but here is a function that I use when we need to do field mapping between two featureclasses. Append might be a better option over cursors, but I don't know the specifics of what you want moved. If you must use cursors, this still might help with getting an idea how to get a list of field names to iterate over.

def fieldMapping(destfc=None, sourcefc=None, keepfields=None): """ Function to assist in making joins by mapping all the fields in the destination fc and including a list of fields that is wanted within the source dataset.. """ try: trglist = [i.name for i in arcpy.ListFields(destfc)] # List of fields to keep during Join operation fldMap = arcpy.FieldMappings() # Creating field maps for the two files fldMap.addTable(destfc) fldMap.addTable(sourcefc) keepfields.extend(trglist) # Removing unwanted fields for field in fldMap.fields: if field.name not in keepfields: fldMap.removeFieldMap(fldMap.findFieldMapIndex(field.name)) return fldMap # Error handling... except RuntimeError as e: raise Exception("{} failed.\n{}".format(sys._getframe().f_code.co_name, e.message))

Use the returned field map:

# Process: Create the field map for the join keepfields = ['address', 'zip'] zcodesMap = fieldMapping('destination fc', 'source fc', keepfields)

If there are fields that need to be manually mapped, you can so so like:

# using the field map created by the function, add the fields that have different names zcodesMap.addInputField(destfc, 'fieldname in sql') zcodesMap.addInputField(sourcefc, 'old field name') usefldmap_name = zcodesMap.outputField usefldmap_name.name = 'fieldname in sql' zcodesMap.outputField = usefldmap_name

Then you can use the field map in append, join, etc to move your data.

arcpy.Append_management(srcAdd, address, "NO_TEST", zcodesMap)

Re: How to copy new rows from a shapefile to a feature class

MaryamPanji — Wed, 27 Jan 2021 11:01:08 GMT

Thanks @Anonymous User , I've used cursors cause I wanted to find non-existing rows and then insert them into the feature class. I can change the field names and don't need to do field mapping. There is one field which is unique and I can compare the feature class with the shapefile. if I want to summarise what I want to do is that:

check the shape file based on a field (GeoUnit) and if the values don't exist in GeoUnit of the feature class then insert the rows in feature class.

Re: How to copy new rows from a shapefile to a feature class

MaryamPanji — Wed, 27 Jan 2021 11:27:05 GMT

import arcpy def getPrimaryFieldValues(fc, field): values = [] with arcpy.da.SearchCursor(fc, [field]) as cursor: for row in cursor: values.append(row[0]) return values def getSelectCursor(fc, whereClause): return arcpy.da.SearchCursor(fc, fieldName, whereClause) def diff(a, b): b = set(b) return [aa for aa in a if aa not in b] source = "Original" destination = "Test" fieldName = ['OBJECTID','F_1','F_2','Description','Class','CatShortDesc','CatLongDesc','Condition','GeoUnit','Easting','Northing','created_user','created_date','last_edited_date','SHAPE@XY','Comments','Class_Code'] sourceValues = getPrimaryFieldValues(source, fieldName) destinationValues = getPrimaryFieldValues(destination, fieldName) additions = diff(sourceValues,destinationValues) with arcpy.da.InsertCursor(destination, fieldName) as insertCursor: for a in additions: insertRows = getSelectCursor(source, fieldName) for r in insertRows: insertCursor.insertRow(r)

Re: How to copy new rows from a shapefile to a feature class

MaryamPanji — Wed, 27 Jan 2021 11:28:08 GMT

Thanks Dan, I fixed the script.

Re: How to copy new rows from a shapefile to a feature class

Anonymous User — Wed, 27 Jan 2021 13:49:01 GMT

Thanks for pasting the script.

In your updated code, the whereclause is missing. Using the whereclause in the original pasting, it looks like you were trying to select the fieldname by the sql statement, which will fail because the whereclause is for filtering/selecting the data.

If you are only comparing the one (GeoUnit) field, you only need to grab that value in your lists. I refractored your code a little to:

import arcpy def diff(a, b): # Filter out existing values return [aa for aa in a if aa not in b] if __name__ == "__main__": source = "Original" destination = "Test" fieldNameList = ['OBJECTID', 'F_1', 'F_2', 'Description', 'Class', 'CatShortDesc', 'CatLongDesc', 'Condition', 'GeoUnit', 'Easting', 'Northing', 'created_user', 'created_date', 'last_edited_date', 'SHAPE@XY', 'Comments', 'Class_Code'] additions = diff([x[0] for x in arcpy.da.SearchCursor(source, 'GeoUnit')], [x[0] for x in arcpy.da.SearchCursor(destination, 'GeoUnit')]) # iterate over list of GeoUnit values that are not in the featureclass for a in additions: # select the values in the shapefile based on the whereclause whereclause = f"""GeoUnit = '{a}'""" with arcpy.da.SearchCursor(source, fieldNameList, whereclause) as cur: for row in cur: # Insert the values from the shapefile cursor into the fc. # The order of the fields in the fieldlist matters here so match their index to the index of the destination FC that you are using. with arcpy.da.InsertCursor(destination, fieldNameList) as insertCursor: insertCursor.insertRow(row[0])

Hope this helps, and that I understood your intention. You could do this with dictionaries as well, so you don't have to iterate over the shapefile a second time but this might work just as well without getting too complex.