'ADD_TO_SELECTION' works on the layer that is getting selected, if it already has a selection. Since you are creating the selection on the counties layer first (by attribute), this wells layer is getting a NEW_SELECTION based on the intersect instead of the ADD_TO. Hope that makes sense.
I think you are missing a step though, as @DavidSolari and @JoshuaBixby brought up-making a Featurelayer using the attribute selections on both county and wells first, can set you up for the select by location and invert. *Wells* is the equivalent of sql's %Wells%, so it will include featureclasses with _WaterWells in the name.
fcCounties = r'CountiesLayer'
for fc in arcpy.ListFeatureClasses("*_WaterWells"):
countyName = fc.split(".")[0] #assuming that the feature classes all have the same pattern
# create selection on the county layer by sql
sel_county = arcpy.management.MakeFeatureLayer(fcCounties, "county", f"NAME = '{countyName}'")
# select the wells that are 'assigned' to the county by sql(?)
sel_wells_by_attribute = arcpy.management.MakeFeatureLayer(fc, 'selwells', f"COUNTY = '{countyName}'")
# Intersect the filtered county layer and filtered well layer, removing the intersecting (wells that are within the county) leaving the outliers
outliers = arcpy.management.SelectLayerByLocation(sel_wells_by_attribute, "INTERSECT", sel_county, '', "NEW_SELECTION", "INVERT")
# Copy outliers
arcpy.management.CopyFeatures(outliers, f"{countyName}_WaterWells_Outliers")