Problem when I try to overwrite with Python

[Anonymous User]

Original User: diegollamas

Hi,

I create a geoprocessing model that basically compare two features class and if it find a new record it write it to a feature class. we export this model to python script so this process could run every day at 6 pm automatically. when we run this model in model builder it works just find but when we run it after export it i got an error saying that some layers already exist but thats why we are overwriting the information.

this is the script and we added two lines to force it to overwrite
[ATTACH=CONFIG]19544[/ATTACH]
[ATTACH=CONFIG]19545[/ATTACH]
[ATTACH=CONFIG]19546[/ATTACH]

and when we run this process from Python this is the error
IDLE 2.6.5      ==== No Subprocess ====
>>>
Traceback (most recent call last):
  File "C:\CrimeMapping\Replicacion\POLICIA\SyncComandanciaCarolina.py", line 86, in <module>
    arcpy.gp.DetectarDiferenciasA(Incidencia_2012, IncidenciaSDE_A)
  File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\geoprocessing\_base.py", line 474, in <lambda>
    return lambda *args: val(*gp_fixargs(args))
ExecuteError: Failed to execute. Parameters are not valid.
WARNING 000258: Output C:\CrimeMapping\workspace\CAROLINAGIS.gdb\Equals already exists
WARNING 000258: Output C:\CrimeMapping\workspace\CAROLINAGIS.gdb\Diferencia_Incidentes already exists
WARNING 000258: Output C:\CrimeMapping\workspace\CAROLINAGIS.gdb\Incidentes_tmp already exists
WARNING 000970: The join field id_incidente in the join table Incidentes_tmp is not indexed. To improve performance, we recommend that an index be created for the join field in the join table.
WARNING 000258: Output C:\CrimeMapping\workspace\CAROLINAGIS.gdb\Nuevos_Incidentes already exists
ERROR 000655: An error was encountered while Validating  Synchronize.
Failed to execute (DetectarDiferenciasA).

>>>
True
Traceback (most recent call last):
  File "C:\CrimeMapping\Replicacion\POLICIA\SyncComandanciaCarolina.py", line 86, in <module>
    arcpy.gp.DetectarDiferenciasA(Incidencia_2012, IncidenciaSDE_A)
  File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\geoprocessing\_base.py", line 474, in <lambda>
    return lambda *args: val(*gp_fixargs(args))
ExecuteError: Failed to execute. Parameters are not valid.
WARNING 000258: Output C:\CrimeMapping\workspace\CAROLINAGIS.gdb\Equals already exists
WARNING 000258: Output C:\CrimeMapping\workspace\CAROLINAGIS.gdb\Diferencia_Incidentes already exists
WARNING 000258: Output C:\CrimeMapping\workspace\CAROLINAGIS.gdb\Incidentes_tmp already exists
WARNING 000970: The join field id_incidente in the join table Incidentes_tmp is not indexed. To improve performance, we recommend that an index be created for the join field in the join table.
WARNING 000258: Output C:\CrimeMapping\workspace\CAROLINAGIS.gdb\Nuevos_Incidentes already exists
ERROR 000655: An error was encountered while Validating  Synchronize.
Failed to execute (DetectarDiferenciasA).


As you can see we have a true, that means that the command line for overwrite is working

Does any one has any idea what could be happening? why is working find if I run the model but after exporting to python we got that error?


Regards,

Diego Llamas

[Anonymous User]

Original User: bosewicht

In my experience, I've seen that exporting a model to python and running it sometimes has odd results like what you are experiencing.Are you trying to append the new data? if so, just use arcpy.Append_management() in your script. Or perhaps someone much better at python than myself would have a better solution for you.

[JamesCrandall]

I'd move some of the join management stuff you are doing to the IN_MEMORY space.  You can still perform your selections and count processes there, but you can mitigate some of the locking issues (from my experience) by working with intermediate data in_memory.

Also, from what I understand the overwrite setting will not work as expected if you are not properly releasing references along the way.  This is another reason why I like the in_memory space to perform these kinds of things -- it is very easy to "clear" it when needed.

[Anonymous User]

Original User: Wayne_Whitley

I question whether the model was exported to script properly, and this is a common problem, see the blog:

Considerations when exporting a model to a Python script
by jpardy84 and Dale Honeycutt on June 24, 2011
http://blogs.esri.com/esri/arcgis/2011/06/24/exportmodeltopy/

...I'd check the code along the lines and heed the warnings at:
arcpy.gp.toolbox

And if this was appropriately interpreted, it could be the env var overwrite property is being overridden by the toolbox settings, i.e., may be a default value.

[Anonymous User]

You can use if statements to check if it exists on disk first, then delete it before creating a new file. However, most of the time the overwriteOutput works fine for me.  Make sure you do not have these files open in Catalog or ArcMap first because that can create locks.  This is how I handle this problem when the overwriteOutput is not properly working:

if arcpy.Exists(feature_class):
    arcpy.Delete_management(feature_class)

I would put this in front of all the feature classes that are not being overwritten with the environment setting to be safe.

[Anonymous User]

Original User: Wayne_Whitley

Yeah, interesting and I did not check this out to the length I would like, but I have a problem with where the error trace back was made...although you guys are right about 'clean-up' or control of the outputs so that new outputs can be made, I cannot help being suspicious about the reference to _base.py from the geoprocessing folder in the install directory.  To messages are reported in reverse order with the failure being invalid args passed to the aforementioned _base.py.

You do have another execute error toward the beginning in the detecting differences, Validating Synchronize, so that's a good lead too.  I believe the Warnings are still executing the overwrites, just you are being served notice.

So I still think you have errors in converting the model that should be looked at - in fact you have 'red ink' comment lines recorded by the ModelBuilder export process telling you that hey, Houston, we (may) have a problem!  [hey, just a little humor I'm trying for here, so excuse me].

Hope that helps at least a little, sorry don't have more - suggest reading the ESRI blog on porting model to py provided.

[DarrenWiens2]

Apparently the parameters are not valid in the line:

arcpy.gp.DetectarDiferenciasA(Incidencia_2012, IncidenciaSDE_A)

So, what parameters does that tool require, and are you sure they match with "Incidencia_2012" and "IncidenciaSDE_A"?

[Anonymous User]

Original User: Wayne_Whitley

Did you solve your problem?  I am interested to know more, particularly how existing model and script tools are called by other custom gp tools or scripts....I don't know what you'd call this, a 'compound' tool?  It can make things a little more challenging to trace back and find where something isn't passed properly (and, as I suspect in your case, where something didn't convert properly from modelbuilder to python)

Anyway, I agree with Darren that the 'lost in translation' is evident at DetectarDifferenciasA, which if I am not mistaken is a separate custom model component?  If you can, would you please attach your custom toolbox housing these models for further troubleshooting - or provide some feedback, if you have solved your problem, how you did it?