Python script running slower in PyScripter than with IDLE

[BillBass]

Up until recently we used the IDLE interface to write and then run Python scripts for ArcGIS. I had PyScripter (PyScripter v2.5.3 32-bit version) installed on my workstation which is a Windows7 64-bit machine. I am running ArcGIS 10.0 SP3 and the Python version is 32-bit; below is the first line when opening the IDLE program that came with the ArcGIS installation.
Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit (Intel)] on win32

Post installation I noticed several versions of PyScripter IDE to choose from in the Start menu: PyScripter for Python 2.4 thru 3.3, and also one for 'Latest Version'. I assumed that the 2.6 version was the best one to use as from the above message in IDLE, it seems our Python version is 2.6.5. I have noticed that when running Python scripts from inside PyScripter they run much slower (5 times slower) than when running the same program from either a DOS command prompt or the IDLE program.

Any thought as to why this may be occurring?

Thanks.

[MathewCoyle]

Are you sure it is actually running slower or just not updating the print outs as fast? I've noticed in pyscripter if you are rapidly sending print statements it starts bogging down the GUI much more than IDLE or command/power shell. It always ran at the same actual times for me though.

[PhilMorefield]

Mostly speculation, but it might be because Pyscripter is running everything through a true debugger. IDLE is not.

[BillBass]

Some additional observations and run times (HH:MM:SS).
I ran the python script in 3 ways:
1. Using the Windows Command Prompt window. 00:03:43
2. From within PyScripter (no debugging, just a straight run using Run command). 1:21:33
3. Using the Windows Command Prompt Window, but I happended to have the python script open in PyScripter at the same time. 1:21:33
Note: Scenario 3 was the most interesting as it ran just as slow as scenario 2. The only difference being that the python file was open in PyScripter while trying to run from Windows Command Prompt. But if the python script was open in IDLE or Notepad while attempting to run from the Windows Command Prompt, it ran just as fast as in scenario 1 with in a few seconds.

I also noticed under the Run > Python Engine menu, there are 4 options: Internal, Remote, Remote (Tk), and Remote (Wx). By default PyScripter has selected Remote. The following website explained the different options and limitations http://code.google.com/p/pyscripter/wiki/RemoteEngines. So I tried changing it to Internal from Remote. But that just made it worse, as the program never seemed to get through the first step and PyScripter went into Not Responding mode as I was forced to exit the application.

[PhilMorefield]

Some additional observations and run times (HH:MM:SS).
I ran the python script in 3 ways:
1. Using the Windows Command Prompt window. 00:03:43
2. From within PyScripter (no debugging, just a straight run using Run command). 1:21:33
3. Using the Windows Command Prompt Window, but I happended to have the python script open in PyScripter at the same time. 1:21:33
Note: Scenario 3 was the most interesting as it ran just as slow as scenario 2. The only difference being that the python file was open in PyScripter while trying to run from Windows Command Prompt. But if the python script was open in IDLE or Notepad while attempting to run from the Windows Command Prompt, it ran just as fast as in scenario 1 with in a few seconds.

I also noticed under the Run > Python Engine menu, there are 4 options: Internal, Remote, Remote (Tk), and Remote (Wx). By default PyScripter has selected Remote. The following website explained the different options and limitations http://code.google.com/p/pyscripter/wiki/RemoteEngines. So I tried changing it to Internal from Remote. But that just made it worse, as the program never seemed to get through the first step and PyScripter went into Not Responding mode as I was forced to exit the application.

Wow, that's an amazing difference. Much to my relief, I'm not seeing the same problem. I have a process I run hundreds of times that takes about 2 minutes and 30 seconds in IDLE and is maybe just a few seconds slower in Pyscripter (I've always used the Remote engine). I'm not even sure there is a difference, really.

Maybe a reinstall of Pyscripter is in order? Seems like something is definitely off in your case.

[DavidTrillo_Montero]

I am telling my impression about working with PyScripter running ArcGIS scripts.

I was programming a large process (took about 5-7 minutes). Everything was OK because the time of execution was more or less the same inside or outside ArcGIS.

After running several times the script, I notice that PyScripter had become too slow, even for a simple and easy step. I don't know if something is beeing storing somewhere or whatever, but the thruth is that the script run slower!

I decided to try on another computer, so I prepared PyScripter and execute. The same. At the beginning, the execution time was really fast, but after a few tries, the time of running the script became again too slow.

Any idea?

The engine I'm using is "Remote"

[PhilMorefield]

I am telling my impression about working with PyScripter running ArcGIS scripts.

I was programming a large process (took about 5-7 minutes). Everything was OK because the time of execution was more or less the same inside or outside ArcGIS.

After running several times the script, I notice that PyScripter had become too slow, even for a simple and easy step. I don't know if something is beeing storing somewhere or whatever, but the thruth is that the script run slower!

I decided to try on another computer, so I prepared PyScripter and execute. The same. At the beginning, the execution time was really fast, but after a few tries, the time of running the script became again too slow.

Any idea?

The engine I'm using is "Remote"

The original post was comparing run time in Pyscripter to run times in IDLE. Have you tried running this script in IDLE to see if the problem persists?

[DavidTrillo_Montero]

After running my script step by step, I got possible solutions.

One was that arcpy.Exist() take to much if the path has a lot of files! It's faster to use os.path.exist()

I have modified the script in order to avoid a directory increasing and increasing with new files every minute, and takes longer to read with arcpy.

[Anonymous User]

After running my script step by step, I got possible solutions.

One was that arcpy.Exist() take to much if the path has a lot of files! It's faster to use os.path.exist()

I have modified the script in order to avoid a directory increasing and increasing with new files every minute, and takes longer to read with arcpy.

Using os.path.exists() will be faster, but you have to be careful. This will only work for features stored in a file system workspace; os.path.exists() cannot pick up geodatabase feature classes. 

>>> import arcpy, os
>>> fc = r'\\Arcserver1\gis\RICO\M23106103\ESRI\Themes\Connections\Connections.gdb\Connection_Parcels'
>>> shp = r'\\Arcserver1\gis\RICO\_Basemap\ESRI\Themes\Soils\Rico_Soils.shp'
>>> for feat in [shp, fc]:
 print '{0} Exists according to arcpy: {1}'.format(os.path.basename(feat), arcpy.Exists(feat))
 print '{0} Exists according to os: {1}'.format(os.path.basename(feat), os.path.exists(feat))

 
Rico_Soils.shp Exists according to arcpy: True
Rico_Soils.shp Exists according to os: True
Connection_Parcels Exists according to arcpy: True
Connection_Parcels Exists according to os: False

[DavidTrillo_Montero]

I already know it, but ...
Thanks for the tip!