No more handles (SWT Error) while batch processing .gdb in python

Question

Hello,

I am processing large number of .gdb files the following way:

* load 500 .gdb objects in CE
* process these 500 objects
* export them as .gdb
* repeat

during this process, CE briefly displays and closes import and export gdb dialog windows as it should but after few iterations (the above process is repeated ~500 times) It hangs up on one of these dialog windows and the script stops, UI freezes thus CE is irresponsible and I can't do anything with it other than close it. When I restart it, everything works as it should after I run the script again and it does another 500 iterations..

the problem that it displays is on the following screenshots - I am unable to post .logs of all the errors because UI is frozen at this state so I cannot open "save log" dialog window but I migh be able to make screenshots of all of them if it is necesarry:

I was searching about this "no more handles SWT error" and found out that it's a common eclipse bug that has something to do with limited number of handles that OS provides to eclipse and when these handles run out an error occours or maybe it's a memory problem but I have 32GB and CE only uses 9GB

Maybe this is not CE-related bug and more like Eclipse-related but I would like to ask if anybody has any idea how to fix it or workaround:

Possible solutions that I can think of:

* increase number of handles that OS provides somehow (maybe edit something in CityEngine.ini)
* make CE cleanup itself after some iterations (~400) by using some python code (maybe run garbage collector etc..)
* force CE to close after some iterations (~400)  and then start again and continue where it left - and repeat - this can work - I can restart CE but I don't know how to force it to run a specific script after startup

any help is appreciated

SimonHaegler · Answer

hi marek

apologies for the late response. it indeed looks like that the export dialog is leaking OS handles, i hope there is time to fix this for 2017.1. for now, let's look at the possible workarounds:

* increase OS handles: there seems to be some advanced ways to increase the OS handle limit (e.g. What's the upper limit on GDI objects for one process in Windows 7? - Stack Overflow ). i doubt you'll be able to tweak your system to get up to 500k exports. also, this is not an CityEngine setting, it is an OS setting and might have undesired side-effects on your workstation.
* restarting CE (part 1): looks like the best workaround for now (although slow). if a script called "startup.py" is present in the workspace root, it will execute automatically at startup (Help). you can use system properties to communicate with the script (e.g. to specify where to continue working):* for example, you start CityEngine with: "CityEngine -vmargs -Dfoo=bar"
* in python, run this:
from java.lang import System
print System.getProperty("foo") # this will print "bar"

* restarting CE (part 2): to shutdown CE correctly please use this script (we are working on a more convenient way):from java import lang
from org.corebounce.threads import ExecDomain
from org.corebounce.threads import Exec
from org.eclipse.swt import SWT
from org.corebounce.rcp import Restart

class SWTThread(lang.Runnable):
     def run(unused):
         doit()

def swt(cmd):
     global doit
     doit = cmd
     ExecDomain.SWT.schedule(SWTThread(), Exec.SYNC)

# shutdown
swt(lambda: Restart.saveAndExit(False))

* changing your workflow: is there a way you can increase the number of objects (500) per exporter run to decrease the number of export runs?

hope this helps a bit,

simon

ps: if you are interested in new things in CE 2017.1, feel free to sign up to the 2017.1 beta program: Welcome to our Feedback Community

MarekDekys · Answer

Hi, I was working with 2016.1 in that time. Now, when 2017.0. (final, non-beta) came out, I tested it again and I believe I have same (or similiar) error: this error happens while exporting meshes of very large area into small tiles in .obj format. No more handles (SWT Error) occoured after around 10k exports (maybe this is the max limit of handles that OS provides to CE) The problem is that the entire area has about 90k tiles - so 90k exports for one layer. If we export several layers for one small tile, by multiplying this number, we can end up having about 500k exports in total. which means restarting it for 50times or more depending where the automatic process stopped working which can take weeks. Is there any way to solve this? maybe increase the number of handles for CE or to decrease handle usage? below is the error log (I changed the long path of our original python script location to C:\python_script.py for better readability): eclipse.buildId=unknown java.version=1.7.0_80 java.vendor=Oracle Corporation BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=cs_CZ Command-line arguments: -os win32 -ws win32 -arch x86_64 -data @noDefault -data @noDefault -clean Error Mon Jul 31 20:22:43 CEST 2017 No more handles (PyException) - Traceback (most recent call last): File "", line 1, in File "C:\python_script.py", line 327, in export_meshes(return_coords(xyz,yy,x,y),xyz_yy,x_y,"mesh") File "C:\python_script.py", line 247, in export_meshes ce.export(ce.getObjectsFrom(ce.scene, ce.isGraphLayer), exportSettings) File "C:\Users\WORKER\.CityEngine\2017.0R.win32.win32.x86_64\jythonCache\jscripting\CE.py", line 434, in export PythonBridge.invoke3(None, 'CE', 'export', (objects), (settings), _jbool(interactive)) at org.python.pydev.jython.PythonBridge$MethodInfo.invoke(Unknown Source) at org.python.pydev.jython.PythonBridge.invoke(Unknown Source) at org.python.pydev.jython.PythonBridge.invoke3(Unknown Source) at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) org.python.pydev.jython.ScriptError: org.python.pydev.jython.ScriptError: No more handles at org.python.core.Py.JavaError(Py.java:516) at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:188) at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:204) at org.python.core.PyObject.__call__(PyObject.java:373) at org.python.core.PyObject.__call__(PyObject.java:377) at jscripting.CE$py.export$24(C:\Users\WORKER\.CityEngine\2017.0R.win32.win32.x86_64\jythonCache\jscripting\CE.py:434) at jscripting.CE$py.call_function(C:\Users\WORKER\.CityEngine\2017.0R.win32.win32.x86_64\jythonCache\jscripting\CE.py) at org.python.core.PyTableCode.call(PyTableCode.java:165) at org.python.core.PyBaseCode.call(PyBaseCode.java:301) at org.python.core.PyBaseCode.call(PyBaseCode.java:157) at org.python.core.PyFunction.__call__(PyFunction.java:368) at org.python.core.PyMethod.__call__(PyMethod.java:151) at org.python.pycode._pyx1327.export_meshes$10(C:\python_script.py:255) at org.python.pycode._pyx1327.call_function(C:\python_script.py) at org.python.core.PyTableCode.call(PyTableCode.java:165) at org.python.core.PyBaseCode.call(PyBaseCode.java:184) at org.python.core.PyFunction.__call__(PyFunction.java:380) at org.python.pycode._pyx1327.f$0(C:\python_script.py:277) at org.python.pycode._pyx1327.call_function(C:\python_script.py) at org.python.core.PyTableCode.call(PyTableCode.java:165) at org.python.core.PyCode.call(PyCode.java:18) at org.python.core.Py.runCode(Py.java:1312) at org.python.core.Py.exec(Py.java:1356) at org.python.pycode._pyx1328.f$0(:1) at org.python.pycode._pyx1328.call_function() at org.python.core.PyTableCode.call(PyTableCode.java:165) at org.python.core.PyCode.call(PyCode.java:18) at org.python.core.Py.runCode(Py.java:1312) at org.python.core.Py.exec(Py.java:1356) at org.python.util.PythonInterpreter.exec(PythonInterpreter.java:206) at org.python.pydev.jython.JythonPlugin.exec(Unknown Source) at org.python.pydev.jython.JProcess.run(Unknown Source) Caused by: org.python.pydev.jython.ScriptError: No more handles at org.python.pydev.jython.PythonBridge$MethodInfo.invoke(Unknown Source) at org.python.pydev.jython.PythonBridge.invoke(Unknown Source) at org.python.pydev.jython.PythonBridge.invoke3(Unknown Source) at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:186) ... 30 more Caused by: org.eclipse.swt.SWTError: No more handles at org.eclipse.swt.SWT.error(SWT.java:4387) at org.eclipse.swt.SWT.error(SWT.java:4276) at org.eclipse.swt.SWT.error(SWT.java:4247) at org.eclipse.swt.internal.ImageList.copyWithAlpha(ImageList.java:179) at org.eclipse.swt.internal.ImageList.set(ImageList.java:409) at org.eclipse.swt.internal.ImageList.add(ImageList.java:70) at org.eclipse.swt.widgets.Button._setImage(Button.java:126) at org.eclipse.swt.widgets.Button.setImage(Button.java:943) at org.corebounce.adapters.ObjectAdapterFactory.createPoint3DUI(Unknown Source) at org.corebounce.adapters.ObjectAdapterFactory.createInputField(Unknown Source) at org.corebounce.adapters.ObjectAdapterFactory.createInputFields(Unknown Source) at org.corebounce.inspectors.AdaptableInspector.createInputFields(Unknown Source) at org.corebounce.inspectors.AdaptableInspector.createPartControl(Unknown Source) at org.corebounce.inspectors.AbstractInspector.createPartControl(Unknown Source) at org.corebounce.inspectors.Inspector.createUIElements(Unknown Source) at org.corebounce.inspectors.Inspector.updateUI(Unknown Source) at org.corebounce.inspectors.Inspector.setInput(Unknown Source) at org.corebounce.inspectors.Inspector.setInput(Unknown Source) at com.procedural.ceui.wizards.ExportSettingsPage.createControl(Unknown Source) at org.eclipse.jface.wizard.WizardDialog.updateForPage(WizardDialog.java:1246) at org.eclipse.jface.wizard.WizardDialog.access$4(WizardDialog.java:1238) at org.eclipse.jface.wizard.WizardDialog$8.run(WizardDialog.java:1227) at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70) at org.eclipse.jface.wizard.WizardDialog.showPage(WizardDialog.java:1225) at org.eclipse.jface.wizard.WizardDialog.nextPressed(WizardDialog.java:915) at org.eclipse.jface.wizard.WizardDialog.buttonPressed(WizardDialog.java:428) at org.corebounce.ui.WizardUtilities$ScriptWizardDialog.buttonPressed(Unknown Source) at com.procedural.ceattribute.ScriptInterfaceExport.export(Unknown Source) at sun.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.python.pydev.jython.PythonBridge$MethodInfo$1.run(Unknown Source) at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35) at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:135) at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4144) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3761) at org.corebounce.rcp.CBDisplay.readAndDispatch(Unknown Source) at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2701) at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2665) at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2499) at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:679) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:668) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149) at com.procedural.cityengine.release.Application.start(Unknown Source) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584) at org.eclipse.equinox.launcher.Main.run(Main.java:1438)

SimonHaegler · Answer

hi

which version of CityEngine are you using? we already fixed a number of handle leaks for CE2017.0 and are currently trying to reproduce your issue.

in the mean time and if you have the time, you could sign up for the 2017.0 beta and try again:

Esri CityEngine

best,

simon