arcmap.exe crashes when exit button is pressed

[iadavuth]

Hello,

I'm debugging a plug-in for arcmap. Arcmap crashes just after the exit button is pressed. These crashes appear on windows vista 64 sp2 or vista 32 (don't remember if sp) but not on windows xp sp3. According to the crash log produced by arcmap, it appears that I have some heap issues, so I debug with page heap activated.

Almost starting from scratch, I took the logo layer example code in which I gradually added my code. My plug-in adds a button which creates my tool. The tool and the button are two separate COM objects. The button is always created, but the tool object is created only if I push the button.
Now, I added some code in the tool part, the code is not executed, the tool object isn't even created, but arcmap crashes. Without that part of code, it does not. It doesn't seem to be dependent of that part of the code, when I comment it and add some code elsewhere (which won't be executed too), arcmap crashes the same way.

adplus tells me : corrupted heap pointer or using wrong heap
xx : Heap used in the call
xx : Heap block
xx : Block size
xx : Heap owning the block

Any idea or debugging strategy I could start with ?

Thanks.

[EdgarBejarano]

Just to clarify, are you developing an Add-In button and tool, or a DLL custom component?  And if so, did you start with an ESRI template in Visual Studio?  I know our community sometimes refers to either our Add-Ins and custom COM components as plug-ins, but I want to make sure this does not involve a Plug-in Data Source and also whether this is an Add-In or custom COM component/DLL.  And which Exit button are you referring to?  ArcMap's built-in Exit button (red X Close button or File menu > Exit) or is this an Exit button you have developed?

I am trying to get a bigger picture of what this involves.

[iadavuth]

I think it's an add-in button and tool : it provides a new button into arcmap.
I don't know how it has been created, I did not start the project but I think it was from scratch using the visual studio 6 wizard.
For the exit button, I talked about the ArcMap's built-in Exit button.

[EdgarBejarano]

Visual Studio 6.0?  What version of ArcGIS Desktop and ArcObjects SDK are you developing with?  It must be 9.3 or 9.3.1, not 10.0?

If 9.3/9.3.1, then the following IDE's are supported:
Visual Basic 6
Visual Studio 2005 or Visual Studio 2008
-Add-Ins are new to ArcGIS 10.0 so you could not be developing an Add-In for ArcGIS 9.3/9.3.1, only custom components that result in a DLL which you register with ArcGIS Desktop.

If 10.0, then the following IDE's are supported:
Visual Studio 2008 or Visual Studio 2010

Regarding
"Now, I added some code in the tool part, the code is not executed, the tool object isn't even created, but arcmap crashes"

May we know what you are adding and where in the custom tool class you add this code?  Is it in the constructor or elsewhere?

[iadavuth]

It's arcmap 9.3.1 build 40500 with service pack 2.
Yes it's a dll that I register with regsvr32. It's a custom component then.

In the button part of the code which is a ICommand, as in the exemple code LogoLayer, the onCreate method contains only
m_ipApp = hook;
the onClick() method contains the creation of the tool.

My tool is a ILayer, some initialisations are made in an init() method, some code in the Draw() method using computations made in another DLL.

Even without clicking on the Button, when the code into the Draw() method is commented, the program works, when not, the debugger stops on the heap issue.

Wondering if the COM objects creation settings have been well done in visual, I tried to put my code in the custom logo layer cexample code. It does the same : heap issue if the draw method code isn't commented even if I don't push the button.

edit : after going a little further, I notice that the declaration of one my object into the .h file of the layer could cause problems. It is declared as MyObject *_pMyObject;
That object is never used. But when i comment that declaration, arcmap seems ok.
Turning MyObject *_pMyObject; to MyObject _myObject;
stills throw the heap problem.

But adding MyObject *_pMyObject; in the .h of a new Custom logo layer example code doesn't reproduces the problem.

[EdgarBejarano]

I understand a little more now but I am still having a hard time picturing it.  I am now realizing that when you say "tool", you are not referring to a custom tool that you are defining in a class that implements ITool or inherits BaseTool, and which when registered successfully, will end up being a tool which you add to a toolbar in ArcMap or ArcCatalog, e.g. Select tool or Identify tool.  Instead, what you mean by tool seems to be something else:

"My tool is a ILayer, some initialisations are made in an init() method, some code in the Draw() method using computations made in another DLL."

I cannot picture how a tool is an ILayer.  Almost sounds like you are referring to a class you are defining in your project, not a tool.  A tool can involve an ILayer type object somewhere in its code but it cannot be an ILayer.

How about if you show an excerpt of the class (showing the name of the class and what it is inheriting or implementing) as well as the procedure (or method if this is C#) and the faulty statement where the issue occurs so I can better see what ArcObjects it involves.

So, I follow this:
"In the button part of the code which is a ICommand, as in the exemple code LogoLayer, the onCreate method contains only
m_ipApp = hook;"

but I do not follow this:
"the onClick() method contains the creation of the tool."

The OnClick of your custom command contains the creation of the tool?  Sounds like you are instantiating one of your classes.  Hopefully that is not an ITool or BaseTool-derived class.  Hopefully what you mean by tool is something else because a custom tool gets created automatically like a custom command when ArcMap (or ArcCatalog) opens and when it finds the tool in the registry. 

I am starting to get the feeling you are working in VC++, specifically.  Is this the case?

[iadavuth]

Yes it is C++.

I've got one COM object MyCommand

class ATL_NO_VTABLE CLogoLayerCommand : 
 public CComObjectRootEx<CComSingleThreadModel>,
 public CComCoClass<CMyCommand, &CLSID_MyCommand>,
 public ISupportErrorInfo,
 public IMyCommand,
 public ICommand

The tool is created using the following code in the onClick() method

hr = m_ipGtScopeLayer.CreateInstance(CLSID_CustomLogoLayer);

The tool part is declared as follow :

class ATL_NO_VTABLE CMyTool : 
 public CComObjectRootEx<CComSingleThreadModel>,
 public CComCoClass<CMyTool, &CLSID_MyTool>,
 public ISupportErrorInfo,
 public ICustomLogoLayer,
 public ILayer,
 public ILayerEffects,
 public IPersistStream