Monday, February 4, 2008

Debug Visual C++ DLL from VB.NET

FYI, I am running .NET 2003.  I have a VB.NET project that calls functions in a C++ DLL.  Every once in a while, there is a problem with functions in the DLL, and usually we just put MessageBox statements in the DLL, rebuild it and then run the VB.NET program to try to figure out what is wrong.  However, this process takes up time as we have to continually go back and forth until we figure out the problem.  What I wanted was to just debug my C++ code, from the debugger.  I did some google searches and came up with bits and pieces.  The one thing that I found was that a lot of people have asked this question in a number of message boards without any answers.

Using some of the hints suggested, I was able to come up with a method that works.

I created a brand new Visual C++ dll project (called testcplusplusdll), with one cpp file (mainfile.cpp).  This is what it had:

#include <windows.h>
#define CALLINGCONV __declspec(dllexport) __stdcall

int CALLINGCONV Trace() {
    int hgt,wid;
        hgt = 2;
        wid = 2;
        hgt = hgt + wid;
        MessageBox(NULL,"trace1","Trace13",0);   

  return 1;
}

In the Project Properties.  Project->testcplusplusdll Properties
Under Configuration Properties -> Debugging.  For the command I entered the path to the executable of the VB.NET program (we'll get into that in a little bit).  For Attach, I have the value "No", for Debugger Type, I have "Native Only" but others might work (just haven't tried it yet), everything else on that page was left to the default. 

Under the Linker Folder, I kept the default settings, but they are:
under Debugging. for Generate Debug Info I have "Yes (/DEBUG)"
Generate Program Database File - "$(OutDir)/testcplusplusdll.pdb"

That's it for the C++ project.  Build it, and make sure it compiles.  The compiled dll ended up in a Folder called "Debug"

Now, on to the VB.NET project.  I created a new Project.  The location where it put the project wasn't in the same location as the C++ project.  On the form, I placed a button.  And Below is the code that I added to the Form.

    Private Declare Function Trace Lib "..\..\testcplusplusdll\debug\testcplusplusdll.dll" Alias "?Trace@@YGHXZ" () As Integer

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Trace()
    End Sub


A couple of points.  To figure out the Alias of the C++ function, I used the Depends application that came with VB6.  The value for Lib, is  the path to the testcplusplusdll.dll that I compiled earlier.

I build the project.  I am still in the Debug Configuration.

Now, I go back to my C++ project, and as I mentioned earlier, and under onfiguration Properties -> Debugging->Command, I entered the path to the executable of the VB.NET project I just built.

Now I set a beakpoint in the C++ code, and run the C++ project (it takes a while but be patient).  It will popup the Form1, and when I click on Button1, it will stop in my C++ code.  Yeah.


A couple of things that I read about that I tried that didn't make a difference:
1.  In the VB.NET program, under Project Properties under Debugging, there is a checkbox for "Unmanaged Code Debugging", checking it or unchecking it didn't make a difference.
2.  In the VB.NET, if you click on the Solution, and then go to Project->Properties.  You can do "Debug Source Files" and "Debug Symbol Files".  I had entered the location of the c++ source files to this location, and the location of the c++ pdb file in the Symbol Files location, but when I removed it, it didn't seem to make a difference.

Hopefully this is helpful to someone.


No comments: