//
you're reading...
Interop Marshaling, Managed Structures

Passing a Pointer to a Structure from C# to C++ Part 2.

1. Introduction.

1.1 In Passing a Pointer to a Structure from C# to C++ Part 1 I demonstrated the basic principles of marshaling a pointer to a structure from managed code (C#) to unmanaged code (C++).

1.2 In part 1, I used a simple managed structure with a single Int32 type member. The idea is to demonstrate a managed structure with only blittable types.

1.3 Here in part 2, I move up the level of complexity to include non-blittable types in the managed structure.

1.4 Here is where special care must be taken to ensure not only proper marshaling but also proper memory disposal.

1.5 A non-blittable type is one which does not have a common memory representation between managed and unmanaged memory.

1.6 A managed string is a good example. It can be represented in unmanaged memory as a BSTR, or as a C-style NULL-terminated character array. Furthermore, as a C-style string, it may be Unicode or ANSI based.

1.7 Such non-blittable types, when required to be marshaled to unmanaged memory, will require being decorated with MarshalAsAttributes in order to specify to the interop marshaler how to perform proper marshaling.

1.8 And, as this blog will show, these MarshalAsAttributes also will specify to the interop marshaling how to perform proper memory deallocation for these non-blittable types.

2. A Simple Structure with a Non-Blittable Type.

2.1 Let’s say we have the following managed structure :

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1)]
public struct TestStructComplex
{
  [MarshalAs(UnmanagedType.LPStr)]
  public string m_str01;
}

This structure has a single member field m_str01 of type string. As mentioned in section 1, a string is a non-blittable type. Here, the string member is decorated with the MarshalAsAttribute indicating that it will be marshaled across as a C-style NULL-terminated array of ANSI characters (the CharSet argument for the StructLayoutAttribute indicates that the character set to be used for marshaling is ANSI).

2.2 Such a structure would have the following C++ counterpart :

#pragma pack(1)

struct TestStructComplex
{
  LPCSTR m_str01;
};

2.3 Let’s say we have the following API written in C++ :

void __stdcall DisplayTestStructComplex
(
  /*[in]*/ TestStructComplex test_struct_complex
)
{
  printf
  (
    "test_struct_complex.m_str01 : [%s].\r\n",
    test_struct_complex.m_str01
  );
}

The TestStructComplex parameter test_struct_complex is passed by value into the API.

2.4 The following is how it would be declared in C# :

[DllImport("TestDLL01.dll", CallingConvention = CallingConvention.StdCall)]
private static extern void DisplayTestStructComplex([In] TestStructComplex test_struct_complex);

2.5 And the following is a sample call to this API :

static void DisplayTestStructComplex()
{
  TestStructComplex test_struct_complex = new TestStructComplex();

  test_struct_complex.m_str01 = "ABCDEFGHIJ";

  DisplayTestStructComplex(test_struct_complex);
}

2.6 When the DisplayTestStructComplex() API is to be called, the interop marshaler will build an unmanaged version of TestStructComplex and then pass it to the API.

2.7 Now because the TestStructComplex parameter is to be passed by value, the unmanaged version of the structure will be built on the call stack.

2.8 When DisplayTestStructComplex() completes, the memory occuppied by test_struct_complex (on the stack) is automatically recovered.

2.9 However, what happens to the m_str01 field of the structure ? How is it allocated in the first place to allow the API to access it flawlessly ? And how about when the memory for the test_struct_complex parameter is recovered ? How was the memory for m_str01 recovered ?

2.10 This is where the interop marshaler comes into play. When the unmanaged version of TestStructComplex is created on the stack, additional memory is allocated for the m_str01 field. Since m_str01 is declared to be marshaled as a LPSTR of ANSI characters, the number of bytes to allocate depends on the value of the managed m_str01 field at runtime :

test_struct_complex.m_str01 = "ABCDEFGHIJ";

Hence 11 ANSI characters (including the NULL-terminating character) will be allocated which works out to 11 bytes. The memory allocation is performed by Marshal.AllocCoTaskMem() (which eventually calls the ::CoTaskMemAlloc() API).

After these 11 bytes have been allocated somewhere in unmanaged memory, the characters from the managed m_str01 field will be copied to this buffer. Note that the managed m_str01 field is a string and so it internally stores its characters in Unicode. The interop marshaler (under directive from the StructLayoutAttribute’s CharSet argument which equals CharSet.Ansi) will convert this Unicode string into ANSI.

Thereafter, a pointer to this ANSI character array will be set as the value of the m_str01 field (an LPCSTR) of the unmanaged TestStructComplex structure.

2.11 When the DisplayTestStructComplex() API is being executed, the following will be the contents of the test_struct_complex parameter :

It contains the byte sequence of its single member field m_str01 in hex : 10 f7 76 00. This works out to the memory address 0x0076f710.

When we examine this memory address, we will see the following :

The ANSI NULL-terminated C-style string value is clearly displayed.

2.12 When the DisplayTestStructComplex() API completes, the string at 0x0076f710 is automatically deleted by the interop marshaler.

3. When the Structure is Passed as a Pointer.

3.1 If TestStructComplex is to be passed as a pointer to an API, the rules are the same : an unmanaged representation of the structure must be constructed in unmanaged memory. After such an unmanaged structure has been built up, a pointer to it can be passed to the API.

3.2 Let’s say we have the following C++ API :

void __stdcall DisplayTestStructComplex_ByPointer
(
  /*[in]*/ TestStructComplex* ptest_struct_complex
)
{
  printf
  (
    "ptest_struct_complex -> m_str01 : [%s].\r\n",
    ptest_struct_complex -> m_str01
  );
}

3.3 The following is how DisplayTestStructComplex_ByPointer() is declared in C# :

[DllImport("TestDLL01.dll", CallingConvention = CallingConvention.StdCall)]
private static extern void DisplayTestStructComplex_ByPointer([In] IntPtr ptest_struct_complex);

3.4 Now, similar to the situation that we saw in part 1, because the parameter is declared simply as an IntPtr with no MarshalAsAttribute attached, the interop marshaler will not be able to help us perform automatic marshaling.

3.5 An IntPtr can point to anything and so the interop marshaler will not know what exactly to do with it concerning marshaling. We must do the marshaling manually in code.

3.6 Once again, the various Marshal class methods come to the rescue. The following sample C# code demonstrates how a pointer to a TestStructComplex structure is to be prepared and how the DisplayTestStructComplex_ByPointer() API is called :

static void DisplayTestStructComplex_ByPointer()
{
  TestStructComplex test_struct_complex = new TestStructComplex();

  test_struct_complex.m_str01 = "ABCDEFGHIJ";

  int iSizeOfTestStructComplex = Marshal.SizeOf(typeof(TestStructComplex));
  IntPtr ptest_struct_complex = Marshal.AllocHGlobal(iSizeOfTestStructComplex);
  Marshal.StructureToPtr(test_struct_complex, ptest_struct_complex, false);

  DisplayTestStructComplex_ByPointer(ptest_struct_complex);

  Marshal.DestroyStructure(ptest_struct_complex, typeof(TestStructComplex));
  Marshal.FreeHGlobal(ptest_struct_complex);
  ptest_struct_complex = IntPtr.Zero;
}

This code is very similar to the one we met in part 1 (point 3.7) but with an additional call to the Marshal.DestroyStructure() method (highlighted in bold in the code).

The following are the pertinent points :

  • A TestStructComplex structure is allocated and its m_str01 member set with a string value (“ABCDEFGHIJ”).
  • Using Marshal.SizeOf(), we calculate the size of the TestStructComplex as it would be in unmanaged memory. That is, we want the size of an unmanaged version of TestStructComplex. This will work out to 4 (the size of a pointer to a C-style string).
  • We then use Marshal.AllocHGlobal() to allocate in unmanaged memory a block of memory of this size.
  • Next, we use Marshal.StructureToPtr() to copy the field values of the managed TestStructComplex structure to its unmanaged counterpart structure.
  • Now, because the m_str01 field has been declared to be marshaled as a pointer to a C-style ANSI string, the Marshal.StructureToPtr() function will internally allocate in memory a string buffer that would fit the current string value of the managed m_str01 expressed as a NULL-terminated ANSI character string. After this buffer is created, the string value in m_str01 is copied to this buffer, expressed as an ANSI string.
  • Then the DisplayTestStructComplex_ByPointer() API may be called with a pointer to the unmanaged TestStructComplex structure passed as parameter.
  • After the API completes, we need to manually free the memory occuppied by the unmanaged TestStructComplex as well as the string buffer pointed to by its m_str01 member.
  • This is very important. In the example code of section 2, where we passed TestStructComplex by value, the freeing of this buffer is done automatically by the interop marshaler.
  • But this time, because the interop marshaler is not used, we must free this memory ourselves. This is where the Marshal.DestroyStructure() function comes in.
  • Marshal.DestroyStructure() will free up any sub-structures or non-blittable fields of a target structure. This will be so as long as the appropruate MarshalAsAttributes have been used on the fields.
  • Calling it on ptest_struct_complex will ensure that the string buffer pointed to by its m_str01 member will be deallocated.
  • Now, if Marshal.DestroyStructure() is not called on ptest_struct_complex, the string buffer will remain in memory and will result in memory leakage.
  • After the Marshal.DestroyStructure() has been called, the entire memory block occuppied by the unmanaged structure must itself be freed. This is done by calling Marshal.FreeHGlobal().

3.7 Through snapshots of the memory viewer, I will demonstrate the freeing of the unmanaged m_str01 field. When the DisplayTestStructComplex_ByPointer() API is being executed, the following is the memory layout of the ptest_struct_complex parameter :

This is just like the example code demonstrated in section 2 previously. If we take a look at the memory address where the m_str01 field points to (i.e. 0x005EB1D0), we will see the following :

So surprise so far as things closely resembles the example given in section 2 above.

3.8 Now, the difference between this example and the previous one is that after DisplayTestStructComplex_ByPointer() returns, this memory buffer for the C-style string will not be freed.

3.9 It is only freed when Marshal.DestroyStructure() is called on ptest_struct_complex. This is a very important piece of enhancement over the code example given in part 1 section 3.

3.10 Hence the code in point 3.6 really should be the one used as boilerplate code for calling APIs that take structures by pointer. Even if the structure were to contain only blittable field types, calling Marshal.DestroyStructure() will not cause any harm.

3.11 This concept can easily extend to even more complex unmanaged structures that may contain SAFEARRAYs, BSTRs or even other structures. As long as MarshalAsAttributes have been properly decorated on these fields, Marshal.DestroyStructure() can be used to free them.

4. Limitations of Marshal.DestroyStructure().

4.1 Note well, however, that this will not work for fields which are themselves IntPtr types, e.g. :

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1)]
public struct TestStructComplex2
{
  [MarshalAs(UnmanagedType.LPStr)]
  public string m_str01;
  public IntPtr pSomething;
}

In the above structure, pSomething is declared simply as of type IntPtr.

4.2 If we have code like the following (which closely resembles the code in DisplayTestStructComplex_ByPointer()) :

static void DemonstrateTestStructComplex2()
{
  TestStructComplex2 test_struct_complex_2 = new TestStructComplex2();

  test_struct_complex_2.m_str01 = "ABCDEFGHIJ";
  test_struct_complex_2.pSomething = Marshal.StringToCoTaskMemAnsi("12345");

  int iSizeOfTestStructComplex2 = Marshal.SizeOf(typeof(TestStructComplex2));
  IntPtr ptest_struct_complex_2 = Marshal.AllocHGlobal(iSizeOfTestStructComplex2);
  Marshal.StructureToPtr(test_struct_complex_2, ptest_struct_complex_2, false);

  Marshal.DestroyStructure(ptest_struct_complex_2, typeof(TestStructComplex2));
  Marshal.FreeHGlobal(ptest_struct_complex_2);
  ptest_struct_complex_2 = IntPtr.Zero;
}

When Marshal.DestroyStructure() is called on ptest_struct_complex_2, the string pointed to by the pSomething field will remain. It will not be freed.

4.3 To free the memory occuppied by pSomething, we need to use Marshal.FreeCoTaskMem() since it matches how the memory was allocated previously, using Marshal.StringToCoTaskMemAnsi() :

Marshal.FreeCoTaskMem(test_struct_complex_2.pSomething);

4.4 However, note that there is no generic way to free the memory pointed to by structure fields which are only declared as of type IntPtr (e.g. TestStructComplex2.pSomething). It must be freed using the memory freeing method conjugal to the memory allocation method.

5. In Conclusion.

5.1 I hope the reader will now have a clearer picture of how structures may be passed by value and by pointer.

5.2 A good understanding of memory allocation for structures and sub-structures is very essential indeed to ensure adequate memory protection against leakage.

5.3 In part 3, I shall demonstrate the freeing of a structure of even greater complexity : one which embeds another structure.

Advertisements

About Lim Bio Liong

I've been in software development for nearly 20 years specializing in C , COM and C#. It's truly an exicting time we live in, with so much resources at our disposal to gain and share knowledge. I hope my blog will serve a small part in this global knowledge sharing network. For many years now I've been deeply involved with C development work. However since circa 2010, my current work has required me to use more and more on C# with a particular focus on COM interop. I've also written several articles for CodeProject. However, in recent years I've concentrated my time more on helping others in the MSDN forums. Please feel free to leave a comment whenever you have any constructive criticism over any of my blog posts.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: