//
you're reading...
Interop Marshaling

Passing Multi-Dimensional Managed Array To C++ Part 1

1. Introduction.

1.1 Multi-dimensional arrays, when exchanged between managed and unmanaged code, form an interesting subject. It is technically possible to achieve this but a good understanding of how managed and unmanaged arrays are organized is crucial in ensuring successful interop transfer.

1.2 In this series of blogs, we shall discuss how managed arrays can be exchanged between managed code and (specifically C++) unmanaged code. To keep things simple and easily extrapolatable, we shall focus on arrays with blittable data types, specifically the integer data type. We shall also focus on the passing of arrays as parameters, not as part of a structure.

1.3 Here in part 1, we shall first explore how to pass the data of a multi-dimensional managed array to unmanaged code using C-style array techniques.

1.4 In the second part of the series, we shall continue our study of multi-dimensional array passing from managed code to unmanaged code using SafeArray techniques (a considerably complex subject of its own).

1.5 In later blogs we shall look into receiving multi-dimensional array data from unmanaged code to managed code. We shall finally discuss how to exchange multi-dimensional arrays bi-directionally.

2. A Primer on C-Style Multi-Dimensional Arrays.

2.1 This section provides a brief discussion of C-style arrays (which are relevant for C++ as well) with a specific focus on multi-dimensional array indexing and data storage.

For greater detail on C/C++ arrays, please refer to the following documentation :

http://www.cplusplus.com/doc/tutorial/arrays/

2.2 An array, by definition, holds data that is stored contiguously in memory. When it comes to being multi-dimenional, a C-style array stores its data exactly the same way as a single-dimensional array of the same size. For example :

int int_array_1[3][5];
int int_array_2[15];

Here, “int_array_1” and “int_array_2” are logically the same. Their data contents are stored in exactly the same manner.

2.3 For example, a 2-D array with values like the following :

int_array_1[0][0] = 0;
int_array_1[0][1] = 1;
int_array_1[0][2] = 2;
int_array_1[0][3] = 3;
int_array_1[0][4] = 4;
int_array_1[1][0] = 5;
int_array_1[1][1] = 6;
int_array_1[1][2] = 7;
int_array_1[1][3] = 8;
int_array_1[1][4] = 9;
int_array_1[2][0] = 10;
int_array_1[2][1] = 11;
int_array_1[2][2] = 12;
int_array_1[2][3] = 13;
int_array_1[2][4] = 14;

will have its data laid out in memory as follows :

0x0000 0x0001 0x0002 0x0003 0x0004 0x0005 0x0006 0x0007 0x0008 0x0009 0x000A 0x000B 0x000C 0x000D 0x000E

This will be no different from the memory layout of a single-dimensional array of the same size :

int_array_2[0] = 0;
int_array_2[1] = 1;
int_array_2[2] = 2;
int_array_2[3] = 3;
int_array_2[4] = 4;
int_array_2[5] = 5;
int_array_2[6] = 6;
int_array_2[7] = 7;
int_array_2[8] = 8;
int_array_2[9] = 9;
int_array_2[10] = 10;
int_array_2[11] = 11;
int_array_2[12] = 12;
int_array_2[13] = 13;
int_array_2[14] = 14;

In C/C++, multidimensional array syntax (i.e. the use of ‘[]’ square brackets for index referencing) merely forms an abstraction for programmers. This abstraction is provided by the compiler for convenience purposes. This, however, often gives the illusion of complexity of storage which is not true. Index-wise, the data is simply laid out in memory in sequence.

2.4 The data of a multi-dimensional C-style array can be expressed as a single dimensional array with indexing coded with some semblance of a formula. The following is an example :

void __stdcall Demonstrate4DCPPArrayOfInt()
{
  // We declare a 2x3x4x5 4D array of integers.
  int int4DArray[2][3][4][5];
  int iDim1 = 2;
  int iDim2 = 3;
  int iDim3 = 4;
  int iDim4 = 5;
  // Assign values into this 4D array.
  for (int i = 0; i < iDim1; i++)
  {
    for (int j = 0; j < iDim2; j++)
    {
      for (int k = 0; k < iDim3; k++)
      {
        for (int l = 0; l < iDim4; l++)
        {
          int4DArray[i][j][k][l] = l + (k * iDim4) + (j * (iDim3 * iDim4)) + (i * (iDim2 * iDim3 * iDim4));
        }
      }
    }
  }
  // Express int4DArray as a 1-dimensional array
  // pointed to by pint4DArray.
  int* pint4DArray = (int*)int4DArray;
  // Access each element of int4DArray via pint4DArray
  // and display it.
  for (int i = 0; i < iDim1; i++)
  {
    for (int j = 0; j < iDim2; j++)
    {
      for (int k = 0; k < iDim3; k++)
      {
        for (int l = 0; l < iDim4; l++)
        {
          int value = pint4DArray[l + (k * iDim4) + (j * (iDim3 * iDim4)) + (i * (iDim2 * iDim3 * iDim4))];
          printf ("%d\r\n", value);
        }
      }
    }
  }
}

The expression highlighted in bold serves as a kind of formula for accessing individual elements of the multi-dimensional array when expressed as a single-dimensional array.

2.5 Note well another important thing : in C/C++, a pointer to a multi-dimensional array is the same as a pointer to a single-dimensional array. Hence, in the above code, although “int4DArray” is a 4-dimensional array of integers, it can be cast into a pointer to an integer and its address assigned to a pointer to an integer :

int* pint4DArray = (int*)int4DArray;

2.6 Note also that a double pointer to an integer is not a pointer to a 2-dimensional array :

int** p;

In fact, it can be interpreted as a pointer to an array of pointers. Such a pointer cannot be used to dereference a multi-dimensional array.

3. How Managed Code Stores Array Data.

3.1 Now that we have learnt how an unmanaged C/C++ array is stored in memory, we turn our attention to how a managed array stores its data in memory.

3.2 As it turns out, a managed array stores its data contiguously and sequentially just like a C-style array. I shall demonstrate this with some C# code below :

static void Demonstrate2DArrayOfInt01InMemory()
{
  // Define a 2D array of ints.
  int[,] TwoDArrayOfInt = new int[3, 5];
  int k = 0;
  // Fill up values. For ease of reference,
  // simply insert sequential values 1 through 15.
  for (int i = 0; i < 3; i++)
  {
    for (int j = 0; j < 5; j++)
    {
      TwoDArrayOfInt[i, j] = k++;
    }
  }
  // Note that the way a managed array
  // arranges its data is very similar
  // to the way a C/C++ program arranges
  // array data : sequentially.
  //
  // To demonstrate this, pin the array data
  // in memory...
  GCHandle gch = GCHandle.Alloc(TwoDArrayOfInt, GCHandleType.Pinned);
  // Get a pointer to the array data...
  IntPtr pArrayData = gch.AddrOfPinnedObject();
  // Via the debugger's memory viewer, confirm
  // that the data is laid in memory in sequence.
  gch.Free();
}

In the above example code, we created a 3×5 2D array of integers. We then assigned values to this array. For ease of reference, the data assigned is a sequence of integers from 0 through 15.

We then used the services of the GCHandle class to help us pin the memory of the “TwoDArrayOfInt” array object. Thereafter, we used the handy GCHandle.AddrOfPinnedObject() method to obtain a pointer to the inner buffer of the array object. From this memory address, we will be able to see that the data is indeed laid out in memory in sequence.

3.3 Furthermore, as we shall see later on, if the array data is blittable, a direct pointer to the array data buffer may be passed to an unmanaged API.

4. The Various Options for Managed Code.

4.1 When it comes to interoping data between managed and unmanged code, the interop marshaler comes into play immediately. To perform its job properly, the interop marshaler relies on MarshalAsAttributes (together with an UnmanagedType enumeration value argument) which are applied to the array type parameters.

4.2 Let’s have a look at the various UnmanagedType enumerations that pertain to arrays :

  • ByValArray
  • LPArray
  • SafeArray

Of these “ByValArray” applies only to an array declared as part of a managed structure. We shall not be discussing this in this series of blogs.

4.3 The “LPArray” enumeration applies to C-style arrays which can prove very useful indeed as we shall see in section 5 where we shall demonstrate how to pass a multi-dimensional array of integers (expressed as a C-style array) from managed code to an unmanaged API.

4.4 The “SafeArray” option should by far be the best option but unfortunately, when used to store multi-dimenional arrays, it can prove challenging. We shall discuss this at length in part 2.

5. Passing a Managed Array as a C-Style Array.

5.1 Listed below is a C++ API that takes a pointer to an int (presumbly to a 2D array of integers) together with 2 dimension specifying parameters :

void __stdcall Set2DArrayOfInt01(int* p2DIntArray, int iDim1, int iDim2)
{
  int iIndex = 0;  // iIndex is used as an index into the int array.
  // For a C/C++ program, a multi-dimensional array is treated the
  // same way as a single-dimensional array. The data for a multi-dimensional
  // array is arranged in sequence.
  for (int i = 0; i < iDim1; i++)
  {
    for (int j = 0; j < iDim2; j++)
    {
      // Print out the contents of the int array.
      // Elements of the array are arranged in
      // sequence. Hence we may refer to a
      // multi-dimensional array just like
      // a single-dimensional array albeit
      // the total numner of elements is the
      // product of the number of dimensions.
      printf ("%d\r\n", p2DIntArray[iIndex++]);
    }
  }
}

The API treats the incoming int pointer as a pointer to a 2-dimensional array of integers. It then accesses each element of the array and displays it.

5.3 This API should be declared in C# as follows :

[DllImport("TestDLL.dll", CallingConvention = CallingConvention.StdCall)]
public static extern void Set2DArrayOfInt01
(
  [In][MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.I4)] int[,] pArrayOfInt,
  int iDim1,
  int iDim2
);

Note the way the first parameter is declared : it is to be marshaled as UnmanagedType.LPArray with the ArraySubType declared as a 4-byte integer.

This indicates that “pArrayOfInt” is to be marshaled to unmanaged code as a C-style array. This is perfect for the Set2DArrayOfInt01() API.

5.4 Listed below is a sample C# code that makes the API call :

static void Call_Set2DArrayOfInt01()
{
  // Define a 2D array of ints.
  int[,] TwoDArrayOfInt = new int[3, 5];
  int k = 0;
  // Fill up values. For ease of reference,
  // simply insert sequential values 1 through 15.
  for (int i = 0; i < 3; i++)
  {
    for (int j = 0; j < 5; j++)
    {
      TwoDArrayOfInt[i, j] = k++;
    }
  }
  // Call the unmanaged C++ function.
  // Note that the way a managed array
  // arranges its data is very similar
  // to the way a C/C++ program arranges
  // array data : sequentially.
  //
  // Also, since the 2D array is marshaled
  // as a C-style array, a pointer to the
  // internal array buffer of the managed
  // array is passed to the unmanaged API.
  GCHandle gch = GCHandle.Alloc(TwoDArrayOfInt, GCHandleType.Pinned);
  IntPtr pArrayData = gch.AddrOfPinnedObject();
  Set2DArrayOfInt01(TwoDArrayOfInt, 3, 5);
  gch.Free();
}

I have added the calls to GCHandle to determine the memory address of the data buffer of TwoDArrayOfInt().

5.5 Now the interesting thing that will happen is that when we perform debugging and step into the C++ code of TwoDArrayOfInt(), we will see that “p2DIntArray” actually equals the value of “pArrayData” (from the C# code).

5.6 This is possible because the type of the data contained in the array is blittable. Hence as an optimization, when the array data is to be marshaled across to unmanaged code as “UnmanagedType.LPArray” (i.e. a C-style array), no marshaling actually takes place. The entire data buffer of the managed array is accessible to unmanaged code.

5.7 The C++ code that loops through “p2DIntArray” will display the correct values as stored in the 2-D array.

6. In Conclusion

6.1 Through the benefit of standardization (of array data layout), we are able to access the inner buffer of a multi-dimensional managed array from unmanaged code.

6.2 Please remember, however, that this is only possible if the contained data type is a blittable type (see point 3.3 above). Non-blittable data require references and marshaling the array data necessarily requires temporary buffer allocation where the data of individual array elements are serialized into flat memory.

6.3 In part 2, we shall be exploring the use of SafeArrays generated from a multi-dimensional managed array.

Advertisements

About Lim Bio Liong

I've been in software development for nearly 20 years specializing in C , COM and C#. It's truly an exicting time we live in, with so much resources at our disposal to gain and share knowledge. I hope my blog will serve a small part in this global knowledge sharing network. For many years now I've been deeply involved with C development work. However since circa 2010, my current work has required me to use more and more on C# with a particular focus on COM interop. I've also written several articles for CodeProject. However, in recent years I've concentrated my time more on helping others in the MSDN forums. Please feel free to leave a comment whenever you have any constructive criticism over any of my blog posts.

Discussion

11 thoughts on “Passing Multi-Dimensional Managed Array To C++ Part 1

  1. This does not work. Debugger shows the array gets corrupted when sending to C function.

    Posted by nug700nug700 | June 23, 2014, 9:25 am
  2. Hello nug700nug700,

    1. The code works on my side.

    2. Can you show me some of your code which did not work ?

    – Bio.

    Posted by Lim Bio Liong | July 3, 2014, 8:41 am
  3. Hi Lim Bio Liong,

    This is a great blog ! However, I am still struggling passing double instead of int from c# to c++.

    is this the right code to use for double (R8 instead of I4 and double instead of int) .. ?

    [DllImport(“TestDLL.dll”, CallingConvention = CallingConvention.StdCall)]
    public static extern void Set2DArrayOfInt01
    (
    [In][MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.R8)] double[,] pArrayOfInt,
    int iDim1,
    int iDim2
    );

    Many thanks.

    Posted by steve | October 20, 2015, 12:00 pm
    • Hello Steve,

      1. Yes, to use the double type instead of the int type, change the ArraySubType to UnmanagedType.R8.

      2. Of course the C++ side would have to change too. The pointer to the array must be changed to a pointer to a double :

      void __stdcall Set2DArrayOfDoubles(double* p2DDoubleArray, int iDim1, int iDim2);

      3. I have tested this scenario and it worked.

      – Bio.

      Posted by Lim Bio Liong | October 21, 2015, 4:23 am
      • Hi Bio,
        You are right, I still had a missing double definition. It’s working now. However, great blog !
        Best ..

        Posted by Steve | October 22, 2015, 5:56 am
      • Bio, maybe you have an idea.
        The following code to pass a 2D string array from C# to C++ works fine up to GCHandle, where it gives an error: An unhandled exception of type ‘System.ArgumentException’ occurred in mscorlib.dll Additional information: The Object does not consist any primitive information.

        [DllImport(“C:\\Users\\Redstone\\Documents\\Visual Studio 2015\\Projects\\Win32Project2\\Debug\\Win32Project2.dll”,EntryPoint = “DDentry”,CallingConvention = CallingConvention.StdCall)]

        public static extern void DDentry
        (
        [In][MarshalAs(UnmanagedType.LPArray, ArraySubType=UnmanagedType.BStr)] string[,] arrayReadDat, int iDim1, int iDim2
        );

        private void button6_Click_1(object sender, EventArgs e)
        {
        int lastRow = excelWorksheet1.UsedRange.Rows.Count;
        int lastCol = excelWorksheet1.UsedRange.Columns.Count;

        string[,] arrayReadDat = new string[lastRow+1, lastCol+1];

        for (int i = 2; i <= lastRow; i++)
        {
        for (int j = 1; j <= lastCol; j++)
        {
        arrayReadDat[i, j] = excelWorksheet1.Cells[i, j].Value2.ToString();
        }
        }

        GCHandle gch = GCHandle.Alloc(arrayReadDat, GCHandleType.Pinned);
        IntPtr pArrayData = gch.AddrOfPinnedObject();

        DDentry(arrayReadDat, lastRow+1, lastCol+1);

        gch.Free();
        }

        Posted by steve | November 4, 2015, 12:13 pm
  4. Dear Bio,,
    Obviously, I learned something from you blog. However, the passing of the 2D string array still do not work .. Any idea or advice .. ? Maybe, something is not correct with the Dllimport or MarshalAs conventions. Many thanks.

    [DllImport(“C:\\Users\\Win32Project2.dll”,
    EntryPoint = “DDentry”,
    CallingConvention = CallingConvention.StdCall)]

    public static extern void DDentry
    (
    [In][MarshalAs(UnmanagedType.LPArray,
    ArraySubType = UnmanagedType.LPStr)] string[,] arrayReadDat, int iDim1, int iDim2
    );

    private void button6_Click_1(object sender, EventArgs e)
    {
    TestStruct arrayReadDat = new TestStruct();
    arrayReadDat.stringArray = new string[lastRow+1, lastCol+1];
    string strK = “testify”;
    for (int i = 2; i <= lastRow; i++)
    {
    for (int j = 1; j <= lastCol; j++)
    {
    arrayReadDat.stringArray[i, j] = strK;
    }
    }

    int size = Marshal.SizeOf(typeof(TestStruct));
    IntPtr strPointer = Marshal.AllocHGlobal(size);
    Marshal.StructureToPtr(arrayReadDat, strPointer, false);

    DDentry(arrayReadDat.stringArray, lastRow+1, lastCol+1);

    Marshal.FreeHGlobal(strPointer);
    }

    Here the C++ code, which did not get the data:

    _declspec(dllexport) void DDentry(string *p2DIntArray, int iDim1, int iDim2)
    {
    int iIndex = 0;
    for (int i = 2; i <= iDim1; i++)
    {
    for (int j = 1; j <= iDim2; j++)
    {
    arrayREAD[i][j] = p2DIntArray[iIndex++];
    }
    }
    }

    Posted by steve | November 5, 2015, 10:49 am
    • Hello Steve,

      1. Sorry for my late reply.

      2. It is not possible to to pin C# Strings or any managed object in memory.

      3. This is so whether they are in an array or not.

      4. This is because managed objects are not blittable. That is, they do not have a memory representation that is commonly shared by corresponding types in unmanaged applications.

      5. Types which a memory representation that is commonly shared by unmanaged applications are known as primitive types in .NET. These include : int, short, long, float, double, char, byte.

      6. These primitive types are called blittable because instances of these types can have their values transferred from managed code to unmanaged (and vice versa) directly without any need for conversion.

      7. An array of blittable types is consequently also blittable. And in the same way, an array of non-blittable types is also non-blittable.

      8. This is why you get the System.ArgumentException on the call to GCHandle.Alloc() with the complaint : “Additional information: Object contains non-primitive or non-blittable data.” when you try to call GCHandle.Alloc() on the array of strings.

      9. To resolve your situation, use SafeArrays which is explained in Part 2 of my article.

      – Bio.

      Posted by Lim Bio Liong | November 7, 2015, 8:41 am
      • Bio,
        Your SafeArray example in part 2 worked perfectly with an 2D integer array !

        I tried to use it with “2D string arrays” and had some issues. Here the C# code ————–:

        [DllImport(“C:\\Users\\Win32Project2.dll”, EntryPoint = “DDentry”, CallingConvention = CallingConvention.Cdecl)]
        public static extern void DDentry
        ([MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_LPWSTR)] string[,] pArrayOfInt);
        public void button6_Click_1(object sender, EventArgs e)
        {
        string [,] TwoDArrayOfInt = new string [3, 5];
        for (int i = 0; i < 3; i++)
        {
        for (int j = 0; j < 5; j++)
        {
        TwoDArrayOfInt[i, j] = String.Format("Cell[{0};{1}]", i, j);
        }
        }
        DDentry(TwoDArrayOfInt);
        }

        Here the C++ code ————-:

        _declspec(dllexport) void DDentry(SAFEARRAY *p2DStringArray)
        {
        long lLbound = 0;
        long lUbound = 0;
        SafeArrayGetLBound(p2DStringArray, 1, &lLbound);
        SafeArrayGetUBound(p2DStringArray, 1, &lUbound);
        long lDim2Size = lUbound – lLbound + 1;
        SafeArrayGetLBound(p2DStringArray, 2, &lLbound);
        SafeArrayGetUBound(p2DStringArray, 2, &lUbound);
        long lDim1Size = lUbound – lLbound + 1;
        for (int i = 0; i < lDim2Size; i++)
        {
        for (int j = 0; j < lDim1Size; j++)
        {
        long rgIndices[2];
        int value;
        rgIndices[0] = i;
        rgIndices[1] = j;
        SafeArrayGetElement
        (
        p2DStringArray, rgIndices, (void FAR*) &value
        );
        }
        }
        }

        the debugger runs perfectly through C# but diodn't step into DDentry (C++). The message were:
        – Could't find or open the PDB file
        – Module was built without symbols.
        – Exception thrown: 'System.Runtime.InteropServices.SafeArrayTypeMismatchException' in WindowsFormsApplication2.exe ..

        Many thanks for your comments .. !

        Posted by steve | January 5, 2016, 2:04 pm
  5. Bio,
    I just followed your example from the blog, but obviously something doesn’t work. There is always a heap error when stepping into the C++ code. In particualr, is the MarshalAs(UnmanagedType.ByValArray, SizeConst = 30) correct for the 2D string array .. ? Maybe, you could help .. thanks.

    Here the C# code:

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public struct TestStruct
    {
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 30)]
    public string[,] m_stringArray;
    }

    [DllImport(“C:\\Win32Project2.dll”,
    EntryPoint = “callCplus”,
    CallingConvention = CallingConvention.StdCall)]
    private static extern void callCplus(IntPtr stdPointer,30,30);

    public void button6_Click_1(object sender, EventArgs e)
    {
    TestStruct test_struct = new TestStruct();
    test_struct.m_stringArray = new string[30,30];

    string strK = “testify”;

    for (int i = 2; i < 30; i++)
    {
    for (int j = 1; j < 30; j++)
    {
    test_struct.m_stringArray[i, j] = strK;
    }
    }

    int size = Marshal.SizeOf(typeof(TestStruct));
    IntPtr pTestStruct = Marshal.AllocHGlobal(size);
    Marshal.StructureToPtr(test_struct, pTestStruct, false);

    callCplus(pTestStruct,30,30);

    Marshal.FreeHGlobal(pTestStruct);

    }

    Here parts of the C++ code:

    extern "C"
    {
    #pragma pack(1)
    struct TestStruct
    {
    string m_stringArray[30][30];
    };

    ..

    _declspec(dllexport) void callCplus(TestStruct *p2DIntArray,30,30)
    {
    for (int i = 2; i < 30; i++)
    {
    for (int j = 1; j m_stringArray[i][j];
    }
    }

    Posted by steve | November 6, 2015, 2:33 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: