Problem calling an external C++ dll from C#

[Train]

Asking here because I am getting no responses on several C# boards, even the MSDN forum is failing to answer this one.

I have a C# application that uses a legacy C++ dll that runs an operation on a set of Images. During testing some of the output images created by the dll had strange artifacts in them, weird cropping and resizing going on that it wasn't supposed to do.

After some pretty intensive testing it appears that I can get any image to render just fine after my C# app starts, but if I change the source image, I start to get errors. So I restart the app, and this time begin with the image that gave me errors... several renders, but this time no problem, if I switch to the image that was ok the first run, now that one gives me problems. It appears the external call only causes these weird issues after the input image parameters have changed.

My call to the external dll is within a method, and all of the objects that get passed to this DLL are created WITHIN that method (local scope) (and even explicitly destroyed before exiting the method), so new ones are created for each call. Below is the method call, as well as the DllImport declaration.

Is there a way to ensure each call is "fresh" just like the first one after the app starts? It seems the only way to ensure the external call goes OK is to restart my C# app before doing another render with new parameters.

public void render()
{
string CFX = "proprietary render script goes here";
byte[] LocalCacheDir = Encoding.Default.GetBytes(Config.Paths.ImageCacheDir);
byte[] url_spec = Encoding.Default.GetBytes(CFX);
int usr_num = -1;
StringBuilder type = new StringBuilder(string.Empty);
StringBuilder image = new StringBuilder(string.Empty);
byte[] outputFile = Encoding.Default.GetBytes(Config.Paths.RenderedImage.FullPath);
bool result = GenerateImageExt(LocalCacheDir, url_spec, usr_num, type, image, outputFile);
}

[DllImport("cfxengineext.dll", SetLastError = true, CallingConvention = CallingConvention.StdCall)]
static extern bool GenerateImageExt(
byte[] LocalCacheDir,
byte[] url_spec,
int user_num,
[MarshalAs(UnmanagedType.LPStr)] StringBuilder type,
[MarshalAs(UnmanagedType.LPStr)] StringBuilder image,
byte[] outputFile
);

 

[Crusty]

The only thing I can think of is to change your CallingConvention to Cdecl, if that doesn't work it sounds like a bug in the DLL file.

 

[Train]

Originally posted by: Crusty
The only thing I can think of is to change your CallingConvention to Cdecl, if that doesn't work it sounds like a bug in the DLL file.

It actually was Cdecl, I changed it to StdCall and repeated all the tests with the exact same results.

Unfortunately fixing a bug on the DLL is out of the question.

 

[nickbits]

The easiest thing to do would be to make a helper app that just calls that external function, then you would call that program from that function instead. This ensures a fresh copy is loaded each time via spawning of a new process. If you are calling that function a lot it might not be ideal however.

Alternatively, you could look at doing your own interop via LoadLibrary, GetProcAddress, FreeLibrary and the Marshal class.

 

[Markbnj]

Originally posted by: Train

Originally posted by: Crusty
The only thing I can think of is to change your CallingConvention to Cdecl, if that doesn't work it sounds like a bug in the DLL file.

It actually was Cdecl, I changed it to StdCall and repeated all the tests with the exact same results.

Unfortunately fixing a bug on the DLL is out of the question.

If it's a 3rd party lib and you can't tear it open, then about all that's left is to change what you're doing to the DLL and see how the behavior changes. It sounds a little like memory corruption to me, perhaps on the stack given the apparent parameter mangling you're seeing.

I guess I would start by taking out all but the simplest operation on the image and then adding stuff back in until the problem begins to occur.

But then you've probably done most of this crap already.

 

[Train]

Originally posted by: Markbnj

Originally posted by: Train

Originally posted by: Crusty
The only thing I can think of is to change your CallingConvention to Cdecl, if that doesn't work it sounds like a bug in the DLL file.

It actually was Cdecl, I changed it to StdCall and repeated all the tests with the exact same results.

Unfortunately fixing a bug on the DLL is out of the question.

If it's a 3rd party lib and you can't tear it open, then about all that's left is to change what you're doing to the DLL and see how the behavior changes. It sounds a little like memory corruption to me, perhaps on the stack given the apparent parameter mangling you're seeing.

I guess I would start by taking out all but the simplest operation on the image and then adding stuff back in until the problem begins to occur.

But then you've probably done most of this crap already.

Ya I've gotten it even more specific since I posted this thread. Ran about 200 test cases, recording every one in a spreadsheet with every detail recorded. The exact problem is that when you make another call, the size of the image changes. Somehow whatever the dimensions of the first call you made, the dll hangs on to those dimensions, because with every subsequent image, if its bigger, it will shrink it to the exact size of the first image, but then still output to the size you wanted, with that extra space being either black or some kind of "bleeding" effect. If its larger, it will make a weird cropping. So the bug in the dll is that it is persisting the first dimensions it gets for some reason for the rendering part, but then will use the passed in dimensions on the output, kinda screwy but thats what it does.

I think i'm going to take nickbits advice and make a background exe, just a quick call to the dll, then exit with a return code. Which is way better than the other solution I was thinking of, which was to paste all images into a "greatest common denominator" size image then crop the result back to the correct size. While it might work, probably a lot more coding involved and probably still more overhead than calling another exe.

 

[degibson]

Originally posted by: Train

Originally posted by: Markbnj

Originally posted by: Train

Originally posted by: Crusty
The only thing I can think of is to change your CallingConvention to Cdecl, if that doesn't work it sounds like a bug in the DLL file.

It actually was Cdecl, I changed it to StdCall and repeated all the tests with the exact same results.

Unfortunately fixing a bug on the DLL is out of the question.

If it's a 3rd party lib and you can't tear it open, then about all that's left is to change what you're doing to the DLL and see how the behavior changes. It sounds a little like memory corruption to me, perhaps on the stack given the apparent parameter mangling you're seeing.

I guess I would start by taking out all but the simplest operation on the image and then adding stuff back in until the problem begins to occur.

But then you've probably done most of this crap already.

Ya I've gotten it even more specific since I posted this thread. Ran about 200 test cases, recording every one in a spreadsheet with every detail recorded. The exact problem is that when you make another call, the size of the image changes. Somehow whatever the dimensions of the first call you made, the dll hangs on to those dimensions, because with every subsequent image, if its bigger, it will shrink it to the exact size of the first image, but then still output to the size you wanted, with that extra space being either black or some kind of "bleeding" effect. If its larger, it will make a weird cropping. So the bug in the dll is that it is persisting the first dimensions it gets for some reason for the rendering part, but then will use the passed in dimensions on the output, kinda screwy but thats what it does.

I think i'm going to take nickbits advice and make a background exe, just a quick call to the dll, then exit with a return code. Which is way better than the other solution I was thinking of, which was to paste all images into a "greatest common denominator" size image then crop the result back to the correct size. While it might work, probably a lot more coding involved and probably still more overhead than calling another exe.

I think the second process approach is a good one. It not only provides the functionality you want, but it also provides fault isolation and encapsulation. Its a technique I've used successfully in situations very similar to this. Buggy third-party close-source code is lame.