Calling a windows DLL from linux

[Armitage]

Does anybody know if this is possible and how to go about it?

There is a library I need to use that is only available on windows - just a number cruncher, shouldn't be any graphics/gui/system stuff in it. Pass some data in, get some data out.

Basically, I have C++ code, built on Linux with the Gnu toolchain. I want to be able to call some functions from a Windows DLL

 

[cleverhandle]

Windows and Linux use completely different executable formats (PE vs. ELF), so there's no way you're going to call a DLL directly from Linux. You might be able to do something simple using WINE to run a Windows .exe under Linux, but you can't link in Windows DLL code to a Linux program.

 

[xtknight]

$ wine rundll32.exe <library> call ?

Other than that I have no idea. Make a Windows program to wrap the DLL and call that also from wine I guess.

 

[Armitage]

I've read some interesting bits suggesting it may be possible via winelib. But nothing like a step-by-step or demo. Considering that I'm not a windows developer at all (so have never used a dll) and have never used wine, it looks like a rough road.

 

[xtknight]

You're sure you can't find a native equivalent of this library? It's something specialized, proprietary, and closed source I assume?

 

[Armitage]

Originally posted by: xtknight
You're sure you can't find a native equivalent of this library? It's something specialized, proprietary, and closed source I assume?

Yep - very specialized piece of numerical analysis produced by the government and tightly controlled. I know for certain that there isn't a linux version, but porting it should be relatively trivial - though V&V of the result will be a bear. We're trying to get access to the source code, but it's unlikely for a variety of reasons.

 

[xtknight]

Well I believe the ntfs-3g (NTFS on Linux) project calls a Windows version of ntfs.sys (obviously closed-source) using some kind of Windows emulation. Maybe you can find out what they do. Supposedly it's very stable.

Edit: that is a sys file, I'm not sure how close those are to DLLs.

 

[kamper]

Originally posted by: xtknight
Well I believe the ntfs-3g (NTFS on Linux) project calls a Windows version of ntfs.sys (obviously closed-source) using some kind of Windows emulation. Maybe you can find out what they do. Supposedly it's very stable.

That's in kernel space though right?

 

[xtknight]

Originally posted by: kamper

Originally posted by: xtknight
Well I believe the ntfs-3g (NTFS on Linux) project calls a Windows version of ntfs.sys (obviously closed-source) using some kind of Windows emulation. Maybe you can find out what they do. Supposedly it's very stable.

That's in kernel space though right?

It uses FUSE (filesystem in userspace) but it's a kernel driver so I'm not sure. What would that have to do with calling DLLs though?

 

[drag]

Your confused.

First off NTFS-3d uses ntfsprogs, which is completely open source. It is FUSE though.
Captive-NTFS uses the wine + the ntfs sys _driver_ file to emulate Windows.. And it sucks. NTFS-3g kick's it's ass.

AS for the DLL file and what to do with it, I don't know.

I can't program my way out of a paper sack and I have even less knowledge about programming with DLLs and Wine and all that. I had a idea though...

It depends on how networking works in Wine, which again I know next to nothing about. But You could do something like write a simple program to take input from a network socket and proccess it using the DLL's capabilities. Figure out some simple protocol like it retreives the configuration information for the job via XML on one socket, then the program waits and then proccesses the raw data it receives and sends it back out on another socket.

Something simple stupid like that.

Or use RPC or something like that. Don't know enough about programming, but I hope you get what I am thinking of.

Just setup a simple program to run in Wine using that DLL that listens for input then outputs the proccessed information. Then have your real program in Linux just send and receive the data as it needs to.

 

[drag]

For using DLLs in Linux maybe take a look at Mplayer.

They've developed wrappers to running multimedia DLLs from Windows in Linux. It's kinda close to what you want.. the DLLs deal with a lot of information and intensive CPU loads and such. Maybe there is some documentation on how they accomplished that. Also other projects have followed such as gstreamer.

For codecs you usually just copy them to something like /usr/lib/win32 or something like that (depends on configuration).

 

[kamper]

Originally posted by: xtknight
It uses FUSE (filesystem in userspace) but it's a kernel driver so I'm not sure. What would that have to do with calling DLLs though?

Well, I just figured that linking in a kernel module would be completely different than linking in a userland library.

 

[kamper]

Originally posted by: Armitage

Originally posted by: xtknight
You're sure you can't find a native equivalent of this library? It's something specialized, proprietary, and closed source I assume?

Yep - very specialized piece of numerical analysis produced by the government and tightly controlled. I know for certain that there isn't a linux version, but porting it should be relatively trivial - though V&V of the result will be a bear. We're trying to get access to the source code, but it's unlikely for a variety of reasons.

Just curious, if it'd be a pain to validate the ported code, are you really that confident that linking it to a linux binary and passing data structures back and forth would be any more reliable? What happens if you want to run your application on another architecture (or other unix even)?

Following drag's idea of writing a windows process to wrap the dll and then talk to it with sockets, perhaps the safest thing would be to set up a windows box with said process to make sure it's running in the proper environment. Even if it's vmware on the same machine.

 

[Armitage]

Originally posted by: drag
Your confused.

First off NTFS-3d uses ntfsprogs, which is completely open source. It is FUSE though.
Captive-NTFS uses the wine + the ntfs sys _driver_ file to emulate Windows.. And it sucks. NTFS-3g kick's it's ass.

AS for the DLL file and what to do with it, I don't know.

I can't program my way out of a paper sack and I have even less knowledge about programming with DLLs and Wine and all that. I had a idea though...

It depends on how networking works in Wine, which again I know next to nothing about. But You could do something like write a simple program to take input from a network socket and proccess it using the DLL's capabilities. Figure out some simple protocol like it retreives the configuration information for the job via XML on one socket, then the program waits and then proccesses the raw data it receives and sends it back out on another socket.

Something simple stupid like that.

Or use RPC or something like that. Don't know enough about programming, but I hope you get what I am thinking of.

Just setup a simple program to run in Wine using that DLL that listens for input then outputs the proccessed information. Then have your real program in Linux just send and receive the data as it needs to.

Yea, I've considered things like that. Wouldn't be to hard to do. The problem is that the bit we need this library for is very cpu intensive. Overall the app runs on a small beowulf cluster using a lower fidelity version of the library. The one we need will probably be about an order of magnitude slower. Serving out every of of those calls to an outside process will kill it.

Interesting idea on mplayer ... I'll take a look at it.

 

[Armitage]

Originally posted by: kamper

Originally posted by: Armitage

Originally posted by: xtknight
You're sure you can't find a native equivalent of this library? It's something specialized, proprietary, and closed source I assume?

Yep - very specialized piece of numerical analysis produced by the government and tightly controlled. I know for certain that there isn't a linux version, but porting it should be relatively trivial - though V&V of the result will be a bear. We're trying to get access to the source code, but it's unlikely for a variety of reasons.

Just curious, if it'd be a pain to validate the ported code, are you really that confident that linking it to a linux binary and passing data structures back and forth would be any more reliable?

Yea, it's going to be a PITA to validate in any case.

What happens if you want to run your application on another architecture (or other unix even)?

Following drag's idea of writing a windows process to wrap the dll and then talk to it with sockets, perhaps the safest thing would be to set up a windows box with said process to make sure it's running in the proper environment. Even if it's vmware on the same machine.

The stuff we need from this library will be called 100s of millions of times in a typical run - often inside tight loops. Passing it out to another process will kill it dead.

 

[DaveSimmons]

Sounds like the best solution is to run the DLL inside a Windows app.

Is there something linux-specific in the rest of your app that would keep you from implementing it in Windows instead of linux?

If there isn't much user interface it's very easy to build a MFC dialog-based app in Visual Studio / VC++, with visual editing of the UI and wizard support to hook the buttons up to message handler functions.

 

[itachi]

you can write a pe to elf converter or use a kernel pe loader.
the first one would be a pain in the ... to write and would modify the contents of the binary.
the latter would require each node in the cluster have the loader and may require the kernel be rebuilt, which probably won't sit well with the admin and users outside your group. you can take a look at one implemented for netbsd here http://sourceforge.net/projects/peace.
both cases you'll probably have to load the functions manually (using dlopen).

winelib won't work since it requires you have access to the source code.

why not compile it as a windows app and run it using wine?

 

[EagleKeeper]

Another option could be to setup a wrapper in Linux to push the input data out TCP/IP as a packet.

Have a wrapper on the Windows side to receive the packet, run the DLL and return via another packet the result.

Have the Linux transmitter wait for the Windows return poacket, extract the data and continue.


App(Linux) -> TCP/IP -> Windows -> DLL -> Windows -> TCP/IP -> App(Linux)

 

[Armitage]

Originally posted by: EagleKeeper
Another option could be to setup a wrapper in Linux to push the input data out TCP/IP as a packet.

Have a wrapper on the Windows side to receive the packet, run the DLL and return via another packet the result.

Have the Linux transmitter wait for the Windows return poacket, extract the data and continue.


App(Linux) -> TCP/IP -> Windows -> DLL -> Windows -> TCP/IP -> App(Linux)

I may try this with our lower fidelity (and native) library via PVM. But as I said above, I think the overhead of this technique is going to kill it. Not to mention the added complexity.

 

[Armitage]

Originally posted by: DaveSimmons
Sounds like the best solution is to run the DLL inside a Windows app.

Is there something linux-specific in the rest of your app that would keep you from implementing it in Windows instead of linux?

Primarily the hardware. It runs on a beowulf cluster and while there has been some work on computational clustering with windows, it's not very mature in my opinion. That and there's no money to build a 2nd cluster and converting the existing cluster to MS would mean porting this code and supporting scripts, databases, etc. + significantly disrupting other code and users.

If there isn't much user interface it's very easy to build a MFC dialog-based app in Visual Studio / VC++, with visual editing of the UI and wizard support to hook the buttons up to message handler functions.

There's no UI - it's purely a command-line number cruncher.