ImpLib SDK combines free programming tools for authoring custom import libraries for Windows DLL (32 and 64-bit). The SDK supports advanced features, like cdecl2stdcall conversion, importing by ordinal, removing original thunk, accessing internal undocumented functions:
- Importing symbolic names by ordinal values. For example, there's a DLL function you want to call:
MYDLL!dummy_function ord.20. Its symbolic name isdummy_functionand its ordinal value is 20. Usually, it's not a good idea to import a symbol by ordinal because the ordinal may change in a future version of the DLL. However, if you are sure that the DLL version won't change or the developer of the DLL guarantees backward compatibility of the ordinals (i.e., you are the developer of the DLL and you always keep the ordinals unchanged or you just distribute the same DLL version with your project), then you can benefit from using ordinals. Importing by ordinal significantly reduces the load time of the executable and makes the import section size smaller. By the way, if you plan to always use the same DLL build, consider binding your executables. Binding further speeds up the load time. - Storing symbols from different DLL files in a single import library. For example, you can make a single import library containing the exported entries from KERNEL32, USER32, GDI32 and other common Windows API DLLs and name it something like win32api.lib. Then, you can always link to that single file instead of having to specify kernel32.lib, user32.lib, gdi32.lib and so on.
- When interfacing non-STDCALL DLLs from Visual Basic 6, PureBasic and other popular development environments, sometimes the calling convention might not be supported. ImpLib SDK is able to generate adaptation code to convert CDECL and other calling conventions to STDCALL with very little overhead.
- ImpLib SDK can generate libraries without original thunk. When linking to a "stripped" library without original thunk, the final executable won't contain it either. Therefore, you can make your executable smaller by using "stripped" libraries. To make a "stripped" import library, add the settings
KEEP_ORIGINAL_THUNK equ 0just before the firstimplibmacro. A "stripped" import directory is not bindable, but otherwise perfectly valid for all Windows versions.
The import libraries generated by the ImpLib SDK are compatible with any linker supporting the MS COFF object file format. The following linkers were tested extensively:
- Pelle's Linker (polink) is a free reliable linker created by Pelle Orinius, the developer of Pelle's C. No compatibility issues detected when using 32 or 64-bit, regular and "stripped" import libraries.
- MS Linker, which is part of Visual C++ and the Windows SDK. All versions are supported. The 32-bit version may produce error LNK1102 when loading import libraries larger than a certain size. The ImpLib SDK will issue a compatibility warning when the import library exceeds the maximum size supported by the 32-bit MS linker. Typically, this may happen when the DLL exports more than 1500 symbols. The size of each symbol matters as well. The workaround is to use the 64-bit linker. When using the MS linker, it is not recommended to combine regular and "stripped" import libraries. It is safer to use "stripped"-only or regular-only libraries, but not both. Otherwise, the import tables in the final executable might get corrupted.
- GNU Linker (LD) can be found in the MinGW distributions. Both 32 and 64-bit DLL are supported. This linker can be used to crosscompile to Win32 or Win64 from Linux or other OS. "Stripped" libraries are supported.
Installation and usage
Unpack the ImpLib SDK release into any directory. Then launch build_libs.bat. This batch file builds or rebuilds all the sample import libraries: MSVCRT, Win32 and Win64 system API libraries.
The building process may take several minutes to complete due to the large size of the sample libraries.
The output libraries are stored under the lib subdirectory. Two import libraries are created for each input file: a regular import library including original thunk and a "stripped" version without original thunk. The latter is stored into the stripped subdirectory.
If you add new import library definitions or modify the sample files, just rerun build_libs.bat. Only new or moified files will be recompiled.
How does it work?
First, it's necessary to list the symbols (i.e. functions) exported by the given DLL. The list is saved in plain text format. The syntax is very simple and similar to Microsoft's DEF file format.
For example, let's make an import lib with some file I/O functions exported by KERNEL32.DLL:
include 'implib.inc' ; KERNEL32.DLL implib kernel32.dll, CreateFileA implib kernel32.dll, CreateFileW implib kernel32.dll, ReadFile implib kernel32.dll, SetFilePointer implib kernel32.dll, CloseHandle endlib
Everything preceded by a ';' is a comment. Before starting to list the symbols, it's necessary to add a valid reference to either implib.inc (32-bit DLL) or implib64.inc (64-bit DLL).
A relative or full path is required unless you place implib.inc or implib64.inc in the same directory where the current script file is located.
After completing the list, don't forget to add the endlib macro. Every symbol definition is created with the implib macro, followed by the DLL file name and the symbolic name of the function.
You may specify the ordinal value instead of the symbolic name, prefixing it with 'ord.':
implib kernel32.dll, ord.5
If you want a symbol to be indexed by the linker with a different name, append another argument with the alias name:
implib kernel32.dll, CreateFileA, CreateFile
When referring to CreateFile in your object file, the linker will actually resolve it as KERNEL32!CreateFileA.
Thus, by default, the 'A' suffix is not required in the object file. This feature is useful mostly for mangling symbolic names.
For example, MS tools assume that a 32-bit STDCALL function CreateFileA with 7 arguments should be named as _CreateFileA@28.
So, if we are linking to the 32-bit KERNEL32.DLL, let's rewrite our first example in a way compatible with the MS naming conventions:
include 'src\implib.inc' ; KERNEL32.DLL implib kernel32.dll, CreateFileA, _CreateFileA@28 implib kernel32.dll, CreateFileW, _CreateFileW@28 implib kernel32.dll, ReadFile, _ReadFile@20 implib kernel32.dll, SetFilePointer, _SetFilePointer@16 implib kernel32.dll, CloseHandle, _CloseHandle@4 endlib
As you see, 32-bit STDCALL names are prefixed with an underscore (_) and suffixed with the at sign and a decimal number (@#). The decimal number is the size in bytes of the argument list. Usually, it equals to the number of arguments x4 because most arguments on the stack are either 32-bit or aligned to a 32-bit boundary (32 bits = 4 bytes).
This naming convention is not used in 64-bit mode. Therefore, if linking to the 64-bit KERNEL32.DLL, the list would look like follows:
include 'src\implib64.inc'
; KERNEL32.DLL
implib kernel32.dll, CreateFileA
implib kernel32.dll, CreateFileW
implib kernel32.dll, ReadFile
implib kernel32.dll, SetFilePointer
implib kernel32.dll, CloseHandle
endlib
As you may notice, the DLL is named KERNEL32, even though the target platform is Win64. Microsoft is intentionally keeping the same DLL names.
Save the def-file (either the 32 or 64-bit version). Let's call it kernel32.def, but you can choose any name you like. Now we are ready to compile it from the command line:
bin\fasm kernel32.def kernel32.lib flat assembler version 1.67.23 (533081 kilobytes memory) DLL symbols indexed: 5 3 passes, 4313 bytes. |
We've got a file named kernel32.lib! By the way, FASM is an assembler, but its preprocessor is so powerful that we can use it to run the ImpLib scripting engine.
There are more arguments to the implib macro. Just open implib.inc or implib64.inc in a plain text editor and read the header for additional information.
DLL2DEF
It's a command-line utility to extract the dynamic-link library symbols in plain text format. DLL2DEF supports both 32 and 64-bit DLL.
Since big DLLs might contain tons of useful functions, listing them manually in a text file would be very time-consuming. This tool, available in the \bin subdirectory, is useful to speed up the process. Running dll2def without arguments produces the following output:
USAGE: dll2def [options] file [output]
file - input file name.
output - optional output file name.
options:
/COMPACT - Don't place any comments.
/PB - Enable PureBasic mod.
/VB - Enable Visual Basic 6 mod.
Example:
dll2def \windows\system32\kernel32.dll test.def |
Let's give it a try:
dll2def c:\windows\system32\kernel32.dll
We've got a file named kernel32.def listing all the KERNEL32.DLL functions.
The reference to implib.inc (32-bit DLL) or implib64.inc (64-bit DLL) contains the full path name if the inc-file is located in the same directory as dll2def.exe, so that you can move the def-file to any location without having to update the inc-file path.
Every symbol definition starts with some comments, for example:
; KERNEL32.AddVectoredExceptionHandler ord.9 ; -> NTDLL.RtlAddVectoredExceptionHandler implib kernel32.dll, AddVectoredExceptionHandler
The comments include the full symbolic name (if available), the ordinal value and the forwarder chain, if any. Knowing the ordinal is useful if you have good reasons to replace the symbol definition with its ordinal. The forwarder chain means that the given symbol is actually located in another DLL. In the example above the symbol AddVectoredExceptionHandler exported by KERNEL32.DLL is actually forwarded to NTDLL. You can update the symbol definition to import the given symbol directly from the target DLL, but generally doing so is not recommended for compatibility reasons.
If you don't like to see those comments in the script, add /COMPACT to the command line before the input filename:
bin\dll2def /COMPACT c:\windows\system32\kernel32.dll
Mods
Currently ImpLib contains 2 mods:
- Visual Basic mod useful particularly for interfacing CDECL and other non-STDCALL DLLs in Visual Basic 6.
- PureBasic mod useful for making PureBasic Userlibs linking to external DLLs, especially CDECL DLLs.
Replace the implib macro with vbimplib or pbimplib in the def-file to use the Visual Basic or PureBasic mods respectively. While calling DLL2DEF prepend a /VB or /PB argument to make it use the given mod scheme instead of the general purpose implib. Check the tutorials below for a quick start.
Pick one of the following minitutorials for a real usage example:
- General purpose: MSVCRT.lib for MASM32 and NASM.
- PureBasic mod: OpenAL PureBasic Userlib.
MSVCRT.lib for MASM32 and NASM
Microsoft Visual C Run-Time library (MSVCRT.DLL) exports over 1300 functions, automating many common programming tasks, like formatted console I/O, memory management, string manipulation, sorting and so on.
There is nothing you can't achieve by using the Windows API directly, but since the C Run-Time is almost always preinstalled in all Windows versions (except in Windows 95), sometimes it makes sense linking to CRT DLLs instead of hardcoding printf(), malloc(), etc. So, the goal of our first tutorial is making a couple of sample programs in MASM32 and NASM using MSVCRT.DLL.
Let's start with MASM32. You will probably find a header file for MSVCRT, named msvcrt.inc, in the latest MASM32 release, but there is no msvcrt.lib file required by the linker. If you have Visual Studio installed, you can get the dynamic version of msvcrt.lib from there, but there is one little naming conflict with functions div() and fabs(). You can't use div or fabs as the name of a function in either MASM32 or NASM because there are homonymous mnemonics in the x86 instruction set. The same conflict arises when using names, containing only numeric characters, or any of the words reserved by the compiler, like proc, label, etc. Import libraries in Visual Studio sometimes introduce additional dependencies from OLDNAMES.lib, UUID.lib and others. It seems, you don't have permission to redistribute a Visual Studio's import library and that's why MASM32 doesn't include msvcrt.lib.
So, we'll make our own MSVCRT.lib not copyrighted by any third party, not conflicting with MASM's or NASM's reserved words.
The first step is making the def-file, containing all the public names found in MSVCRT.DLL:
dll2def /COMPACT \windows\system32\msvcrt.dll
Next, we need to solve the naming conflict. So, open msvcrt.def in Notepad and search for the following line:
implib msvcrt.dll, div
Since div is the unsigned integer division mnemonic, we need to rename this function to something else.
Steve Hutchesson (Hutch), the creator of the MASM32 SDK, suggested prefixing it with '_crt_' (there's a thread on this topic at masm32's official forum).
So, here's the replacement:
implib msvcrt.dll, div, _crt_div
Do the same to fabs function. It is recommended adding a leading '_' preceding the thunk name to all function names not already starting with it, so that calling these functions would be possible via invoke thunk from MASM32. For example:
; MSVCRT.printf ord.742 implib msvcrt.dll, printf
It is recommended to update it in the following way:
; MSVCRT.printf ord.742
implib msvcrt.dll, printf, _printf
The same definition using the ordinal value would be:
; MSVCRT.printf ord.742
implib msvcrt.dll, ord.742, _printf, __imp__printf
Since the real function name is lost when using its ordinal, it is recommended to specify the thunk name (_printf) and public import name (__imp__printf) explicitly this time. ImpLib won't complain if you don't specify them, but calling printf() will become more tricky.
Proceed compiling the library. Since ImpLib runs interpreted by the FASM's preprocessor and the def-file contains a huge number of functions, it might take some time to complete the task:
\bin\fasm msvcrt.def msvcrt.lib flat assembler version 1.67.23 (518320 kilobytes memory) DLL symbols indexed: 1372 3 passes, 21.4 seconds, 845227 bytes.
Now we have our own msvcrt.lib. Or a list of syntax errors, duplicate symbols and other sad messages. Just correct the typos and compile again.
A ready to use msvcrt.def is available under \src\MSVCRT32.
In order to call CRT functions in MASM32 using invoke, you need to specify the prototypes.
There's an example of a header file with CRT prototypes in \src\MSVCRT32\test_masm\msvcrt.inc.
As an alternative, you can use PROTO instead of externdef syntax. For example:
printf PROTO C :DWORD,:VARARG
Calling printf() using invoke with the above prototype will generate a call <jmp msvcrt!printf> instead of a direct call msvcrt!printf. Direct calls are usually faster. It is recommended to prototype the extern functions in the externdef manner:
cdecl_dword_vararg typedef PROTO C :DWORD,:VARARG externdef _imp__printf:PTR cdecl_dword_vararg printf equ <_imp__printf>
This time invoking printf() will generate a direct call.
Here's a sample MASM32 code performing a call to MSVCRT!printf:
.386
.model flat,stdcall
; MSVCRT API
include msvcrt.inc
includelib msvcrt.lib
.CODE
format db "Hello, world!",0
start:
invoke printf,OFFSET format
ret
END start
Let's compile, link and run it from the command line:
\masm32\bin\ml /c /coff test.asm \masm32\bin\link /SUBSYSTEM:CONSOLE test.obj test Hello, world!
The same example for NASM:
EXTERN __imp__printf
%define printf [__imp__printf]
section .text
format db "Hello, world!",0
GLOBAL _start
_start:
push format
call printf
add esp,4 ; fix the stack
ret
Compile, link and run:
\nasm\nasm -fwin32 test.asm \nasm\polink /ENTRY:start /SUBSYSTEM:CONSOLE test.obj msvcrt.lib test Hello, world!
MSVCRT.LIB tutorial ends here.
OpenAL PureBasic Userlib
Our intention is building a PureBasic Userlib interfacing an external DLL. The example DLL is OPENAL32.DLL from the OpenAL v1.0/v1.1 redistributable. This DLL uses CDECL calling convention for all API functions.
We could use the ImportC syntax and the OpenAL32.lib file found in the OpenAL SDK, or just use OpenLibrary and then CallCFunctionFast every time we need to invoke an OpenAL function. The alternative approach in this tutorial is making a PureBasic Userlib, providing direct access to OpenAL, with a very small overhead.
The caller thunk will be faster compared to CallCFunctionFast, with prototype checking, inline help and any other advantages a Userlib is able to offer.
First of all, we need to create the import description script:
bin/dll2def /PB /COMPACT \windows\system32\OpenAL32.dll
This will create a file named openal32.def, listing all the public symbols found in OpenAL32.dll. For example:
pbimplib OpenAL32.dll, STDCALL, 0, alBuffer3f
The complete pbimplib macro syntax is as follows:
pbimplib dllname, convention, numarguments, real_name, PureBasics_name, public_name
The dllname parameter identifies the DLL's filename. The convention parameter identifies the calling convention for the specified function. The following values are currently supported:
- STDCALL used by default in most Windows DLLs: KERNEL32.DLL, USER32.DLL, ...
- CDECL used mostly in C DLLs: MSVCRT.DLL, OPENAL32.DLL, ...
- PBCALL is PureBasic's calling convention. It's provided just in case and to make an example of extending the PBIMPLIB functionality. Somebody might consider adding more exotic calling conventions, like PASCAL, FASTCALL, etc.
numarguments tells the amount of doubleword (32-bit) values used in the stack frame for argument storage. It generally matches the amount of arguments, except when dealing with 64-bit parameters (double, __int64 and so on). This value is not used in STDCALL.
real_name is the name actually defined in the DLL's export table. The last 2 parameters are optional.
PureBasics_name allows customizing the name of the function in PureBasic. By default, when not specified, real_name is used.
public_name is reserved for very specific issues, generally involving low-level assembly programming. It lets you specify the name of the symbol, used to call a function directly.
Take a look at the generated script file once again:
pbimplib OpenAL32.dll, STDCALL, 0, alBuffer3f
alBuffer3f() is not a STDCALL function and it doesn't receive 0 arguments. DLL2DEF just can't guess the number of arguments and the calling convention. Using "STDCALL, 0" just disables automatic stack fixing after calling the given function via thunk. The correct prototype should be:
pbimplib OpenAL32.dll, CDECL, 5, alBuffer3f
You can update all the other definitions or just use the already fixed script src\PureBasic\openal32.def. Now, let's compile it:
bin\fasm openal32.def Pbopenal.lib
No errors? - Fine. Now you need to create a library descriptor and name it exactly as the newly obtained import lib with a ".desc" extension. So, in the current example this file should be named Pbopenal.desc. Check the [PureBasic]\SDK\Readme.txt file for more information about the LibraryMaker tool.
; Langage used to code the library: ASM or C. You need to select C here. C ; Number of Windows DLL the library needs: none in this example. 0 ; Library type (Can be OBJ or LIB). Select LIB because Pbopenal.lib is a LIB. LIB ; Number of PureBasic libraries needed by this library: none in this example. 0 ; Directory name where the online help is available for the API. Help
The rest of the file depends on the actual library contents. There is an example: src\PureBasic\Pbopenal.desc. Finally, use LibraryMaker.exe to create the Userlib:
LibraryMaker .\PBopenal.desc
Note: When running LibraryMaker, make sure the path to the executable doesn't contain whitespaces.
If the compilation was successful, just move the Userlib into [PureBasic]\PureLibraries\UserLibraries and try compiling a simple test application. You can find multiple PureBasic OpenAL examples in the PureBasic OpenAL SDK and the uFMOD (or µFMOD) package.
Thanks for reading! I hope you found this guide useful.