Sunday, September 24, 2006

Hack#12: Create smaller .EXE files ($SetPEFlags)

No, this post is not about so-called EXE-compressors - I don't believe in using them. And it is not a pure hack in the sense that we're breaking any rules - its just about documenting an undocumented and unknown feature of the Delphi 2006 Win32 compiler (it is not implemented in D7 - I don't know about D2005 yet as I don't have it installed on this laptop anymore).

Background info
When you compile a DLL (or a package which is a Delphi-specific DLL in disguise), the linker includes what is known as a relocation table. This table includes information about what addresses must be fixed up by the OS loader in the (likely) event that the DLL must be loaded at a different address than its intended-at-compile/link-time base address. You see all DLLs come with a base address that is the "ideal" loading address of that module. The OS will try to load the DLL at this address to avoid the overhead of runtime rebasing (patching the in-memory pages of the DLL forces it to be paged in from disk and prevents cross-process sharing of the DLL pages). That's why you should set the Image base option in the Linker page of the project options of DLL and package projects. The default Image base that Delphi uses is $00400000 for both applications, DLLs and packages - and thus by default all DLLs and packages will need to be rebased - as that address is reserved for the process' EXE file.

The implication is that an EXE file will always be loaded at the fixed virtual address $00400000 and that it will never need to be rebased. Alas, it doesn't really need its relocation table and we can safely remove it, shrinking the size of the .EXE without affecting its behavior or performance. It is basically a free lunch!

Up until now Delphi users have had to resort to external stripping tools like Jordan Russell's excellent StripReloc tool. It basically takes the name of an .EXE file as a parameter and removes the relocation table from it. I normally don't bother to strip the reloc table during development, but I do run Jordan's tool before deploying our application. This strips a hefty 350 kB from our 7 MB EXE file - a nice, free 5% reduction right there.

As far as I know, most Microsoft tools produces reloc-free EXE files, while Delphi always includes it for both DLLs and EXEs. Up until now. Delphi 2006 actually has a unknown, undocumented compiler directive feature that allows you to turn off the relocation table for EXE files.

To test it out I first opened the ResXplor project from the Demos\DelphiWin32\VCLWin32\ResXplor folder (a pretty neat application, btw). I compiled it as-is. Project | Information told me that the size of the EXE file was 614400 bytes (and Windows Explorer agreed). Then I added the following line to the top of the project file:

{$SetPEFlags 1}

(We'll look at the source of the mysterious value 1 in a moment).


Then I recompiled the project (there is no need to rebuild it, as the unit files does not need to be recompiled - only the linker must be reinvoked). And the size of the EXE file had shrunk to 577536 bytes - that is a reduction of 36864 bytes or 6 %. Not bad for a one-liner, eh? ;).


The Delphi 2006 help file has the following to say about the $SetPEFlags compiler directive:



"Type: Flag
Syntax: {$SetPEFlags <integer expression>} {$SetPEOptFlags <integer expression>}
Scope: Local


Microsoft relies on PE (portable executable) header flags to allow an application to indicate compatiblity with OS services or request advanced OS services. These directives provide powerful options for tuning your applications on high-end NT systems.


These directives allow you to set flag bits in the PE file header Characteristics field and PE file optional header DLLCharacteristics field, respectively. Most of the Characteristics flags, set using $SetPEFlags, are specifically for object files and libraries. DLLCharacteristics, set using $SetPEOptFlags, are flags that describe when to call a DLL's entry point.


The <integer expression> in these directives can include Delphi constant identifiers, such as the IMAGE_FILE_xxxx constants defined in Windows.pas. Multiple constants should be OR'd together."


So it (and the related SetOptPEFlags directive) is a way of informing the OS about the certain aspects of the application. For instance, it can be used to tell the OS that it is ready and compatible with getting access to virtual memory addresses above the 2 GB boundary. Lets look at the IMAGE_FILE constants defined in the Windows.pas unit.

const
{ Relocation info stripped from file. }
IMAGE_FILE_RELOCS_STRIPPED = $0001;
{ File is executable (i.e. no unresolved externel references)}
IMAGE_FILE_EXECUTABLE_IMAGE = $0002;
{ Line nunbers stripped from file. }
IMAGE_FILE_LINE_NUMS_STRIPPED = $0004;
{ Local symbols stripped from file. }
IMAGE_FILE_LOCAL_SYMS_STRIPPED = $0008;
{ Agressively trim working set }
IMAGE_FILE_AGGRESIVE_WS_TRIM = $0010;
{ App can handle >2gb addresses }
IMAGE_FILE_LARGE_ADDRESS_AWARE = $0020;
{ Bytes of machine word are reversed. }
IMAGE_FILE_BYTES_REVERSED_LO = $0080;
{ 32 bit word machine. }
IMAGE_FILE_32BIT_MACHINE = $0100;
{ Debugging info stripped from file in .DBG file }
IMAGE_FILE_DEBUG_STRIPPED = $0200;
{ If Image is on removable media, copy and run from the swap file}
IMAGE_FILE_REMOVABLE_RUN_FROM_SWAP = $0400;
{ If Image is on Net, copy and run from the swap file. }
IMAGE_FILE_NET_RUN_FROM_SWAP = $0800;
{ System File. }
IMAGE_FILE_SYSTEM = $1000;
{ File is a DLL. }
IMAGE_FILE_DLL = $2000;
{ File should only be run on a UP machine }
IMAGE_FILE_UP_SYSTEM_ONLY = $4000;
{ Bytes of machine word are reversed. }
IMAGE_FILE_BYTES_REVERSED_HI = $8000;

That's a lot of values, but for the purposes of this article, we're really only interested in the first one.

const
{ Relocation info stripped from file. }
IMAGE_FILE_RELOCS_STRIPPED = $0001;

That's where our magic number 1 comes from. If we add Windows to the project uses clause, and move the SetPEFlags directive below the uses clause, we can use this more self-documenting directive.

{$SetPEFlags IMAGE_FILE_RELOCS_STRIPPED}

So this tells the OS that the file does not contain any relocation information, and thus it cannot be rebased at runtime. In addition, the Delphi compiler now also picks up on this information and uses it as an instruction to do as you say - leaving out reloc information from the generated EXE file!


Note: The compiler allows using the same directive to remove reloc info from packages and DLLs, but this is not a recommended practice. It will generate slightly smaller DLLs, but the DLL will fail to load unless it can be loaded at its given base address.


Score: Delphi 7 vs. Delphi 2006: 0 - 1 ;)

10 comments:

Anonymous said...

Cool stuff! Until now i also ran the mentioned tool to strip off the reloc. table, so this nice. BTW can you give a suggestion on how to decide on and set the base address of DLL's of your application, so that they don't need to be rebased?

Anonymous said...

Much better to have undocumented features that WORKS, than documented features that does not work.

http://en.wikipedia.org/wiki/Undocumented_feature

Anonymous said...

BTW, work also with D2005

Anonymous said...

If you use a tool like the free ProcessExplorer from Sysinternals , processes that has relocated dll's will be marked with a yellow background as a warning. Excelent utility...

http://www.sysinternals.com/Utilities/ProcessExplorer.html

Kristofer G. Skaug said...

Hi Hallvard,

very interesting, and maybe it's just me being thick headed or the fact that my brain only switches on after 10AM, but it looks like you skipped the bit about how to decide the Image base address, and deciding when it's OK to strip the relocation table or not.

Roy Nelson said...

Wow good call!
Works in Delphi 2006!

Anonymous said...

Hi,

It does not work at all.
However my app ResXplor is only 562176 bytes. Maybe my D5 version, with upgrade 2, does the thing automatically.
Btw can you mention where you placed the directive exactly

Anonymous said...

@Holger: it does not work for mee, but there somebody else with D5 who is positive. Think about my suggestion that Update 2 Borland incoporated the removal in D5 itself

Anonymous said...

Sorry guys,
I missed a count. I use D6.

wac said...

Many ppl is not aware that forcing your exe to be loaded at a fixed address simply makes it more easily exploitable as all memory addresses are predictable. Security matters!! Please study about ASLR and how it makes at least your software harder to exploit even containing security bugs. You saved some insignificant bytes but killed security!! Also what you say about compressors trashing your executable is not 100% correct. Most trash it but some decompress pages on demand. That is if you don't use some pages in RAM of the program they will never be read at all!! Not even from the hard drive. Anyways thanks for the link to the tool. I need it for something else (not to save size, I don't do that).



Copyright © 2004-2007 by Hallvard Vassbotn