DN4DP#5: Redefining the operators

This post continues the series of The Delphi Language Chapter teasers from Jon Shemitz’ .NET 2.0 for Delphi Programmers book. Last time we talked about the new record syntax. This time we will look at the exciting new operator overloading capabilities.

Note that I do not get any royalties from the book and I highly recommend that you get your own copy – for instance at Amazon.

"Redefining the operators

Long-time Delphi programmers have looked at the operator overloading features of other languages (such as C++) with both fear and envy. The ability for user code to redefine the meaning of operators applied to an instance of a specific class is powerful and elegant if used properly, but it can be highly confusing and erroneous if used improperly.

Delphi now supports operator overloading. This is an advanced concept where a class or (more commonly, and the only kind supported in Win32) a record can have a special method called when a specific operator (such as +, -, /, *, div, mod, etc) is applied to an instance of the class or record. To define operator overloads you must define class operator functions with specific names for each operator. See the Delphi language documentation for the full list of class operator names.

In addition to normal operators, implicit and explicit casts (or conversions) can be implemented. The OperatorOverloading project demonstrates the potentially confusing aspects of operator overloading. It defines a TStrangeInt record where the operator semantics have been reversed:

type
  TStrangeInt = record
  public
    Value: Integer;
    class operator Add(const Left, Right: TStrangeInt): 
TStrangeInt; inline;
    class operator Subtract(const Left, Right: TStrangeInt): 
TStrangeInt; inline;
    class operator Multiply(const Left, Right: TStrangeInt): 
TStrangeInt; inline;
    class operator Divide(const Left, Right: TStrangeInt): 
TStrangeInt; inline;
    class operator Implicit(const AValue: Integer): 
TStrangeInt; inline;
    class operator Implicit(const AValue: TStrangeInt): 
Integer; inline;
    class operator Implicit(const AValue: TStrangeInt): 
string; inline;
    class operator Explicit(const AValue: Double): 
TStrangeInt; inline;
  end;

The implementation of a class operator method should create and return a new record or class instance and not modify any of the parameters. Often operator methods are very short and simple and thus perfect inline candidates[1] (inline routines are covered later in this Chapter).

class operator TStrangeInt.Add(const Left, Right: 
  TStrangeInt): TStrangeInt;
begin
  Result.Value := Left.Value - Right.Value;
end;

Invoking the overloaded operators is just a matter of declaring an instance of the record type and using the standard Delphi operators on it.

var
  Strange: TStrangeInt;
  StrangeResult: TStrangeInt;
begin
  Strange := 42;
  StrangeResult := Strange + Strange * 3;
end;

Because of the reversed TStrangeInt implementation, StrangeResult will be 28 instead of the expected 168. See Chapter 7 for more details on operator overloading.

Tip For a more complete example of operator overloading, see the Borland.Vcl.Complex .NET unit (or the Vassbotn.Vcl.Complex unit in the Demos\DelphiWin32\VCLWin32\ComplexNumbers\ folder for the corresponding Win32 unit).

---

[1] Marking overloaded operators with inline is purely optional, of course. Currently (Delphi 2006) it seems like the compiler manages to inline all simple operators except the Implicit and Explicit conversion operators."

Hack#15: Overriding message and dynamic methods at run-time

Last time we looked at a way of completely changing the class of a running object instance. As we discussed there, that hacking technique had a number of problems. But there are many ways to skin a cat (sorry, cat lovers!), and in the case of trying to fix the TProgressBar flickering issue on Windows Vista without changing the interface declaration of TProgressBar there are at least three other possible solutions.

One of these solution is to patch the existing TProgressBar VMT at runtime. We know from earlier articles that the VMT contains a load of information about a class - including an array of all virtual method implementation (the actual VMT - virtual method table) and a sparse array of all overridden and introduced dynamic and message methods (known as the DMT or dynamic method table). In this article we will look at how we can patch an existing class VMT and replace the DMT with one that fits our need.

Note that the compiler stores the class VMT tables in write-protected code pages (even though they do not strictly contain executable code). This means that if we naively try to overwrite parts of the VMT we will get access violations triggered by the hardware memory protection system. The clean way of writing self-modifying code is to use the VirtualProtect API to change the protection of the memory pages to allow writing before performing the patch, then restoring the original protection when we're done. Here is a routine that will safely overwrite a 4-byte DWORD in the code segment:

procedure PatchCodeDWORD(Code: PDWORD; Value: DWORD);
// Self-modifying code - change one DWORD in the code segment
var
  RestoreProtection, Ignore: DWORD;
begin
  if VirtualProtect(Code, SizeOf(Code^), PAGE_EXECUTE_READWRITE,
    RestoreProtection) then
  begin
    Code^ := Value;
    VirtualProtect(Code, SizeOf(Code^), RestoreProtection, Ignore);
    FlushInstructionCache(GetCurrentProcess, Code, SizeOf(Code^));
  end;
end;

To be on the safe side we follow Microsoft's recommended practice of flushing the instruction cache after performing the patch by calling FlushInstructionCache. This is to avoid nasty things from happening if the patched code has already been pre-fetched by (one of) the CPU.

Now we just have to figure out where to patch and what to patch it with. We have covered the details of how dynamic and message methods work in earlier posts. Looking at the source of TProgressBar and confirmed by peeking with the debugger, there are no dynamic or message method overrides at all in this class. This is good news because it means that the DynamicTable pointer in the VMT is nil and this makes it much easier to perform the patch. So now we know where to patch - the DMT slot in the VMT of TProgressBar. To keep things simple in the initial version we use the vmtDynamicTable constant exported by the System unit.

{ Virtual method table entries }
const
  // ...
  vmtDynamicTable      = -48;

Given a TProgressBar class reference we can calculate the address of the DMT slot in the VMT by using this code:

var
  OriginalDmt: PDWORD;
begin
  OriginalDmt := PDWORD(TProgressBar);
  Inc(OriginalDmt, vmtDynamicTable div SizeOf(DWORD));
  if OriginalDmt^ = 0 then // Assert that there is no existing Dmt

Now we have to find what to replace the nil DMT pointer with. We could try to build a DMT table by hand by using the tables we documented here, but it would be wasted effort. The compiler is much better than us building VMTs and DMTs, so we can just use the TProgressBarVistaFix from the previous post.

type
  TProgressBarVistaFix = class(TProgressBar)
  private
    procedure WMEraseBkgnd(var Message: TWmEraseBkgnd); message
WM_ERASEBKGND;
  end;

procedure TProgressBarVistaFix.WMEraseBkgnd(var Message: TWmEraseBkgnd);
begin
  DefaultHandler(Message);
end;

We can get to the DMT pointer for this new class in the same way and use it to overwrite the (previously nil) DMT pointer in the TProgressBar class. This is my initial patching routine:

procedure PatchTProgressBarDMT;
var
  OriginalDmt, NewDmt: PDWORD;
begin
  OriginalDmt := PDWORD(TProgressBar);
  Inc(OriginalDmt, vmtDynamicTable div SizeOf(DWORD));
  if OriginalDmt^ = 0 then // Assert that there is no existing Dmt
  begin
    NewDmt := PDWORD(TProgressBarVistaFix);
    Inc(NewDmt, vmtDynamicTable div SizeOf(DWORD));
    PatchCodeDWORD(OriginalDmt, NewDmt^);
  end;
end;

Testing this code shows that it works and that the WM_ERASEBKFND handler is called and the flicker is gone on Vista. However, I'm not too satisfied with the code, so I refactor it into a separate general unit - this time using the HVVMT unit and the PVmt record pointer.

unit HVPatching;

interface

uses
  Windows;

procedure PatchCodeDWORD(Code: PDWORD; Value: DWORD);

procedure ReplaceClassDmt(TargetClass, SourceClass: TClass);

implementation

uses
  HVVMT;

procedure PatchCodeDWORD(Code: PDWORD; Value: DWORD);
// Self-modifying code - change one DWORD in the code segment
var
  RestoreProtection, Ignore: DWORD;
begin
  if VirtualProtect(Code, SizeOf(Code^), PAGE_EXECUTE_READWRITE,
    RestoreProtection) then
  begin
    Code^ := Value;
    VirtualProtect(Code, SizeOf(Code^), RestoreProtection, Ignore);
    FlushInstructionCache(GetCurrentProcess, Code, SizeOf(Code^));
  end;
end;

procedure ReplaceClassDmt(TargetClass, SourceClass: TClass);
begin
  Assert(Assigned(TargetClass) and Assigned(SourceClass));
  PatchCodeDWORD(@GetVmt(TargetClass).DynamicTable, 
    DWORD(GetVmt(SourceClass).DynamicTable));
end;

end.

Then we can use it like this:

initialization
  ReplaceClassDmt(TProgressBar, TProgressBarVistaFix);
end.

In this case we were lucky. In other cases the patched class may already have one or more dynamic or message methods - so that there will already be a DMT pointer in the VMT. In this case we must play the compiler and manually allocate a new DMT with one extra entry, copy the existing DMT indicies and method code pointers and finally patch in the new dynamic method index or message-id and the address of the hook routine. This is quite a bit more complex, but if there is interest in digging further down we might revisit this later.

Hack#14: Changing the class of an object at run-time

Sometimes (such as when you for one reason or another need to stay backwards compatible with binary dcus) you may have to employ a hack or two. One such hack is to change the actual run-time class of an object instance. You might need to do this to override a virtual, dynamic or message method, for instance.

Vista and ProgressBar flickering

One such case; for some reason (most probably a bug fix) Microsoft changed the behavior of the ProgressBar control in Windows Vista so that it now sends a WM_ERASEBKGND message every time the progress changes and it needs to redraw itself. The TProgressBar component that wraps the native control does not handle WM_ERASEBKGND - nether as an explicit message override or in the general WndProc method (in fact it doesn't override WndProc). The net result is that each time the progress bar's Position value changes noticeable flicker can be observed as the control first erases the background completely and then draws the progress bar in the correct style. Jordan Russell reported this issue in Quality Central #38178.

Since Delphi 2007 was a release to target Windows Vista issues like this, it would have been simple to add a WM_ERASEBKGND override that skips the default TWinControl logic of FillRect'ing the client area.

type
  TProgressBar = class(TWinControl)
  private
    procedure WMEraseBkgnd(var Message: TWmEraseBkgnd); 
  message WM_ERASEBKGND;
  end;

procedure TProgressBar.WMEraseBkgnd(var Message: TWmEraseBkgnd);
begin
  DefaultHandler(Message);
end;

The trouble is of course that Delphi 2007 was to be non-breaking release, so no interface section changes of existing classes was possible. But there are at least three different ways of hacking a solution anyway. In this article we will look at the simplest and probably least robust solution, changing the class of all TProgressBar instances at runtime.

Changing the class

On the face of it, you might think it is impossible to change the class of an existing object instance. The class of an instance is determined by two things; the class type T that the object reference was declared as at compile time and the class type D that was used to create the object instance and assign to the object reference. D must be assignment compatible to T and thus must be T or derived from D.

type
  T = class
  end;
  D = class(T)
  end;

procedure Foo;
var
  Ref: T;
begin
  Ref := D.Create;
end;

While you cannot change the declared type of the object reference, you can change the runtime type of an object instance. Why is this possible? Well, the runtime type of an object instance is stored in an implicit field of the instance memory - the first 4 bytes of an instance always contain the TClass reference (implemented as a pointer to the class' VMT) of the class that was used to create the object. This reserved field is initialized in the InitInstance class method defined in TObject.

class function TObject.InitInstance(Instance: Pointer): TObject;
begin
  FillChar(Instance^, InstanceSize, 0);
  PInteger(Instance)^ := Integer(Self);

This method performs other tasks as well (such as initializing all interface method table fields), but what interests us is that upon entry the Instance parameter points to a raw uninitialized block of memory (you can see that NewInstance calls GetMem and then InitInstance). This block is first cleared to all zeros by calling FillChar (this is what ensures that all object instance fields are 0, nil, false etc) and then overwrites the first 4 bytes with the TClass reference (which is available in the implicit Self parameter for non-static class methods).

Phew! Now that we know that the de-facto runtime class of an object instance is effectively stored as a field with a known offset (0), it is actually amazingly easy to change the class - we can just overwrite the TClass reference there with another TClass value. To ensure that things don't crash (the compiler makes assumptions about field offsets, virtual method indexes and so on based on the declared type of an object reference) we have to be careful to only overwrite the VMT slot with a class that derives from the current one. And since the object instance has already been allocated with a fixed size, we should not add any instance fields to the derived class.

Let's see how we can use this technique in the TProgressBar case. First lets declare and implement a TProgressBar descendant that fixes the Vista issue.

type
  TProgressBarVistaFix = class(TProgressBar)
  private
    procedure WMEraseBkgnd(var Message: TWmEraseBkgnd); 
  message WM_ERASEBKGND;
  end;

procedure TProgressBarVistaFix.WMEraseBkgnd(var 
  Message: TWmEraseBkgnd);
begin
  DefaultHandler(Message);
end;

It is basically identical to the interface-breaking, clean-solution code above - we only had to change the name of the class and declare that it inherits from TProgressBar. Then we have to write the code that takes an existing TProgressBar instance and changes its runtime class to TProgressBarVistaFix.

procedure PatchProgressBar(ProgressBar: TProgressBar);
type
  PClass = ^TClass;
begin
  if ProgressBar.ClassType = TProgressBar then
    PClass(ProgressBar)^ := TProgressBarVistaFix;
end;

Note that it first checks to see that the actual class type is exactly TProgressBar and not some derived type - that would mess things up because TProgressBarVistaFix derives from TProgressBar.

One of the problems with this hack technique is that you need to explicitly patch each progress bar instance. You could do this in the FormCreate of all forms containing one or more progress bars, for instance. Or even in an button's OnClick event handler as I've done in a simple demo app.

procedure TForm5.PatchBtnClick(Sender: TObject);
begin
  PatchProgressBar(ProgressBar2);
end;

If we were CodeGear (or if we feel adventurous) we could apply the patching to a single point in the implementation of the TProgressBar constructor.

constructor TProgressBar.Create(AOwner: TComponent);
begin
  PatchProgressBar(Self);
  // ...
end;

This hack can be useful when you need to quickly change the runtime class of an object on an ad-hoc manner. It has a number of disadvantages, however:

• It requires patching each object instance explicitly
  
• It changes the class of the instance. In our case ClassName will return 'TProgressBarVistaFix', for instance. One workaround for this would be to move the TProgressBarVistaFix class to a separate VistaFixes unit and name it TProgressBar. Still the RTTI and class reference will be different than the original class. 
  
• It doesn't fix the problem for descendants of the patched class. In the TProgressBar case there are probably not so many of those around..?
  
• To ensure that all object instances are patched, you need to have access to the source of and change the implementation of the original class' constructor.
  
• The compiler creates a new VMT for the hack class, so the code size increases accordingly (yeah, who cares?, right).

The benefits of this hack is:

• It is very simple in the sense that it does not require writing into any protected code pages (which would require the use of VirtualProtect or WriteProcessMemory).
  
• It is easy to extend it to override any number of virtual, message and dynamic methods. It can also be used to promote properties to published so that RTTI is generated for them. Just update the hack class and rely on the compiler to build the proper VMT information for you.
  
• It is flexible in that It is possible to patch only selected object instances instead of all instances of a class

Both this hack and the other variants that we will examine in upcoming blog posts have certain deficiencies that is hard to overcome due to the nature of the VMT and compiler. The hacks work well for leaf classes that do not have any descendants, but if you use them to override a virtual method in a non-leaf class (i.e. one that has descendants), and that virtual method is also overridden in one of the descendant classes, and the descendant method calls back to the inherited method, things will not work as if the method had been overridden in the clean way. Clear as mud? We'll revisit this topic later.

Conclusion

The hack presented here of changing the class of an object instance at runtime can be useful in a limited set of cases. Only use it for leaf classes and when replacing the class VMT with a new one doesn't introduce issues (such as ClassName changing). For most other cases, other hacking techniques would be better. We'll look at a couple of these in upcoming articles. Stay tuned! ;)

[Updated: reduced extensive "boldness" of the text ;)]

Delphi 2007 ESD arrived - one gotcha

We ordered Delphi 2007 ESD from the local Norwegian CodeGear distributor alfacode.no a few days ago and earlier today the email with download link and licence keys arrived (the DVD will arrive in a couple of weeks, they say).

I downloaded the installer stub that then downloads and installs the required binaries - it all worked very smoothly. After the install was done Delphi 2007 ran fine. But I was a little worried for a while - because when I tried to start BDS 2006, I got an error dialog with the following unnerving message:

---------------------------
bds.exe - Entry Point Not Found
---------------------------
The procedure entry point @Uxtheme@BufferedPaintSetAlpha$qqruip11Types@TRectuc could not be located in the dynamic link library rtl100.bpl.
---------------------------
OK
---------------------------

After scratching my head for a while, I searched by harddisk for all rtl*.bpl files. Both Delphi 2007 and BDS 2006 use a binary called rtl100.bpl - and normally it resides in the Windows\System32 directory. The Delphi 2007 installer updates this (and most other VCL bpls) when it installs. The problem is that I had a second copy of this file in the BDS 2006 bin directory (called C:\Delphi2006\bin on my system). This was entirely my own fault because I had copied it there and patched it using Peter Vones tool to hack away startup time as I've blogged about before.

The reason this happens is that Delphi 2007 installer upgrades all the *100.bpls in the System32 directory, but BDS 2006 tries to load the old rtl100.bpl from the bds.exe startup directory that is not compatible with the new vcl100.bpls now residing in System32. The solution was simply to delete or rename the rtl100.bpl in the Delphi2006\bin directory.

Lesson learned: it is dangerous to live on the edge - and you better know (and remember) what you are doing.... ;).

Application Shutdown Changes in Windows Vista

Microsoft Vista new best practices, implementation and APIs for application shutdown notification and behavior: Application Shutdown Changes in Windows Vista

The Delphi Geek: Delphi 2007 is here!

Primoz Gabrijelcic has created a summary with links to all blog posts about the Delphi 2007 beta: The Delphi Geek: Delphi 2007 is here!

Vista User Account Control and Command Line Applications

Fredrik Haglund has a good post on Vista and UAC: Vista User Account Control and Command Line Applications

DN4DP#4: Setting new records

This post continues the series of The Delphi Language Chapter teasers from Jon Shemitz’ .NET 2.0 for Delphi Programmers book. Last time we talked about the new capability of nesting types and constants inside other class declarations. This time we will look at the extended and OOP-like syntax and functionality for plain old record types.

Note that I do not get any royalties from the book and I highly recommend that you get your own copy – for instance at Amazon.

"Setting new records

Records have now been upgraded to proper object-oriented citizens[1]. Instead of being just a collection of passive fields, a record declaration can now contain most of the features of a full-fledged class. This includes visibility specifiers (private, strict private and public), instance methods, constructors (as long as they have at least one parameter), static class methods, class fields, class properties, class constructors (only in .NET) and nested types and constants.

Records do not support inheritance, virtual methods, protected sections, destructors or constructors with no parameters. In .NET a record can implement one or more interfaces and it is also possible to forward-declare a record (just like you can with classes). From the OORecords project

type
  TFoo = record;  // forward record declaration
  IRecordInterface = interface
    procedure Method(Foo: TFoo);
  end;
  TFoo = record(IRecordInterface)
  strict private
    class constructor ClassCreate;
    procedure Method(Foo: TFoo);
    class var FCount: integer;
    var FBar: integer;
  public
    procedure SetBar(const Value: integer);
    class procedure SetCount(const Value: integer); static;
  public
    constructor Create(ABar: integer);
    property Bar: integer read FBar write SetBar;
    class procedure ReportCount; static;
    class property Count: integer read FCount write SetCount;
  end;

It is useful to be able to declare methods as part of a lightweight record object[2] instead of encoding them in separate global routines (as has been customary in native Delphi). This couples data and behavior in a way that is object-oriented, easy to understand and easy to share with other languages. Records with methods is also a requisite for supporting operator overloading, which brings us to our next topic [operator overloading - which we will look at in the next DN4DP blog post - Ed.]

---

[1] Old time Delphi and Turbo/Borland Pascal programmers may remember the old object style classes that could also be used as a kind of object oriented records. Borland chose to extend the record concept instead of reviving the deprecated object syntax, mostly to avoid breaking old code. The old object model supported inheritance and a number of other features that the new OOP records don’t (and cannot in .NET).

[2] See Chapter 5 for more details of value types vs. reference types."

Review: Delphi 2007 for Win32 (Beta) - part three

Read the part one and two first.

What's new in the VCL

There are quite a few bugfixes in the VCL (as well as the RTL), but we'll not go into them in detail here. While remaining binary .dcu compatible, CodeGear has managed the feat of adding new functionality and even new properties on the existing TCustomForm class.

The GlassFrame property hack

In particular, all forms now have a GlassFrame property with sub-properties to control extended glass functionality when running on Windows Vista. Allen Bauer has blogged about this in his post "How to add a "published" property without breaking DCU compatibility". Please read his blog entry first - that will make it easier to understand the rest of this section.

First of all, what does the GlassFrame property contain and what does it do? Let's pretend that it had been implemented the "proper" way by adding a normal property to TCustomForm that is promoted to published in TForm. Then the code changes would look like this:

type
  TCustomForm = class;
  TGlassFrame = class(TPersistent)
  public
    constructor Create(Client: TCustomForm);
    procedure Assign(Source: TPersistent); override;
    function FrameExtended: Boolean;
    function IntersectsControl(Control: TControl): Boolean;
    property OnChange: TNotifyEvent {...};
  published
    property Enabled: Boolean {...} default False;
    property Left: Integer {...} default 0;
    property Top: Integer {...} default 0;
    property Right: Integer {...} default 0;
    property Bottom: Integer {...} default 0;
    property SheetOfGlass: Boolean {...} default False;
  end;
  TCustomForm = class(TScrollingWinControl)
  public
    procedure UpdateGlassFrame(Sender: TObject);
    property GlassFrame: TGlassFrame {...} ;
  end;
  TForm = class(TCustomForm)
  published
    property GlassFrame;
  end;

For simplicity I've left out the private and protected implementation details. Note that the new GlassFrame property has the type TGlassFrame which is a TPersistent descendent. This means that its properties will appear in the Object Inspector as sub-properties of GlassFrame. The Left, Top, Right and Bottom properties defines how large the border of glass around the non-transparent part of the form should be. If you set SheetOfGlass to True, the entire form will be glass. Finally the Enabled property can be used to quickly turn the extra glass effect on or off. Note that the glass effect controlled by the GlassFrame property is in addition to the normal glassed non-client frame (caption and window border) provided by Vista. I don't have Vista running on my low-powered Acer laptop here, so you can take a peek at the screenshots over in Jeremy North's blogpost about the glass effect here and here to see how it looks at design-time and run-time.

The challenge that faced CodeGear in implementing the GlassFrame property and functionality in the non-breaking Delphi 2007 version was four-fold.

• The GlassFrame property needs to be available at runtime and appear to compiling code to reside on all form instances. This is solved using a class helper called TCustomFormHelper.
  
• Class helpers cannot add new fields or per-instance storage. Somehow the storage needs of the GlassFrame property needs to be satisfied. This is done by using a hack and reusing one of the existing TCustomForm private fields (FPixelsPerInch)
  
• While class helpers are great for creating a runtime mirage effect, "fooling" your code to see an injected property on an existing class, they do not help with RTTI and thus getting the property into the Object Inspector. As Allen explained in his article, in BDS 2006 they introduced a new selection editor interface called ISelectionPropertyFilter that makes it possible to dynamically add and remove design-time properties.
  
• Finally, the GlassFrame property needs to be streamed to and from the .dfm storage. This is not solved by the class helper alone, nor the property filter interface. The solution is to use the existing DefineProperties mechanism, but an extra twist is needed to support Property.SubProperty names for defined properties.

Lets dive into each of these issues in more detail. After all, this blog is mostly about hacks and the Delphi 2007 GlassFrame feature is arguably the most high-profile and best (?) hack in the VCL ever :-). Note: The source code I show here is for illustration only and is based on a beta build of Delphi 2007 - check your own copy of Forms.pas in the shipping version.

The CustomFormHelper class helper

Here is what the class helper that provides the runtime GlassFrame property looks like:

  TCustomFormHelper = class helper for TCustomForm
  private
    function GetGlassFrame: TGlassFrame;
    procedure ReadGlassFrameBottom(Reader: TReader);
    procedure ReadGlassFrameEnabled(Reader: TReader);
    procedure ReadGlassFrameLeft(Reader: TReader);
    procedure ReadGlassFrameRight(Reader: TReader);
    procedure ReadGlassFrameSheetOfGlass(Reader: TReader);
    procedure ReadGlassFrameTop(Reader: TReader);
    procedure SetGlassFrame(const Value: TGlassFrame);
    procedure WriteGlassFrameBottom(Writer: TWriter);
    procedure WriteGlassFrameEnabled(Writer: TWriter);
    procedure WriteGlassFrameLeft(Writer: TWriter);
    procedure WriteGlassFrameRight(Writer: TWriter);
    procedure WriteGlassFrameSheetOfGlass(Writer: TWriter);
    procedure WriteGlassFrameTop(Writer: TWriter);
  public
    procedure UpdateGlassFrame(Sender: TObject);
    property GlassFrame: TGlassFrame read GetGlassFrame 
      write SetGlassFrame;
  end;

This provides the public GlassFrame property that is transposed onto TCustomForm and all descendants. It also makes the UpdateGlassFrame method available, but this is mostly used internally in the Forms unit. It is the target of the OnChange event defined in the TGlassFrame class, forcing the form to repaint itself whenever one of the GlassFrame properties changes. Finally there are the ReadXXX and WriteXXX methods used in the TCustomForm.DefineProperties method to stream the GlassFrame properties to and from .dfm files. We'll discuss this in more detail below.

This is a virtual method defined on TPersistent. Luckily, TCustomForm already overrode this method in BDS 2006 (to store the pseudo properties PixelsPerInch, TextHeight and IgnoreFontProperty) so it was simple to add the new GlassFrame properties there.

The FPixelsPerInch storage hack 

As you can see above the class helper does not have (and cannot have) and instance fields. Still the GlassFrame instance pointer has to be stored somewhere - and it needs to be stored per form instance. There are several different potential solutions to this. One possibility is to use some kind of hash-table to map form instances into corresponding GlassFrame instances, but this is complex, could be relatively slow and require extra coordination to make sure the GlassFrame instance is freed when the form is freed etc. The other solution is to stash the information into one of the existing fields of TCustomForm.

This is what CodeGear decided to do. They picked a relatively seldom used private field that does not have its address exposed via property accessors (you can read about why this would have been dangerous here).

type
  TCustomForm = class(TScrollingWinControl)
  private
    FPixelsPerInch: Integer;
  end;

The FPixelsPerInch field is declared as Integer, but in the implementation section it is actually treated as a pointer to a record structure.

{ Hack to overlay GlassFrame on PixelsPerInch in TCustomForm }
type
  PPixelsPerInchOverload = ^TPixelsPerInchOverload;
  TPixelsPerInchOverload = record
    PixelsPerInch: Integer;
    GlassFrame: TGlassFrame;
    RefreshGlassFrame: Boolean;
  end;

The record gives storage for both the PixelsPerInch property (declared on TCustomForm) the new GlassFrame property (injected by TCustomFormHelper) and another private implementation field called RefreshGlassFrame.

The TCustomForm constructor and destructor allocate and deallocate the FPixelsPerInch field as a TPixelsPerInchOverload pointer.

constructor TCustomForm.CreateNew(AOwner: TComponent; Dummy: Integer);
begin
  Pointer(FPixelsPerInch) := AllocMem(SizeOf(TPixelsPerInchOverload));
  inherited Create(AOwner);
  //...
end;

destructor TCustomForm.Destroy;
begin
  //...
    FreeMem(Pointer(FPixelsPerInch));
    inherited Destroy;
  //...
end;

All access to the PixelsPerInch and RefreshGlassFrame fields in the TPixelsPerInchOverload record are delegated to a set of inlined getter and setter routines, such as this one:

function GetFPixelsPerInch(FPixelsPerInch: Integer): Integer; inline;
begin
  Result := PPixelsPerInchOverload(FPixelsPerInch).PixelsPerInch;
end;

Finally the class helper methods can now use the same trick to get and store the GlassFrame property:

function TCustomFormHelper.GetGlassFrame: TGlassFrame;
begin
  Result := PPixelsPerInchOverload(FPixelsPerInch).GlassFrame;
end;

procedure TCustomFormHelper.SetGlassFrame(const Value: TGlassFrame);
begin
  PPixelsPerInchOverload(FPixelsPerInch).GlassFrame.Assign(Value);
end;

Neat hack, no? ;)

Injecting design-time properties

The next step is to convince the Object Inspector to make the GlassFrame compound property available for inspection and editing at design-time. This is achieved using a little known component selection interface called ISelectionPropertyFilter. This interface was first introduced in Delphi 2006 and used to help implement the ControlIndex property injected into all components dropped on a TFlowPanel or TGridPanel. These components are documented in a BDN article by Ed Vander Hoek here and the ISelectionPropertyFilter interface and how it is used is discussed by Tjipke A. van der Plaats here.

The interface is declared in the DesignIntf unit and looks like this:

{ ISelectionPropertyFilter
This optional interface is implemented on the same class that implements
ISelectionEditor.  If this interface is implemented, when the property list
is constructed for a given selection, it is also passed through all the various
implementations of this interface on the selected selection editors.  From here
the list of properties can be modified to add or remove properties from the list.
If properties are added, then it is the responsibility of the implementor to
properly construct an appropriate implementation of the IProperty interface.
Since an added "property" will typically *not* be available via the normal RTTI
mechanisms, it is the implementor's responsibility to make sure that the property
editor overrides those methods that would normally access the RTTI for the
selected objects.

FilterProperties
Once the list of properties has been gathered and before they are sent to the
Object Inspector, this method is called with the list of properties.  You may
manupulate this list in any way you see fit, however, remember that another
selection editor *may* have already modified the list.  You are not guaranteed
to have the original list. }
  ISelectionPropertyFilter = interface
    ['{0B424EF6-2F2F-41AB-A082-831292FA91A5}']
    procedure FilterProperties(const ASelection: IDesignerSelections;
      const ASelectionProperties: IInterfaceList);
  end;

There is no shipping source for it, but one of the IDE packages registers a selection editor for TCustomForm (and thus all descendants) using the DesignIntf.RegisterSelectionEditor routine.

type
{ TBaseSelectionEditor
  All selection editors are assumed to derive from this class. A default
  implemenation for the ISelectionEditor interface is provided in
  TSelectionEditor class. }
  TBaseSelectionEditor = class(TInterfacedObject)
  public
    constructor Create(const ADesigner: IDesigner); virtual;
  end;

  TSelectionEditorClass = class of TBaseSelectionEditor;

procedure RegisterSelectionEditor(AClass: TClass; AEditor: TSelectionEditorClass);

You can see some of the details of how the ISelectionPropertyFilter is used in the DesignEditors unit and the GetComponentProperties routine. This routine is called by the IDE when you select a form and it gets the list of registered selection editors for forms and each of the selection editor classes that implement the ISelectionPropertyFilter interface have their FilterProperties method called. This allows it to remove or add properties to the list that is eventually presented to the user in the Object Inspector. The new TCustomForm selection editor adds an implementation of IProperty for the new ghost property GlassFrame. The net result is that as far as the Object Inspector is concerned, the GlassFrame looks like a published property on TCustomForm, even though it isn't actually there and there is no RTTI for it. Feels like magic! ;).

Defining the streaming properties

The final piece of the puzzle to make the GlassFrame property illusion complete is to gel with the .dfm streaming system. It would help much having the GlassFrame property in the Object Inspector if the values you set didn't persist between runs of the IDE or when loading the form at runtime. The virtual DefineProperties method on TPersistent has always been part of the Delphi streaming system. You override it to store additional information than the published properties. Luckily TCustomForm already overrides DefineProperties (to store PixelsPerInch, TextHeight and IgnoreFontProperty). This means that additional code can be added to store the GlassFrame property without changing the interfaced part of TCustomForm. Here is the new DefineProperties method.

procedure TCustomForm.DefineProperties(Filer: TFiler);
//...
begin
  inherited DefineProperties(Filer);
  Filer.DefineProperty('PixelsPerInch', {...});
  Filer.DefineProperty('TextHeight', {...});
  Filer.DefineProperty('IgnoreFontProperty', {...});
  Filer.DefineProperty('GlassFrame.Bottom', {...});
  Filer.DefineProperty('GlassFrame.Enabled', {...});
  Filer.DefineProperty('GlassFrame.Left', {...});
  Filer.DefineProperty('GlassFrame.Right', {...});
  Filer.DefineProperty('GlassFrame.SheetOfGlass', {...});
  Filer.DefineProperty('GlassFrame.Top', {...});
end;

I have commented out the details about the ReadData, WriteData and HasData parameters of each DefineProperty call. The actual reading and writing has been delegated to the private Read and Write methods of the TCustomFormHelper that we listed at the top of this article.

The special thing to note here is that the GlassFrame properties are stored using a nested 'GlassFrame.SubProperty' name. In earlier versions of Delphi and the streaming subsystem this would not have worked, but there is now extra code in the Classes unit that makes this possible.

procedure TReader.ReadProperty(AInstance: TPersistent);
//...
        PropInfo := GetPropInfo(Instance.ClassInfo, FPropName);
        if PropInfo = nil then
        begin
          // Call DefineProperties with the entire PropPath
          // to allow defining properties such as "Prop.SubProp"
          FPropName := PropPath;
          { Cannot reliably recover from an error in a defined property }
          FCanHandleExcepts := False;
          Instance.DefineProperties(Self);
          FCanHandleExcepts := True;
          if FPropName <> '' then
            PropertyError(FPropName);
          Exit;
        end;

The reason for doing it this way is that this is an important aspect in the future plans for the GlassFrame implementation.

A GlassFrame into the future

In future binary-breaking releases of Delphi (aka Highlander or BDS (CDS?) 2007) the GlassFrame implementation will be folded into the proper classes. Allen talks about this too. So most probably, GlassFrame will become a proper property on TCustomForm (and promoted to published in TForm), the TCustomFormHelper and the tricks discussed above will disappear. The neat thing, though, is that all code and .dfms using it will just continue to work. The 'GlassFrame.Subproperty' names of the defined properties will map directly to the nested properties of the GlassFrame instance. So while the inner workings will change, the externally observable behavior will stay the same. Impressive, don't you think?!

Other VCL changes

We running out of time and space (at least metaphorically), so we'll just quickly mention the other main changes to the VCL in Delphi 2007. Applications built with Delphi has normally been very easy to spot, because the taskbar icon belongs to a special hidden TApplicaton.Handle instead of the actual main form, so the system menu has been a little short ;). This behavior has now been made optional - old applications should have the old behavior while new projects now set a new Application. MainFormOnTaskBar property to True. This ensures that the icon in the taskbar actually belongs to the main form, instead of the application handle. This will fix a few issues in general and on Vista in particular. The new property is again temporarily provided by a class helper called TApplicationHelper, but this time without the extra design-time and streaming shenanigans. This class helper also injects a somewhat cryptically named EnumAllWindowsOnActivateHint property that as far as I can see provides a fix for showing hints for windows that belongs to the process, but was created in another thread.

type
  TApplicationHelper = class helper for TApplication
  public
    property EnumAllWindowsOnActivateHint: Boolean {...};
    property MainFormOnTaskBar: Boolean {...};
  end;

A lot of the other changes in VCL are to properly support themeing and painting correctly with a glass form background, but at design-time and run-time. There are new Vista specific components in the Dialogs unit named TFileOpenDialog, TFileSaveDialog and TTaskDialog. These are all marked with the platform directive, so if you use them directly, they will only work in Windows Vista.

In addition there is a global UseLatestCommonDialogs variable that when set to True (the default) will automatically upgrade the good old TOpenDialog, TSaveDialog and MessageDlg into using the new Vista GUI look when available. Primoz Gabrijelcic has a nice post and good screenshots of these dialogs here. 

The other new features

I'm not a database guy, but the database express architecture has been updated to generation 4 (DBX 4). This involves potential performance improvements, single sourcing database code for both native and managed code, (some) drivers with source code, all Delphi code, backward compatibility with DBX 3 drivers, connection pooling and more. Looks very impressive. The returned CodeGear database guru Steve Shaughnessy knows more about this stuff than anyone else - get the details here. An overview of the database architecture classes (generated using Together) can be seen here in CodeGear's Andreano Lanusse's blog.

Delphi 2007 ships with a new version of IntraWeb that CodeGear dubs VCL for the Web and it includes Ajax functionality. The native Win32 SOAP and WebServices support has had many bugfixes and performance improvements and is more capable than ever. As we noted in the screenshots of the first article in this series, MS Build is now used as the build engine allowing you to customize the build process with pre-compile and post-compile events. And it supports multiple build configurations for debug and release builds, for instance.

The debugger is an invaluable tool for experienced developers and now it is better than ever. I'll probably blog about it in more detail later. Often people use only a fraction of the useful feature in the debugger, mostly because they don't know about them or how to use them. The Call Stack has a number of improvements. There are glyphs indicating if a stack frame has debug info available or not. You can now set break points directly in the Call Stack - the debugger will break when control returns to that point. When you double click an entry in the Call Stack it will now show the locals in that frame in addition to navigating to the correct spot in the source code. In the default keyboard mapping, you can now press Shift+F5 to enable or disable a breakpoint on the current line (ah - nice to reduce those mouse operations!).

Phew! Hope you have enjoyed this little series on the Delphi 2007 beta.

Conclusion

It feels like Delphi 2007 is going to be a rock-solid release. Here is a summary of my conclusions of who Delphi 2007 is for:

• If you're developing on or for Windows Vista, you'll want to get Delphi 2007.
  
• If you're already on BDS 2006 and plan to upgrade to BDS 2007, you should seriously consider buying Delphi 2007 with SA (Software Assurance) or subscription now - that will (by all likelihood) give you BDS 2007 later this year. The transition from Win32 development on BDS 2006 should be very smooth - just keep using the same components without recompiling them. No need to wait for new versions from 3rd parties.
  
• If you are a Delphi 5-7 developer that is still sitting on the fence, this is the time to jump and take this offer. The IDE is much more capable and productive and the performance and flicker-free operation is on par with or better than Delphi 7 now. You can see the list of improvements between Delphi 7 and BDS 20006 in Nick Hodges blog here - all of these goodies (on the native Win32 side) are also part of Delphi 2007, of course.

I highly recommend Delphi 2007 Win32 for all native Delphi developers!

Review: Delphi 2007 for Win32 (Beta) - part two

Read part one here first.

What more is new in the compiler?

New published vs $M+ behavior

The new "W1055 PUBLISHED caused RTTI ($M+) to be added to type '%s' " warning is interesting. It solves one of the issues we discussed earlier. In earlier versions of the compiler, if you compiled code like this:

type
  TMyClass = class
  private
    FName: string;
  published
    property Name: string read FName write FName;
  end;

The published property would not have RTTI generated, but would be silently treated as a public property. This is because TMyClass does not derive from a class compiled with $M+ enabled (such as TPersistent) and $M+ is not specified for the TMyClass itself. When you compile the same code in Delphi 2007 you will now get a warning:

[DCC Warning] ThSort.pas(13): W1055 PUBLISHED caused RTTI ($M+) to be added to type 'TMyClass'

This indicates that despite the missing $M+, the compiler has promoted the published properties to published and generated RTTI fro them. To remove the warning, add a $M+ directive in front of the class - or change published to public - if that is what you really meant. Nice touch!

$DYNAMICBASE and ASLR 

The compiler accepts a new $DYNAMICBASE ON/OFF compiler directive, and a corresponding command line parameter called "--dynamicbase". This is a shortcut for the existing $SetPEOptFlags directive. $DYNAMICBASE ON corresponds to $SetPEOptFlags $40 where $40 is defined by Windows as:

const IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE = $0040; 
  // The DLL can be relocated at load time.

The SetPEOptFlags directive sets a field in the PE header that is also called DllCharacteristics. It is used by the Windows loader to determine capabilities of a loaded module (DLL or BPL, for instance). The bit $40 seem to have a specific meaning in Windows Vista and enables a feature called Address Space Layout Randomization (ASLR). It means that the OS loads the module at a more or less random address (actually a random offset between 0 and 255) instead of trying to load it at exactly at the specified imagebase address. This is a security and anti-remote-hack measure making it harder to construct buffer overflow attacks that call known system DLL routines at fixed addresses. You can read more about it here.

--description command line option

The compiler also has a new command line option --description:<string> that corresponds to the $DESCRIPTION compiler directive. It lets you set the module description entry in the PE header.

What's new in the RTL?

The run time library contains the lowest level building blocks in the Delphi eco-system above the compiler. The special SysInit and System units are tightly bound to the internals of the compiler and linker. When using packages, SysInit is compiled into every module (EXE and package BPLs), while the System unit is only compiled into the rtl100.bpl. That is the reason they are split into to physically different units.

That's enough nitty-gritty background. What is new, then? Well, SysInit hasn't changed. I guess this is an indication that it has stabilized and don't need any changes for fixes. So, good news. Let's dive into the inner secrets of the System unit changes.

The System unit changes

There is a new interfaced boolean global variable with the catchy name NeverSleepOnMMThreadContention. It is used in the new FastMM based memory manager (courtesy of Pierre le Riche) that resides in the getmem.inc file that is included in the implementation section of System.pas. Here is the comment that accompanies the variable declaration:

{Set this variable to true to employ a "busy waiting" loop instead of putting
the thread to sleep if a thread contention occurs inside the memory manager.
This may improve performance on multi-CPU systems with a relatively low thread
count, but will hurt performance otherwise.}
NeverSleepOnMMThreadContention: Boolean; 

When the FastMM memory manager needs to protect shared resources, it uses light-weight atomic operations (such as the lock cmpxchg assembly instruction) instead of the more heavy-duty OS-level critical section APIs. If there is contention (the resource has already been locked by another thread), it normally sleeps to release the CPU and allow the other thread to finish its work and release the lock.

As the comment above alludes to, in some situations on multiprocessor machines (that is becoming mainstream) it may be more efficient to just keep looping and checking the lock availability (also called a spin-lock) while waiting for the other thread to execute on one of the other processors to release the lock. Here is an example of how the Pascal version of this logic looks like:

    while LockCmpxchg(0, 1, @MediumBlocksLocked) <> 0 do
    begin
      if not NeverSleepOnMMThreadContention then
      begin
        Sleep(InitialSleepTime);
        if LockCmpxchg(0, 1, @MediumBlocksLocked) = 0 then
          break;
        Sleep(AdditionalSleepTime);
      end;
    end;

Another improvement to the assembly versions (which gets used by default) of these spin-locks is usage of the pause x86 instruction to be more processor and OS-friendly:

asm
  //...
  {The pause instruction improves spinlock performance}
  pause 
  //...
end;

In addition the FastMM code size has been slightly reduced (I haven't measured) by using the equivalent of a method-extract refactoring on some of the assembly code (the logic for resizing a large memory block has been refactored into an internal ReallocateLargeBlock routine). 

Finally, it looks like there is some improved logic to support segmented large blocks. Pierre's comment in the code about this is:

{Is this large block segmented? I.e. is it actually built up from more than
one chunk allocated through VirtualAlloc? (Only used by large blocks.)} 

Optimized RTL routines

As Steve Threfethen has already blogged about there are now updated and new FastCode replacement functions included in the stock RTL. The compiler magic System routine _LStrCmp is used whenever you compare strings, so it is a performance critical piece of code. Pierre le Richie wrote a new, faster version of this routine (based on the FastCode winner of the CompareStr challenge) and submitted it to Quality Central #31328. This routine has now been incorporated into the RTL.

Note that _LStrCmp is a general routine that is called for all string compares, including the string compare operators >, >=, =, <>, < and <=. In Pierre's QC entry he suggests changing the compiler so that it calls a different, more specific function (_LStrEqual) - in most cases, it would be able to determine non-equalness simply by comparing the strings' lengths. As this change would probably break binary .dcu compatibility, it is not included in Spacely, but it is a possibility for Highlander (that will probably become BDS 2007).

The other FastCode replacement functions reside in SysUtils. The UpperCase and LowerCase were already FastCode functions (by Aleksandr Sharahov) in BDS 2006 - there are now even faster versions by John O'Harrow in Delphi 2007. In addition CompareStr and StrLen are new FastCode functions from Pierre le Riche.

In addition the FileAge and FileSearch functions have been optimized. FileAge now calls the more efficient GetFileAttributesExA API (when available) instead of the FindFirst/FindClose pair. FileSearch now leaves early if the Name parameter is an empty string. These two changes also helps improve the speed of the compiler and linker.

Finally the hash-table logic used in the CheckForDuplicateUnits routine during loading of packages has been further optimized. This code was greatly improved in BDS2006 and it is now even faster. This helps reduce the startup time of applications that loads lots of packages dynamically (such as the Delphi IDE itself).

Desktop Window Manager API

There is a new DwmApi unit (in the source\Win32\rtl\win directory) that dynamically binds to the Windows Vista specific APIs exported by DWMAPI.DLL. On other platforms the routines are stubbed out to return the E_NOTIMPL error code. The constants and routines in this unit is the basis of all the new Windows Vista and Glass specific functionality added to the VCL and Delphi IDE. It has exiting sounding routine names like DwmExtendFrameIntoClientArea and DwmUpdateThumbnailProperties. Most of the time the VCL shields you from using these nitty-gritty APIs directly, but its handy to know where to find them.

Math unit and complete boolean evaluation

The math unit has had a couple of interesting changes to the trio (Integer, Int64 and Double versions) of the InRange functions. Basically, the code has been rearranged from (BDS 2006 version):

function InRange(const AValue, AMin, AMax: Integer): Boolean;
begin
  Result := (AValue >= AMin) and (AValue <= AMax);
end;

to (Delphi 2007 version):

function InRange(const AValue, AMin, AMax: Integer): Boolean;
var
  A,B: Boolean;
begin
  A := (AValue >= AMin);
  B := (AValue <= AMax);
  Result := B and A;
end;

The code changes forces the compiler to evaluate both boolean expressions before setting the Result variable. This reduces the number of conditional jumps and thus typically improves performance (particularly in the cases where InRange returns true - then it has to evaluate both expressions anyway). The generated code differences are as follows - here is the assembly code generated for the old BDS 2006 version of InRange:

Result := (AValue >= AMin) and (AValue <= AMax);
//0040841C 3BD0             cmp edx,eax
//0040841E 7F04             jnle $00408424
//00408420 3BC8             cmp ecx,eax
//00408422 7D03             jnl $00408426
//00408424 33C0             xor eax,eax
//00408426 C3               ret

and here is the assembly code for the new version:

  A := (AValue >= AMin);
//0040842C 3BD0             cmp edx,eax
//0040842E 0F9EC2           setle dl
  B := (AValue <= AMax);
//00408431 3BC8             cmp ecx,eax
//00408433 0F9DC0           setnl al
  Result := B and A;
//00408436 22C2             and al,dl
end;
//00408438 C3               ret

As you can see the second version does not have any branches, while the first one has two conditional branch instructions. Modern processors work better with branchless code, so the net effect is better performance. We might revisit this topic in the future - and the alternative way of accomplishing the same thing - using the $B+ compiler directive to enable complete boolean evaluation.

That concludes part two of this review - part three will follow shortly. Stay tuned! ;)

Review: Delphi 2007 for Win32 (Beta) - part one

Nick Hodges of CodeGear contacted me and gave me permission to talk about the upcoming Delphi 2007 for Win32 product - codenamed Spacely. Note that this review is based on a pre-release Beta build 2063 from mid February. Anything you see here is subject to change in the release version.

The installer

Here are a few screenshots of how the new installer looks like - click them to see them in full size. The installer has been created with InstallAware (which is also bundled with Delphi 2007) and it should allow CodeGear to release patches and potentially new versions in an easier and more integrated way.

 

 

Delphi 2007 is a non-breaking version 

What does it mean that Delphi 2007 is a non-breaking version? Well, according to CodeGear roadmaps there will be a new BDS (or CDS?) version later this year code named Highlander. To reduce the pain (for developers, 3rd party component providers etc) of having two new full Delphi versions in less than a year, CodeGear decided to release Delphi 2007 as a non-breaking version. The result is that you should be able to use code and components designed for and compiled in BDS 2006 or Turbo Delphi 2006 without even recompiling any .dcu (delphi compiled unit) files. So even .dcu only shareware components should continue to just work.

Delphi 2007 is special in that it is (mostly) a non-breaking version in relation to BDS 2006 and Turbo Delphi 2006. This means that it is binary compatible at the .dcu level and no breaking changes has been made to the interface of any units. This means that there are no changes to existing classes or routines, for instance. Note that the Delphi unit interface checks are granular enough to allow new classes and identifiers to be added, as long as no existing identifiers are modified in any way.

A side-effect is that while it is relatively easy to fix RTL and VCL bugs (that only need changes in the implementation section), and to introduce new classes and components, it is very hard to introduce new functionality, methods and properties on existing classes. Hard, but not impossible, of course. CodeGear has not been shy to apply the odd hack or two to add native support for extended Vista Glass functionality in the TForm class, for instance. Allen Bauer has already spilled the beans on this and how it was achieved. We will take a closer look at the RTL and VCL changes in Delphi 2007 (relative to BDS 2006) further down in the review.

What's new in the compiler?

It also means that the compiler has not been updated with new features that would break binary compatibility, so there are no generics support yet, for instance. However, a little investigation shows that there have been improvements to the compiler. For instance, there are a slew of new compiler hints, warnings and errors. It also looks like the compiler has spiffed up its XML documentation generation capabilities. Some of the new error messages are probably due to bug fixes. Code that would earlier generate internal errors now generate clean compile-time errors with explanations of what is wrong with the code. Here is an example list of the new hints, warnings and errors that I've identified:

New Hints:

• H2445 Inline function '%s' has not been expanded because its unit '%s' is specified in USES statement of IMPLEMENTATION section and current function is inline function or being inline function
  
• H2451 Narrowing given WideChar constant (#$%04X) to AnsiChar lost information
  
• H2456 Inline function '%s' has not been expanded because contained unit '%s' uses compiling unit '%s'

New Warnings:

• W1055 PUBLISHED caused RTTI ($M+) to be added to type '%s'
  
• W1201 XML comment on '%s' has badly formed XML -- 'Whitespace is not allowed at this location.'
  
• W1202 XML comment on '%s' has badly formed XML -- 'Reference to undefined entity '%s'.'
  
• W1203 XML comment on '%s' has badly formed XML -- 'A name was started with an invalid character.'
  
• W1204 XML comment on '%s' has badly formed XML -- 'A name contained an invalid character.'
  
• W1205 XML comment on '%s' has badly formed XML -- 'The character '%c' was expected.'
  
• W1206 XML comment on '%s' has cref attribute '%s' that could not be resolved
  
• W1207 XML comment on '%s' has a param tag for '%s', but there is no parameter by that name
  
• W1208 Parameter '%s' has no matching param tag in the XML comment for '%s' (but other parameters do)

New Errors:

• E2447 Duplicate symbol '%s' defined in namespace '%s' by '%s' and '%s'
  
• E2448 An attribute argument must be a constant expression, typeof expression or array constructor
  
• E2449 Inlined nested routine '%s' cannot access outer scope variable '%s'
  
• E2450 There is no overloaded version of array property '%s' that can be used with these arguments

Linker errors:

• F2446 Unit '%s' is compiled with unit '%s' in '%s' but different version '%s' found

What's new in the IDE

In general the IDE feels very fast and nice to work with - it has noticeably less flickering when switching between debug and design layouts, for instance.

Here are a couple of screen shots showing some of the news in the IDE.

I've marked the changes with numbers:

1. I've added a new toolbar button to toggle a debugger setting called Notify on Language Exceptions
   
2. The new TForm property GlassFrame to control the Vista Glass effect on the form's area. This is achieved cunningly with class helpers, property injectors and some extra hacks.
   
3. The Tool Palette has improved partial search matching. Any component that contains the string you're typing will be included in the list. So starting to type "but" will show all button components. This is great!

Here is a another screen shot.

1. The new File Browser view is very handy. It works like a mini Explorer, reducing the need to use an external Explorer or File | Open all the time. And of course, this view can be docked anywhere - or you can keep it floating, or unpin it so that it scrolls away when you're not using it.
   
2. The project's options for Compiler, Compiler Warnings, Linker and Directories/Conditionals can now be set separately for a Release and Debug configurations. In addition you can create more named configurations. Finally!
   
3. Here you can again see the new compiler warnings that can be toggled on or off
   
4. I added a Windows Vista specific component called TFileOpenDialog - the compiler issues a platform warning for this. Nice.

This review will be updated with more information and details in the coming few days. Stay tuned ;)