Review: Delphi 2007 for Win32 (Beta) - part three

Read the part one and two first.

What's new in the VCL

There are quite a few bugfixes in the VCL (as well as the RTL), but we'll not go into them in detail here. While remaining binary .dcu compatible, CodeGear has managed the feat of adding new functionality and even new properties on the existing TCustomForm class.

The GlassFrame property hack

In particular, all forms now have a GlassFrame property with sub-properties to control extended glass functionality when running on Windows Vista. Allen Bauer has blogged about this in his post "How to add a "published" property without breaking DCU compatibility". Please read his blog entry first - that will make it easier to understand the rest of this section.

First of all, what does the GlassFrame property contain and what does it do? Let's pretend that it had been implemented the "proper" way by adding a normal property to TCustomForm that is promoted to published in TForm. Then the code changes would look like this:

type
  TCustomForm = class;
  TGlassFrame = class(TPersistent)
  public
    constructor Create(Client: TCustomForm);
    procedure Assign(Source: TPersistent); override;
    function FrameExtended: Boolean;
    function IntersectsControl(Control: TControl): Boolean;
    property OnChange: TNotifyEvent {...};
  published
    property Enabled: Boolean {...} default False;
    property Left: Integer {...} default 0;
    property Top: Integer {...} default 0;
    property Right: Integer {...} default 0;
    property Bottom: Integer {...} default 0;
    property SheetOfGlass: Boolean {...} default False;
  end;
  TCustomForm = class(TScrollingWinControl)
  public
    procedure UpdateGlassFrame(Sender: TObject);
    property GlassFrame: TGlassFrame {...} ;
  end;
  TForm = class(TCustomForm)
  published
    property GlassFrame;
  end;

For simplicity I've left out the private and protected implementation details. Note that the new GlassFrame property has the type TGlassFrame which is a TPersistent descendent. This means that its properties will appear in the Object Inspector as sub-properties of GlassFrame. The Left, Top, Right and Bottom properties defines how large the border of glass around the non-transparent part of the form should be. If you set SheetOfGlass to True, the entire form will be glass. Finally the Enabled property can be used to quickly turn the extra glass effect on or off. Note that the glass effect controlled by the GlassFrame property is in addition to the normal glassed non-client frame (caption and window border) provided by Vista. I don't have Vista running on my low-powered Acer laptop here, so you can take a peek at the screenshots over in Jeremy North's blogpost about the glass effect here and here to see how it looks at design-time and run-time.

The challenge that faced CodeGear in implementing the GlassFrame property and functionality in the non-breaking Delphi 2007 version was four-fold.

• The GlassFrame property needs to be available at runtime and appear to compiling code to reside on all form instances. This is solved using a class helper called TCustomFormHelper.
  
• Class helpers cannot add new fields or per-instance storage. Somehow the storage needs of the GlassFrame property needs to be satisfied. This is done by using a hack and reusing one of the existing TCustomForm private fields (FPixelsPerInch)
  
• While class helpers are great for creating a runtime mirage effect, "fooling" your code to see an injected property on an existing class, they do not help with RTTI and thus getting the property into the Object Inspector. As Allen explained in his article, in BDS 2006 they introduced a new selection editor interface called ISelectionPropertyFilter that makes it possible to dynamically add and remove design-time properties.
  
• Finally, the GlassFrame property needs to be streamed to and from the .dfm storage. This is not solved by the class helper alone, nor the property filter interface. The solution is to use the existing DefineProperties mechanism, but an extra twist is needed to support Property.SubProperty names for defined properties.

Lets dive into each of these issues in more detail. After all, this blog is mostly about hacks and the Delphi 2007 GlassFrame feature is arguably the most high-profile and best (?) hack in the VCL ever :-). Note: The source code I show here is for illustration only and is based on a beta build of Delphi 2007 - check your own copy of Forms.pas in the shipping version.

The CustomFormHelper class helper

Here is what the class helper that provides the runtime GlassFrame property looks like:

  TCustomFormHelper = class helper for TCustomForm
  private
    function GetGlassFrame: TGlassFrame;
    procedure ReadGlassFrameBottom(Reader: TReader);
    procedure ReadGlassFrameEnabled(Reader: TReader);
    procedure ReadGlassFrameLeft(Reader: TReader);
    procedure ReadGlassFrameRight(Reader: TReader);
    procedure ReadGlassFrameSheetOfGlass(Reader: TReader);
    procedure ReadGlassFrameTop(Reader: TReader);
    procedure SetGlassFrame(const Value: TGlassFrame);
    procedure WriteGlassFrameBottom(Writer: TWriter);
    procedure WriteGlassFrameEnabled(Writer: TWriter);
    procedure WriteGlassFrameLeft(Writer: TWriter);
    procedure WriteGlassFrameRight(Writer: TWriter);
    procedure WriteGlassFrameSheetOfGlass(Writer: TWriter);
    procedure WriteGlassFrameTop(Writer: TWriter);
  public
    procedure UpdateGlassFrame(Sender: TObject);
    property GlassFrame: TGlassFrame read GetGlassFrame 
      write SetGlassFrame;
  end;

This provides the public GlassFrame property that is transposed onto TCustomForm and all descendants. It also makes the UpdateGlassFrame method available, but this is mostly used internally in the Forms unit. It is the target of the OnChange event defined in the TGlassFrame class, forcing the form to repaint itself whenever one of the GlassFrame properties changes. Finally there are the ReadXXX and WriteXXX methods used in the TCustomForm.DefineProperties method to stream the GlassFrame properties to and from .dfm files. We'll discuss this in more detail below.

This is a virtual method defined on TPersistent. Luckily, TCustomForm already overrode this method in BDS 2006 (to store the pseudo properties PixelsPerInch, TextHeight and IgnoreFontProperty) so it was simple to add the new GlassFrame properties there.

The FPixelsPerInch storage hack 

As you can see above the class helper does not have (and cannot have) and instance fields. Still the GlassFrame instance pointer has to be stored somewhere - and it needs to be stored per form instance. There are several different potential solutions to this. One possibility is to use some kind of hash-table to map form instances into corresponding GlassFrame instances, but this is complex, could be relatively slow and require extra coordination to make sure the GlassFrame instance is freed when the form is freed etc. The other solution is to stash the information into one of the existing fields of TCustomForm.

This is what CodeGear decided to do. They picked a relatively seldom used private field that does not have its address exposed via property accessors (you can read about why this would have been dangerous here).

type
  TCustomForm = class(TScrollingWinControl)
  private
    FPixelsPerInch: Integer;
  end;

The FPixelsPerInch field is declared as Integer, but in the implementation section it is actually treated as a pointer to a record structure.

{ Hack to overlay GlassFrame on PixelsPerInch in TCustomForm }
type
  PPixelsPerInchOverload = ^TPixelsPerInchOverload;
  TPixelsPerInchOverload = record
    PixelsPerInch: Integer;
    GlassFrame: TGlassFrame;
    RefreshGlassFrame: Boolean;
  end;

The record gives storage for both the PixelsPerInch property (declared on TCustomForm) the new GlassFrame property (injected by TCustomFormHelper) and another private implementation field called RefreshGlassFrame.

The TCustomForm constructor and destructor allocate and deallocate the FPixelsPerInch field as a TPixelsPerInchOverload pointer.

constructor TCustomForm.CreateNew(AOwner: TComponent; Dummy: Integer);
begin
  Pointer(FPixelsPerInch) := AllocMem(SizeOf(TPixelsPerInchOverload));
  inherited Create(AOwner);
  //...
end;

destructor TCustomForm.Destroy;
begin
  //...
    FreeMem(Pointer(FPixelsPerInch));
    inherited Destroy;
  //...
end;

All access to the PixelsPerInch and RefreshGlassFrame fields in the TPixelsPerInchOverload record are delegated to a set of inlined getter and setter routines, such as this one:

function GetFPixelsPerInch(FPixelsPerInch: Integer): Integer; inline;
begin
  Result := PPixelsPerInchOverload(FPixelsPerInch).PixelsPerInch;
end;

Finally the class helper methods can now use the same trick to get and store the GlassFrame property:

function TCustomFormHelper.GetGlassFrame: TGlassFrame;
begin
  Result := PPixelsPerInchOverload(FPixelsPerInch).GlassFrame;
end;

procedure TCustomFormHelper.SetGlassFrame(const Value: TGlassFrame);
begin
  PPixelsPerInchOverload(FPixelsPerInch).GlassFrame.Assign(Value);
end;

Neat hack, no? ;)

Injecting design-time properties

The next step is to convince the Object Inspector to make the GlassFrame compound property available for inspection and editing at design-time. This is achieved using a little known component selection interface called ISelectionPropertyFilter. This interface was first introduced in Delphi 2006 and used to help implement the ControlIndex property injected into all components dropped on a TFlowPanel or TGridPanel. These components are documented in a BDN article by Ed Vander Hoek here and the ISelectionPropertyFilter interface and how it is used is discussed by Tjipke A. van der Plaats here.

The interface is declared in the DesignIntf unit and looks like this:

{ ISelectionPropertyFilter
This optional interface is implemented on the same class that implements
ISelectionEditor.  If this interface is implemented, when the property list
is constructed for a given selection, it is also passed through all the various
implementations of this interface on the selected selection editors.  From here
the list of properties can be modified to add or remove properties from the list.
If properties are added, then it is the responsibility of the implementor to
properly construct an appropriate implementation of the IProperty interface.
Since an added "property" will typically *not* be available via the normal RTTI
mechanisms, it is the implementor's responsibility to make sure that the property
editor overrides those methods that would normally access the RTTI for the
selected objects.

FilterProperties
Once the list of properties has been gathered and before they are sent to the
Object Inspector, this method is called with the list of properties.  You may
manupulate this list in any way you see fit, however, remember that another
selection editor *may* have already modified the list.  You are not guaranteed
to have the original list. }
  ISelectionPropertyFilter = interface
    ['{0B424EF6-2F2F-41AB-A082-831292FA91A5}']
    procedure FilterProperties(const ASelection: IDesignerSelections;
      const ASelectionProperties: IInterfaceList);
  end;

There is no shipping source for it, but one of the IDE packages registers a selection editor for TCustomForm (and thus all descendants) using the DesignIntf.RegisterSelectionEditor routine.

type
{ TBaseSelectionEditor
  All selection editors are assumed to derive from this class. A default
  implemenation for the ISelectionEditor interface is provided in
  TSelectionEditor class. }
  TBaseSelectionEditor = class(TInterfacedObject)
  public
    constructor Create(const ADesigner: IDesigner); virtual;
  end;

  TSelectionEditorClass = class of TBaseSelectionEditor;

procedure RegisterSelectionEditor(AClass: TClass; AEditor: TSelectionEditorClass);

You can see some of the details of how the ISelectionPropertyFilter is used in the DesignEditors unit and the GetComponentProperties routine. This routine is called by the IDE when you select a form and it gets the list of registered selection editors for forms and each of the selection editor classes that implement the ISelectionPropertyFilter interface have their FilterProperties method called. This allows it to remove or add properties to the list that is eventually presented to the user in the Object Inspector. The new TCustomForm selection editor adds an implementation of IProperty for the new ghost property GlassFrame. The net result is that as far as the Object Inspector is concerned, the GlassFrame looks like a published property on TCustomForm, even though it isn't actually there and there is no RTTI for it. Feels like magic! ;).

Defining the streaming properties

The final piece of the puzzle to make the GlassFrame property illusion complete is to gel with the .dfm streaming system. It would help much having the GlassFrame property in the Object Inspector if the values you set didn't persist between runs of the IDE or when loading the form at runtime. The virtual DefineProperties method on TPersistent has always been part of the Delphi streaming system. You override it to store additional information than the published properties. Luckily TCustomForm already overrides DefineProperties (to store PixelsPerInch, TextHeight and IgnoreFontProperty). This means that additional code can be added to store the GlassFrame property without changing the interfaced part of TCustomForm. Here is the new DefineProperties method.

procedure TCustomForm.DefineProperties(Filer: TFiler);
//...
begin
  inherited DefineProperties(Filer);
  Filer.DefineProperty('PixelsPerInch', {...});
  Filer.DefineProperty('TextHeight', {...});
  Filer.DefineProperty('IgnoreFontProperty', {...});
  Filer.DefineProperty('GlassFrame.Bottom', {...});
  Filer.DefineProperty('GlassFrame.Enabled', {...});
  Filer.DefineProperty('GlassFrame.Left', {...});
  Filer.DefineProperty('GlassFrame.Right', {...});
  Filer.DefineProperty('GlassFrame.SheetOfGlass', {...});
  Filer.DefineProperty('GlassFrame.Top', {...});
end;

I have commented out the details about the ReadData, WriteData and HasData parameters of each DefineProperty call. The actual reading and writing has been delegated to the private Read and Write methods of the TCustomFormHelper that we listed at the top of this article.

The special thing to note here is that the GlassFrame properties are stored using a nested 'GlassFrame.SubProperty' name. In earlier versions of Delphi and the streaming subsystem this would not have worked, but there is now extra code in the Classes unit that makes this possible.

procedure TReader.ReadProperty(AInstance: TPersistent);
//...
        PropInfo := GetPropInfo(Instance.ClassInfo, FPropName);
        if PropInfo = nil then
        begin
          // Call DefineProperties with the entire PropPath
          // to allow defining properties such as "Prop.SubProp"
          FPropName := PropPath;
          { Cannot reliably recover from an error in a defined property }
          FCanHandleExcepts := False;
          Instance.DefineProperties(Self);
          FCanHandleExcepts := True;
          if FPropName <> '' then
            PropertyError(FPropName);
          Exit;
        end;

The reason for doing it this way is that this is an important aspect in the future plans for the GlassFrame implementation.

A GlassFrame into the future

In future binary-breaking releases of Delphi (aka Highlander or BDS (CDS?) 2007) the GlassFrame implementation will be folded into the proper classes. Allen talks about this too. So most probably, GlassFrame will become a proper property on TCustomForm (and promoted to published in TForm), the TCustomFormHelper and the tricks discussed above will disappear. The neat thing, though, is that all code and .dfms using it will just continue to work. The 'GlassFrame.Subproperty' names of the defined properties will map directly to the nested properties of the GlassFrame instance. So while the inner workings will change, the externally observable behavior will stay the same. Impressive, don't you think?!

Other VCL changes

We running out of time and space (at least metaphorically), so we'll just quickly mention the other main changes to the VCL in Delphi 2007. Applications built with Delphi has normally been very easy to spot, because the taskbar icon belongs to a special hidden TApplicaton.Handle instead of the actual main form, so the system menu has been a little short ;). This behavior has now been made optional - old applications should have the old behavior while new projects now set a new Application. MainFormOnTaskBar property to True. This ensures that the icon in the taskbar actually belongs to the main form, instead of the application handle. This will fix a few issues in general and on Vista in particular. The new property is again temporarily provided by a class helper called TApplicationHelper, but this time without the extra design-time and streaming shenanigans. This class helper also injects a somewhat cryptically named EnumAllWindowsOnActivateHint property that as far as I can see provides a fix for showing hints for windows that belongs to the process, but was created in another thread.

type
  TApplicationHelper = class helper for TApplication
  public
    property EnumAllWindowsOnActivateHint: Boolean {...};
    property MainFormOnTaskBar: Boolean {...};
  end;

A lot of the other changes in VCL are to properly support themeing and painting correctly with a glass form background, but at design-time and run-time. There are new Vista specific components in the Dialogs unit named TFileOpenDialog, TFileSaveDialog and TTaskDialog. These are all marked with the platform directive, so if you use them directly, they will only work in Windows Vista.

In addition there is a global UseLatestCommonDialogs variable that when set to True (the default) will automatically upgrade the good old TOpenDialog, TSaveDialog and MessageDlg into using the new Vista GUI look when available. Primoz Gabrijelcic has a nice post and good screenshots of these dialogs here. 

The other new features

I'm not a database guy, but the database express architecture has been updated to generation 4 (DBX 4). This involves potential performance improvements, single sourcing database code for both native and managed code, (some) drivers with source code, all Delphi code, backward compatibility with DBX 3 drivers, connection pooling and more. Looks very impressive. The returned CodeGear database guru Steve Shaughnessy knows more about this stuff than anyone else - get the details here. An overview of the database architecture classes (generated using Together) can be seen here in CodeGear's Andreano Lanusse's blog.

Delphi 2007 ships with a new version of IntraWeb that CodeGear dubs VCL for the Web and it includes Ajax functionality. The native Win32 SOAP and WebServices support has had many bugfixes and performance improvements and is more capable than ever. As we noted in the screenshots of the first article in this series, MS Build is now used as the build engine allowing you to customize the build process with pre-compile and post-compile events. And it supports multiple build configurations for debug and release builds, for instance.

The debugger is an invaluable tool for experienced developers and now it is better than ever. I'll probably blog about it in more detail later. Often people use only a fraction of the useful feature in the debugger, mostly because they don't know about them or how to use them. The Call Stack has a number of improvements. There are glyphs indicating if a stack frame has debug info available or not. You can now set break points directly in the Call Stack - the debugger will break when control returns to that point. When you double click an entry in the Call Stack it will now show the locals in that frame in addition to navigating to the correct spot in the source code. In the default keyboard mapping, you can now press Shift+F5 to enable or disable a breakpoint on the current line (ah - nice to reduce those mouse operations!).

Phew! Hope you have enjoyed this little series on the Delphi 2007 beta.

Conclusion

It feels like Delphi 2007 is going to be a rock-solid release. Here is a summary of my conclusions of who Delphi 2007 is for:

• If you're developing on or for Windows Vista, you'll want to get Delphi 2007.
  
• If you're already on BDS 2006 and plan to upgrade to BDS 2007, you should seriously consider buying Delphi 2007 with SA (Software Assurance) or subscription now - that will (by all likelihood) give you BDS 2007 later this year. The transition from Win32 development on BDS 2006 should be very smooth - just keep using the same components without recompiling them. No need to wait for new versions from 3rd parties.
  
• If you are a Delphi 5-7 developer that is still sitting on the fence, this is the time to jump and take this offer. The IDE is much more capable and productive and the performance and flicker-free operation is on par with or better than Delphi 7 now. You can see the list of improvements between Delphi 7 and BDS 20006 in Nick Hodges blog here - all of these goodies (on the native Win32 side) are also part of Delphi 2007, of course.

I highly recommend Delphi 2007 Win32 for all native Delphi developers!