Sunday, March 11, 2007

Review: Delphi 2007 for Win32 (Beta) - part three

Read the part one and two first.

What's new in the VCL

There are quite a few bugfixes in the VCL (as well as the RTL), but we'll not go into them in detail here. While remaining binary .dcu compatible, CodeGear has managed the feat of adding new functionality and even new properties on the existing TCustomForm class.

The GlassFrame property hack

In particular, all forms now have a GlassFrame property with sub-properties to control extended glass functionality when running on Windows Vista. Allen Bauer has blogged about this in his post "How to add a "published" property without breaking DCU compatibility". Please read his blog entry first - that will make it easier to understand the rest of this section.

First of all, what does the GlassFrame property contain and what does it do? Let's pretend that it had been implemented the "proper" way by adding a normal property to TCustomForm that is promoted to published in TForm. Then the code changes would look like this:

TCustomForm = class;
TGlassFrame = class(TPersistent)
constructor Create(Client: TCustomForm);
procedure Assign(Source: TPersistent); override;
function FrameExtended: Boolean;
function IntersectsControl(Control: TControl): Boolean;
property OnChange: TNotifyEvent {...};
property Enabled: Boolean {...} default False;
property Left: Integer {...} default 0;
property Top: Integer {...} default 0;
property Right: Integer {...} default 0;
property Bottom: Integer {...} default 0;
property SheetOfGlass: Boolean {...} default False;
TCustomForm = class(TScrollingWinControl)
procedure UpdateGlassFrame(Sender: TObject);
property GlassFrame: TGlassFrame {...} ;
TForm = class(TCustomForm)
property GlassFrame;

For simplicity I've left out the private and protected implementation details. Note that the new GlassFrame property has the type TGlassFrame which is a TPersistent descendent. This means that its properties will appear in the Object Inspector as sub-properties of GlassFrame. The Left, Top, Right and Bottom properties defines how large the border of glass around the non-transparent part of the form should be. If you set SheetOfGlass to True, the entire form will be glass. Finally the Enabled property can be used to quickly turn the extra glass effect on or off. Note that the glass effect controlled by the GlassFrame property is in addition to the normal glassed non-client frame (caption and window border) provided by Vista. I don't have Vista running on my low-powered Acer laptop here, so you can take a peek at the screenshots over in Jeremy North's blogpost about the glass effect here and here to see how it looks at design-time and run-time.

The challenge that faced CodeGear in implementing the GlassFrame property and functionality in the non-breaking Delphi 2007 version was four-fold.

  • The GlassFrame property needs to be available at runtime and appear to compiling code to reside on all form instances. This is solved using a class helper called TCustomFormHelper.
  • Class helpers cannot add new fields or per-instance storage. Somehow the storage needs of the GlassFrame property needs to be satisfied. This is done by using a hack and reusing one of the existing TCustomForm private fields (FPixelsPerInch)
  • While class helpers are great for creating a runtime mirage effect, "fooling" your code to see an injected property on an existing class, they do not help with RTTI and thus getting the property into the Object Inspector. As Allen explained in his article, in BDS 2006 they introduced a new selection editor interface called ISelectionPropertyFilter that makes it possible to dynamically add and remove design-time properties.
  • Finally, the GlassFrame property needs to be streamed to and from the .dfm storage. This is not solved by the class helper alone, nor the property filter interface. The solution is to use the existing DefineProperties mechanism, but an extra twist is needed to support Property.SubProperty names for defined properties.

Lets dive into each of these issues in more detail. After all, this blog is mostly about hacks and the Delphi 2007 GlassFrame feature is arguably the most high-profile and best (?) hack in the VCL ever :-). Note: The source code I show here is for illustration only and is based on a beta build of Delphi 2007 - check your own copy of Forms.pas in the shipping version.

The CustomFormHelper class helper

Here is what the class helper that provides the runtime GlassFrame property looks like:

  TCustomFormHelper = class helper for TCustomForm
function GetGlassFrame: TGlassFrame;
procedure ReadGlassFrameBottom(Reader: TReader);
procedure ReadGlassFrameEnabled(Reader: TReader);
procedure ReadGlassFrameLeft(Reader: TReader);
procedure ReadGlassFrameRight(Reader: TReader);
procedure ReadGlassFrameSheetOfGlass(Reader: TReader);
procedure ReadGlassFrameTop(Reader: TReader);
procedure SetGlassFrame(const Value: TGlassFrame);
procedure WriteGlassFrameBottom(Writer: TWriter);
procedure WriteGlassFrameEnabled(Writer: TWriter);
procedure WriteGlassFrameLeft(Writer: TWriter);
procedure WriteGlassFrameRight(Writer: TWriter);
procedure WriteGlassFrameSheetOfGlass(Writer: TWriter);
procedure WriteGlassFrameTop(Writer: TWriter);
procedure UpdateGlassFrame(Sender: TObject);
property GlassFrame: TGlassFrame read GetGlassFrame
write SetGlassFrame;

This provides the public GlassFrame property that is transposed onto TCustomForm and all descendants. It also makes the UpdateGlassFrame method available, but this is mostly used internally in the Forms unit. It is the target of the OnChange event defined in the TGlassFrame class, forcing the form to repaint itself whenever one of the GlassFrame properties changes. Finally there are the ReadXXX and WriteXXX methods used in the TCustomForm.DefineProperties method to stream the GlassFrame properties to and from .dfm files. We'll discuss this in more detail below.

This is a virtual method defined on TPersistent. Luckily, TCustomForm already overrode this method in BDS 2006 (to store the pseudo properties PixelsPerInch, TextHeight and IgnoreFontProperty) so it was simple to add the new GlassFrame properties there.

The FPixelsPerInch storage hack 

As you can see above the class helper does not have (and cannot have) and instance fields. Still the GlassFrame instance pointer has to be stored somewhere - and it needs to be stored per form instance. There are several different potential solutions to this. One possibility is to use some kind of hash-table to map form instances into corresponding GlassFrame instances, but this is complex, could be relatively slow and require extra coordination to make sure the GlassFrame instance is freed when the form is freed etc. The other solution is to stash the information into one of the existing fields of TCustomForm.

This is what CodeGear decided to do. They picked a relatively seldom used private field that does not have its address exposed via property accessors (you can read about why this would have been dangerous here).

TCustomForm = class(TScrollingWinControl)
FPixelsPerInch: Integer;

The FPixelsPerInch field is declared as Integer, but in the implementation section it is actually treated as a pointer to a record structure.

{ Hack to overlay GlassFrame on PixelsPerInch in TCustomForm }
PPixelsPerInchOverload = ^TPixelsPerInchOverload;
TPixelsPerInchOverload = record
PixelsPerInch: Integer;
GlassFrame: TGlassFrame;
RefreshGlassFrame: Boolean;

The record gives storage for both the PixelsPerInch property (declared on TCustomForm) the new GlassFrame property (injected by TCustomFormHelper) and another private implementation field called RefreshGlassFrame.

The TCustomForm constructor and destructor allocate and deallocate the FPixelsPerInch field as a TPixelsPerInchOverload pointer.

constructor TCustomForm.CreateNew(AOwner: TComponent; Dummy: Integer);
Pointer(FPixelsPerInch) := AllocMem(SizeOf(TPixelsPerInchOverload));
inherited Create(AOwner);

destructor TCustomForm.Destroy;
inherited Destroy;

All access to the PixelsPerInch and RefreshGlassFrame fields in the TPixelsPerInchOverload record are delegated to a set of inlined getter and setter routines, such as this one:

function GetFPixelsPerInch(FPixelsPerInch: Integer): Integer; inline;
Result := PPixelsPerInchOverload(FPixelsPerInch).PixelsPerInch;

Finally the class helper methods can now use the same trick to get and store the GlassFrame property:

function TCustomFormHelper.GetGlassFrame: TGlassFrame;
Result := PPixelsPerInchOverload(FPixelsPerInch).GlassFrame;

procedure TCustomFormHelper.SetGlassFrame(const Value: TGlassFrame);

Neat hack, no? ;)

Injecting design-time properties

The next step is to convince the Object Inspector to make the GlassFrame compound property available for inspection and editing at design-time. This is achieved using a little known component selection interface called ISelectionPropertyFilter. This interface was first introduced in Delphi 2006 and used to help implement the ControlIndex property injected into all components dropped on a TFlowPanel or TGridPanel. These components are documented in a BDN article by Ed Vander Hoek here and the ISelectionPropertyFilter interface and how it is used is discussed by Tjipke A. van der Plaats here.

The interface is declared in the DesignIntf unit and looks like this:

{ ISelectionPropertyFilter
This optional interface is implemented on the same class that implements
ISelectionEditor. If this interface is implemented, when the property list
is constructed for a given selection, it is also passed through all the various
implementations of this interface on the selected selection editors. From here
the list of properties can be modified to add or remove properties from the list.
If properties are added, then it is the responsibility of the implementor to
properly construct an appropriate implementation of the IProperty interface.
Since an added "property" will typically *not* be available via the normal RTTI
mechanisms, it is the implementor's responsibility to make sure that the property
editor overrides those methods that would normally access the RTTI for the
selected objects.

Once the list of properties has been gathered and before they are sent to the
Object Inspector, this method is called with the list of properties. You may
manupulate this list in any way you see fit, however, remember that another
selection editor *may* have already modified the list. You are not guaranteed
to have the original list. }
ISelectionPropertyFilter = interface
procedure FilterProperties(const ASelection: IDesignerSelections;
const ASelectionProperties: IInterfaceList);

There is no shipping source for it, but one of the IDE packages registers a selection editor for TCustomForm (and thus all descendants) using the DesignIntf.RegisterSelectionEditor routine.

{ TBaseSelectionEditor
All selection editors are assumed to derive from this class. A default
implemenation for the ISelectionEditor interface is provided in
TSelectionEditor class. }
TBaseSelectionEditor = class(TInterfacedObject)
constructor Create(const ADesigner: IDesigner); virtual;

TSelectionEditorClass = class of TBaseSelectionEditor;

procedure RegisterSelectionEditor(AClass: TClass; AEditor: TSelectionEditorClass);

You can see some of the details of how the ISelectionPropertyFilter is used in the DesignEditors unit and the GetComponentProperties routine. This routine is called by the IDE when you select a form and it gets the list of registered selection editors for forms and each of the selection editor classes that implement the ISelectionPropertyFilter interface have their FilterProperties method called. This allows it to remove or add properties to the list that is eventually presented to the user in the Object Inspector. The new TCustomForm selection editor adds an implementation of IProperty for the new ghost property GlassFrame. The net result is that as far as the Object Inspector is concerned, the GlassFrame looks like a published property on TCustomForm, even though it isn't actually there and there is no RTTI for it. Feels like magic! ;).

Defining the streaming properties

The final piece of the puzzle to make the GlassFrame property illusion complete is to gel with the .dfm streaming system. It would help much having the GlassFrame property in the Object Inspector if the values you set didn't persist between runs of the IDE or when loading the form at runtime. The virtual DefineProperties method on TPersistent has always been part of the Delphi streaming system. You override it to store additional information than the published properties. Luckily TCustomForm already overrides DefineProperties (to store PixelsPerInch, TextHeight and IgnoreFontProperty). This means that additional code can be added to store the GlassFrame property without changing the interfaced part of TCustomForm. Here is the new DefineProperties method.

procedure TCustomForm.DefineProperties(Filer: TFiler);
inherited DefineProperties(Filer);
Filer.DefineProperty('PixelsPerInch', {...});
Filer.DefineProperty('TextHeight', {...});
Filer.DefineProperty('IgnoreFontProperty', {...});
Filer.DefineProperty('GlassFrame.Bottom', {...});
Filer.DefineProperty('GlassFrame.Enabled', {...});
Filer.DefineProperty('GlassFrame.Left', {...});
Filer.DefineProperty('GlassFrame.Right', {...});
Filer.DefineProperty('GlassFrame.SheetOfGlass', {...});
Filer.DefineProperty('GlassFrame.Top', {...});

I have commented out the details about the ReadData, WriteData and HasData parameters of each DefineProperty call. The actual reading and writing has been delegated to the private Read and Write methods of the TCustomFormHelper that we listed at the top of this article.

The special thing to note here is that the GlassFrame properties are stored using a nested 'GlassFrame.SubProperty' name. In earlier versions of Delphi and the streaming subsystem this would not have worked, but there is now extra code in the Classes unit that makes this possible.

procedure TReader.ReadProperty(AInstance: TPersistent);
PropInfo := GetPropInfo(Instance.ClassInfo, FPropName);
if PropInfo = nil then
// Call DefineProperties with the entire PropPath
// to allow defining properties such as "Prop.SubProp"
FPropName := PropPath;
{ Cannot reliably recover from an error in a defined property }
FCanHandleExcepts := False;
FCanHandleExcepts := True;
if FPropName <> '' then

The reason for doing it this way is that this is an important aspect in the future plans for the GlassFrame implementation.

A GlassFrame into the future

In future binary-breaking releases of Delphi (aka Highlander or BDS (CDS?) 2007) the GlassFrame implementation will be folded into the proper classes. Allen talks about this too. So most probably, GlassFrame will become a proper property on TCustomForm (and promoted to published in TForm), the TCustomFormHelper and the tricks discussed above will disappear. The neat thing, though, is that all code and .dfms using it will just continue to work. The 'GlassFrame.Subproperty' names of the defined properties will map directly to the nested properties of the GlassFrame instance. So while the inner workings will change, the externally observable behavior will stay the same. Impressive, don't you think?!

Other VCL changes

We running out of time and space (at least metaphorically), so we'll just quickly mention the other main changes to the VCL in Delphi 2007. Applications built with Delphi has normally been very easy to spot, because the taskbar icon belongs to a special hidden TApplicaton.Handle instead of the actual main form, so the system menu has been a little short ;). This behavior has now been made optional - old applications should have the old behavior while new projects now set a new Application. MainFormOnTaskBar property to True. This ensures that the icon in the taskbar actually belongs to the main form, instead of the application handle. This will fix a few issues in general and on Vista in particular. The new property is again temporarily provided by a class helper called TApplicationHelper, but this time without the extra design-time and streaming shenanigans. This class helper also injects a somewhat cryptically named EnumAllWindowsOnActivateHint property that as far as I can see provides a fix for showing hints for windows that belongs to the process, but was created in another thread.

TApplicationHelper = class helper for TApplication
property EnumAllWindowsOnActivateHint: Boolean {...};
property MainFormOnTaskBar: Boolean {...};

A lot of the other changes in VCL are to properly support themeing and painting correctly with a glass form background, but at design-time and run-time. There are new Vista specific components in the Dialogs unit named TFileOpenDialog, TFileSaveDialog and TTaskDialog. These are all marked with the platform directive, so if you use them directly, they will only work in Windows Vista.

In addition there is a global UseLatestCommonDialogs variable that when set to True (the default) will automatically upgrade the good old TOpenDialog, TSaveDialog and MessageDlg into using the new Vista GUI look when available. Primoz Gabrijelcic has a nice post and good screenshots of these dialogs here

The other new features

I'm not a database guy, but the database express architecture has been updated to generation 4 (DBX 4). This involves potential performance improvements, single sourcing database code for both native and managed code, (some) drivers with source code, all Delphi code, backward compatibility with DBX 3 drivers, connection pooling and more. Looks very impressive. The returned CodeGear database guru Steve Shaughnessy knows more about this stuff than anyone else - get the details here. An overview of the database architecture classes (generated using Together) can be seen here in CodeGear's Andreano Lanusse's blog.

Delphi 2007 ships with a new version of IntraWeb that CodeGear dubs VCL for the Web and it includes Ajax functionality. The native Win32 SOAP and WebServices support has had many bugfixes and performance improvements and is more capable than ever. As we noted in the screenshots of the first article in this series, MS Build is now used as the build engine allowing you to customize the build process with pre-compile and post-compile events. And it supports multiple build configurations for debug and release builds, for instance.

The debugger is an invaluable tool for experienced developers and now it is better than ever. I'll probably blog about it in more detail later. Often people use only a fraction of the useful feature in the debugger, mostly because they don't know about them or how to use them. The Call Stack has a number of improvements. There are glyphs indicating if a stack frame has debug info available or not. You can now set break points directly in the Call Stack - the debugger will break when control returns to that point. When you double click an entry in the Call Stack it will now show the locals in that frame in addition to navigating to the correct spot in the source code. In the default keyboard mapping, you can now press Shift+F5 to enable or disable a breakpoint on the current line (ah - nice to reduce those mouse operations!).

Phew! Hope you have enjoyed this little series on the Delphi 2007 beta.


It feels like Delphi 2007 is going to be a rock-solid release. Here is a summary of my conclusions of who Delphi 2007 is for:

  • If you're developing on or for Windows Vista, you'll want to get Delphi 2007.
  • If you're already on BDS 2006 and plan to upgrade to BDS 2007, you should seriously consider buying Delphi 2007 with SA (Software Assurance) or subscription now - that will (by all likelihood) give you BDS 2007 later this year. The transition from Win32 development on BDS 2006 should be very smooth - just keep using the same components without recompiling them. No need to wait for new versions from 3rd parties.
  • If you are a Delphi 5-7 developer that is still sitting on the fence, this is the time to jump and take this offer. The IDE is much more capable and productive and the performance and flicker-free operation is on par with or better than Delphi 7 now. You can see the list of improvements between Delphi 7 and BDS 20006 in Nick Hodges blog here - all of these goodies (on the native Win32 side) are also part of Delphi 2007, of course.

I highly recommend Delphi 2007 Win32 for all native Delphi developers!


claudio said...

Hi Hallvard

>>I highly recommend Delphi 2007 Win32 for all native Delphi developers!!

I Agree!!

Thank you very much for your very interesting reviews

Best Regards

Claudio Piffer

GrayFace said...

What I don't understand is why they used up existing field for this. They could add extra space in the NewInstance function and store the new fields at negative offset.

Hallvard Vassbotn said...

> They could add extra space in the NewInstance function and store the new fields at negative offset.

Interesting idea, but that would require an override of the NewInstance method - and thus a change in the interface section...

Thomas Schulz said...

Am I the only one that thinks hacks like that are just problems waiting? I.e. when some new developer is assigned maintaining the code, errors will start pop up?

I wonder if they are going to change it in the next Delphi release.

Hallvard Vassbotn said...

Thomas: Yes, that is the nature of hacks. The reduce maintainability, but makes "impossible" things possible - in the short run.

As both Allen and I indicated (see the head line "A GlassFrame into the future"), the hack will be cleaned up in the next binary breaking release (probably Highlander).

Tjipke van der Plaats said...

Thanks for the link to my article!
Which has B.T.W. an interesting comment attached to it from C Johnson: "Class Helpers with an interface that indicate when to show them in the designer would have been a superior experience". Which is a good point!

Anonymous said...

why does SelectionPropertyFilter not known about the version of delphi, or alternatively the RTTI routines etc etc
why is it so hard for CodeGear to decouple the compiler from the vcl version so that older Vcl versions are still supported by later compilers, keeping the investment from shareholders into software companies
I am willing to pay 2000 for a better IDE/debugger, but why change existing code?

Bruneau said...

> This class helper also injects a somewhat cryptically named EnumAllWindowsOnActivateHint property that as far as I can see provides a fix for showing hints for windows that belongs to the process, but was created in another thread.

Your assessment is correct. I thought I'd add that it targets specifically ActiveForms embedded in IE7. IE7 creates ActiveX controls in a worker thread. One of the problems (there are others - specially related to modality) this change creates is that ActiveForms in IE7 do not display hints.
That's because the Hint-displaying logic in TApplication checks that the current ActiveWindow belongs to our thread before proceeding.

EnumAllWindowsOnActivateHint instructs TApplication to enumerate through all top-level windows, not just windows belonging to our thread. We wanted to address the IE7/ActiveForm/Hint
issue in a more transparent fashion but TCustomActiveForm does not override the proper virtual that would allow us to check if it was being created in a worker thread - hence the new property. Future versions might deprecate the property and handle the issue automatically. For now ActiveForms in IE7 that wish to display hints must explicitly enable that property.

Anonymous said...

All the blogs on Delphi 2007 so far seem inaccurate and slanted.

I just got 2007 this morning and have begun to test it out. So far every new feature that has been blogged about in various blogs either doesn't work or doesn't even exist!

Were these features removed at the last minute or something yet were present in the betas? I heard something about PNG support, and large icon support yet I can't find it!

The Aero glass stuff is now drawing properly... even on a new project.

Anonymous said...

Gotta reference this article. It rates how a bunch of controls ACTUALLY perform on glass and has some extremely disappointing results.

Copyright © 2004-2007 by Hallvard Vassbotn