Tuesday, December 28, 2004

Delphi vs C#: Destructors

Delphi's implementation of destructors is very different from C#'s. This means that if you are translating C# code samples to Delphi, you have to be careful with how you convert C# destructors. A question from Ray in the delphi.rtl.dotnet newsgroup illustrates this problem:

"Subject: Garbage collector bug in Delphi.NET but not C#?
Hi there
In testing the Garbage collection in Delphi.NET I found the following.
Delphi.NET code

//******************** 
type
TDemo = class
public
class var Instances: integer;
constructor Create; virtual;
destructor Destroy; override;
end;
//futher down
procedure TWinForm.Button2_Click(sender: System.Object; e: System.EventArgs);
var
x: Integer;
a: TDemo;
begin
Text := 'Running';
for x := 1 to 100000 do begin
a := TDemo.Create;
a.Free;
end;
end;

procedure TWinForm.Timer1_Tick(sender: System.Object; e: System.EventArgs);
begin
Text := TDemo.Instances.ToString;
end;

procedure TWinForm.Button1_Click(sender: System.Object; e: System.EventArgs);
var
x: Integer;
a: TDemo;
begin
for x := 1 to 100000 do begin
a := TDemo.Create;
end;
end;

{ TDemo }

constructor TDemo.Create;
begin
inherited;
Inc(Instances);
end;

destructor TDemo.Destroy;
begin
Dec(Instances);
inherited;
end;
//********************

and the C#


//********************
  public class Demo
  {
    public static long Instances;
    public Demo()
    {
            Instances += 1;
    }
    ~Demo()
    {
       Instances--;
    }
  }
//Further down
  private void button1_Click(object sender, System.EventArgs e)
  {
    int Counter;
    Demo aDemo;
    for (Counter = 0; Counter < 100000; Counter++)
    {
      aDemo = new Demo();
    }
  }
  private void timer1_Tick(object sender, System.EventArgs e)
  {
    Text = Demo.Instances.ToString();
  }
//********************

Now the issue I have is with C# it will have 100 000 classes and then a few seconds later 37125 then click button and it's 2500 etc. Works like the help says...


BUT Delphi is "buggy". Button1 creates 100 000 and does NOT free even after multiple clicks, however button2 with the .Free frees instantly. So it works like Win32. Surely this is a bug (Not that I'm complaining I'd rather it free up memory when I say so).


Am I missing something?


Regards Roy"


This is not a bug - a Delphi destructor is not the same as a C# destructor.


A Delphi destructor implements the IDisposable pattern, while a C# destructor overrides the Finalize method of System.Object (a very rarely recommended practice). Look at the generated IL code from both compilers, and you will see the difference.


To make the comparison relevant, either change the Delphi code to override the Finalize method or change the C# code to implement the IDisposable interface. FWIW, the new C++/CLI standard implements C++ destructors as IDisposable implementations too. C# destructors are a mistake, IMO. Don't use them unless you know exactly what you are doing.


This triggered a follow-up question from Ray:



"I got the C# code from a MSCD.NET CD, and according to the demos / trainer material this is recommended? Besides VB.NET worked the same as C# so does "C# destructors are a mistake" hold true for VB.NET?


http://www.delphi.about.com/od/delphifornet/a/aa060104a.htm


"Every class in the .NET Framework inherits a method called Finalize. The bad news is that the GC calls the Finalize method (and therefore if overridden it should be protected) when the memory for the object is about to be freed. Since the method is called by the garbage collector, you have no control over when it is called.


The good news is that in Delphi for .NET, all objects implicitly implement IDisposable and redirect calls to Dispose to the Destroy destructor. This means that you do not need to implement Dispose yourself ... you can safely continue freeing resources (for objects/classes you develop) in the object's destructor "Destroy" method."


The part that threw me off was "you have no control over when it is called."


That's one of the reasons Finalize is generally not recommended. Another is that objects are kept alive much longer than needed.


Finalize is generally only needed for cleaning up unmanaged resources like file and window handles - and that should in 99% of the time be taken care of by small low-level wrapper classes defined in the .NET framework.


99% of the time you do not want to implement C# destructors in application-level code.


99% of the time(1) you do want to implement IDisposable in application-level code. Writing Delphi destructors makes this easy.


This artible by Brian Long is recommended reading:
http://bdn.borland.com/article/0,1410,29365,00.html


(1) The reason for this is that while finalizers are singular and automatically called by the GC (only the class holding references to unmanaged resource handles need finalizers), IDisposable requires a cascading implementation in wrapper objects. For instance, if a class A is implemented in terms of (by having a private reference to) a class B, and class B implements the IDisposable interface, the wrapping class A should also implement the IDisposable interface. This even includes cases where the implementation of A or B might change in the future to implement IDisposable.

Tuesday, November 16, 2004

Object-to-interface casts

There are several ways to check if an object reference implements an interface or not - but there are some platform differences. The as-cast works (almost) identically in both Win32 and .NET.

var 
MyObject: TMyObject;
MyInterface: IMyInterface;
begin
// ...
MyInterface := MyObject as IMyInterface; // may raise EInvalidCast
MyInterface.Foo;
end;

The platform difference is that on Win32, the declared object type must implement the IInterface (or IUnknown) interface (as TInterfaceObject does, for instance), while in .NET any object type will do. If the cast fails, an exception will be raised. This is often undesirable - often you need to treat objects differently depending on if they implement a specific interface or not, and handling exceptions is too slow and cumbersome. On .NET you can cast from any object to any interface, using a hard-cast syntax. If the cast fails, nil is returned instead.

var 
MyObject: TMyObject;
MyInterface: IMyInterface;
begin
...
MyInterface := IMyInterface(MyObject); // .NET specific
if Assigned(MyInterface) then
MyInterface.Foo;
end;

Currently, the Win32 compiler does not support this cast in a generic way. It does support it in the specific case when the cast can be determined to be valid at compile-time. This can be done when the declared object type statically implements the given interface. If this is not the case, the cast fails with a compile-time error; [Error]: Incompatible types: 'IMyInterface' and 'TInterfacedObject' To check if an object supports an interface without raising exceptions, you can first cast it to IInterface then call the QueryInterface method. This only exists on Win32:

 Intf := IInterface(MyObject); 
if Intf.QueryInterface(IMyInterface, MyInteface) = 0 then
MyInterface.Foo;

The Supports routine in the SysUtils unit gives is a more convenient and cross-platform way of doing the same thing:

 if Supports(MyObject,IMyInterface, MyInteface) then 
MyInterface.Foo;

Supports is also available on .NET, but it is implemented in a way that makes it much slower than using the (currently) .NET-specific hard-cast. This is because the .NET version is implemented in terms of the new 'type of interface' construct:

type 
TInterfaceRef = type of interface;
...
function Supports(const Instance: TObject; const IID: TInterfaceRef; out Intf): Boolean;
begin
Result := Instance is IID;
if Result then
Intf := Instance as IID;
end;

The 'type of interface' construct is pretty neat, actually. It allows you to pass any interface-type as a parameter to a method. This is used instead of a TGUID as an identifier or handle to the given interface. The Supports function itself compiles into the following IL:

.method public hidebysig static bool Supports(object Instance, [mscorlib]System.RuntimeTypeHandle IID, object& Intf) cil managed 
{
// Code Size: 50 byte(s)
.maxstack 3
.locals (
bool flag1,
[mscorlib]System.Type type1)
L_0000: ldarg.1
L_0001: call [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle([mscorlib]System.RuntimeTypeHandle)
L_0006: ldarg.0
L_0007: callvirt instance bool [mscorlib]System.Type::IsInstanceOfType(object)
L_000c: stloc.0
L_000d: ldloc.0
L_000e: brfalse.s L_0030
L_0010: ldarg.2
L_0011: ldarg.1
L_0012: call [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle([mscorlib]System.RuntimeTypeHandle)
L_0017: stloc.1
L_0018: ldloc.1
L_0019: ldarg.0
L_001a: callvirt instance bool [mscorlib]System.Type::IsInstanceOfType(object)
L_001f: brtrue.s L_0029
L_0021: ldarg.0
L_0022: ldloc.1
L_0023: call object Borland.Delphi.Units.System::@CreateCastException(object, [mscorlib]System.Type)
L_0028: throw
L_0029: ldarg.0
L_002a: castclass object
L_002f: stind.ref
L_0030: ldloc.0
L_0031: ret
}

Quite a bit of code!


With a little help from Reflector, I've found that it corresponds to something like this in Delphi code:

function Supports(Instance: TObject; IID: RuntimeTypeHandle; out Intf: TObject): boolean; 
var
InterfaceType: System.Type;
begin
Result := System.Type.GetTypeFromHandle(IID).IsInstanceOfType(Instance);
if Result then
begin
InterfaceType := System.Type.GetTypeFromHandle(IID);
if (not InterfaceType.IsInstanceOfType(Instance)) then
raise EInvalidCast.Create(SInvalidCast);
Intf := Instance;
end;
end;

Notice that the code is a little redundant. The precence of both the is-check and the as-check makes the compiler emit calls to GetTypeFromHandle and IsInstanceOfType twice each. The exception will never be raised in this method. So there is a little room for improvement here. I think that passing 'type of interface' parameters as System.Type instead of RuntimeTypeHandle would also make it a little more efficient - in most (all?) cases the call site has a Type reference anyway and first has to convert it into a RuntimeTypeHandle. The site calling Supports yields this IL:

   ldloc.1 
ldtoken Obj2IntfCastNET.IMyInterface
call [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle([mscorlib]System.RuntimeTypeHandle)
pop
ldtoken Obj2IntfCastNET.IMyInterface
ldloca.s obj2
call bool Borland.Vcl.Units.SysUtils::Supports(object, [mscorlib]System.RuntimeTypeHandle, object&)
stloc.0


By contrast, performing the same operation by using the object-to-interface hard-cast, gives this very simple and efficient IL:


    ldloc.s obj2
   isinst Obj2IntfCastNET.IMyInterface
   stloc.2


Can you guess which is faster? No prizes for this one! So you can use the cross-platform Supports function - which is fairly efficient in Win32, but relatively slow in .NET, or the .NET platform specific and efficient hard-cast. Most of the time, the slow .NET version of Supports does not really matter, but for some performance critical points, you might want to use conditional compilation to select the best solution for each platform, like this:

{$IFDEF WIN32} 
if Supports(MyObject,IMyInterface, MyInterface) then
{$ENDIF}
{$IFDEF CLR}
MyInterface := IMyInterface(MyObject);
if Assigned(MyInterface) then
{$ENDIF}
MyInterface.Foo;

I hope that a future version of the Win32 compiler can be extended to support hard-casts from object-to-interface with the same semantics as on .NET. The cast should return nil if the object does not support the interface (yes, this has been reported to Borland). In .NET you can also use the 'is' operator to check if an object implements an interface or not. This is not currently supported in Win32.

 if MyObject is IMyInterface then 
Writeln('Yup');

Friday, November 12, 2004

Machine Code Hacking

I'm not shy from performing the occasional hack, and feel comfortable working in the CPU view of the debugger. Microsoft hacker Raymend Chen seems to be in another league however. He claims he never uses source level debugging (its too mucked up by the optimizer anyway) and here he gives some tips on how to patch machine code bytes while debugging to make the debuggee do what you want. Useful for those low-level debugging sessions ;).

Tuesday, November 09, 2004

Rocket Scientist wanted!

Our company are looking for the best Delphi and C++ programmers out there. From our website: "Our highly qualified team of developers are looking for new colleagues. If you are an experienced developer looking for a challenging position in an informal organization we would very much like to hear from you." The details about the positions are available here (in Norwegian): "Rocket Scientist" - Borland Delphi / C/C++ So send us an email with your resumé - or tips your favorite Delphi or C++ guru... :) Updated: The job advert is now available in English: http://www.infront.no/pdf/jobad_programmer.pdf

Monday, November 08, 2004

Delphi 2005 Reviewer's Guide

Borland has just made a very comprehensive review guide for Delphi 2005 available. It is written by Cary Jensen and can be downloaded here: http://www.borland.com/products/white_papers/ pdf/delphi_2005_reviewers_guide.pdf If you wonder what's new and how the new features look like, check it out.

Interfaces - benfits and platform differences

Interfaces is a very powerful tool that can improve the quality of your code, if used properly. Interfaces can give the following benefits:

  • Reduced coupling between components
  • Strong encapsulation
  • High extensibility
  • Easy substitutability
  • Run-time discovery of object capabilities

Under the hood, interfaces are implemented very differently in .NET and Win32. Win32 interfaces follow the COM model, with an IUknown base interface that provides basic services for reference counted lifetimes and interface support querying.

A Win32 interface declaration needs a GUID to enable as-casts (or QueryInterface or Supports calls). GUIDs are not needed in .NET - they can optionally be used to expose interfaces to COM with a specific GUID. To generate a GUID in the editor use the Shift+Ctrl+G keyboard shortcut.

type 
IMyInterface = interface
['{6141D774-34AD-4E11-AB16-DD74EFE793F5}']
procedure Foo;
end;

.NET interfaces do away with reference counting and instead relies on the garbage collection mechanism that all object references enjoy. There is no base-interface that all other interfaces inherit from.


Instead of QueryInterface, IL instructions like castclass and isclass and methods of the Type class are used. At runtime, the reference value of an interface variable is identical to the object reference value that implements the interface. The only difference is the type of the variable and how the JIT compiler dispatches method calls on the interface reference. More on casting to and from interfaces differences shortly.

Tuesday, October 26, 2004

The FastCode AnsiStringReplace results are in

The results of the FastCode AnsiStringReplace competition are in:

http://dennishomepage.gugs-cats.dk/AnsiStrReplaceChallenge.htm

The fastest entry is some 7-9 times faster than the RTL reference version (depending on the CPU). My own contribution scored less than 3 times faster... ;). Note that real-world speed gain will vary considerably - as always what you benchmark determines the results.

When that is said, Eric Grange's winning entry has some really neat tricks. The most important trick is to search for and store the position of all hits in the source string before allocating the result string and copying over the string data. Another is to use an upper-case character table to avoid pre-upping strings in the case-insensitive mode.

A much faster StringReplace can be nice, but before you jump in, consider if StringReplace really is the right tool for the job at hand. As Martin James put it in the delphi.non-tech newsgroup:

"In anything other than trivial cases with small strings, I always use a char-by-char state machine anyway, certainly with HTML text parsing/tag handling. In such cases, any 'stringReplace' is hugely inefficient."

My response was:

"Yes, you are right. If you want performance, using StringReplace is most probably the wrong solution (no matter how fast it is). Most of the time you don't want to replace a single string, you want to replace a number of strings/patterns. A custom parser using a state-machine is often the best solution, performance wise. However, such optimizations might be prohibitively expensive to develop for simple problems where StringReplace will suffice."

And on the FastStrings comment from my last blog entry, here are the results on one CPU:

P4 1600A Northwood
StringReplace_EG_MMX_1 8 671 1165 1836 // Eric Grange = 1836 ms
...
StringReplace_HV_IA32_1 4 2846 5018 7864 // Mine = 7864 ms
StringReplace_DKC_Pas_1 8 4565 4990 9555
FastAnsiReplace 0 4382 9248 13630 // FastStrings version = 13630 ms
StringReplaceRef 0 5523 9514 15037 // Bug-fixed RTL version = 15037 ms
StringReplaceRTL 8 6848 13336 20184 // RTL version = 20184 ms

So even my version kicked FastStrings ReplaceString... <bg>. And even as we speak, those delphi.language.basm folks are now cooperating to improve the blind-test winners few notches more...

Friday, October 22, 2004

StringReplace isn't a speed deamon...

Currently, the Win32 version of StringReplace in the SysUtils unit isn't exactly a speed deamon . It has several problems:

1. It grows the result string bit-by-bit. If many replacements are being done, many reallocations and memory copy operations are going on.

2. It calls AnsiPos which calls AnsiStrPos, which at least calls StrLen twice and StrPos once. StrPos internally recalculates the string lengths again (by scanning for #0). So for every AnsiPos call, the entire source string is scanned twice to find the length. This length is constant and known by StringReplace.

3. It copies the 'rest' part of the source string to a separate SearchStr variable. This is a lot of overhead just to know where in the string you are.

4. AnsiPos/AnsiStrPos also have the overhead of multi-byte support even for locales that don't need it. I have written a suggestion for replacement that includes versions of StrPos and StrScan that take length parameters:

function StrLPos(const Str1, Str2: PChar; Len1, Len2: integer): PChar; 

function StrLScan(const Str: PChar; Chr: Char; Len: integer): PChar;

Next I wrote a new AnsiStrLenPos that takes lengths as parameters:

function AnsiStrLenPos(Str, SubStr: PChar; L1, L2: Cardinal): PChar; 

This new version uses StrLPos to quickly find the next position without scanning for the length. It also optimizes the fairly common case when the substring length is 1 by calling StrLScan instead. By explicitly checking for FarEast (implying a multibyte character set), we can speed up the normal case. There are no case-sensitve code here (we're upping the strings for that case), so the plain StrLScan and StrLPos should do fine.


I have uploaded my faster version of StringReplace to CodeCentral: http://codecentral.borland.com/codecentral/ccWeb.exe/listing?id=22677


An earlier version of WebBroker on Steroids used this faster StringReplace, markedly speeding up the template processing (while reducing memory consumption and fragmentation). A later version dropped using StringReplace for a faster algorithm using custom parsing.


Notice that the dedicated assembly hackers over in the borland.public.delphi.langauge.basm newsgroup have a FastCode competition going (one amongst many) to write the fastest implementation of StringReplace. The winner here will probably be orders of magnitude faster than my version ;).


One day these optimized versions of the Delphi RTL routines might end up in the next version of your favorite developement tool.

Wednesday, October 13, 2004

The Delphi 2005 launch in Oslo was a blast!

More than 140 people showed up for a breath-taking demonstration of some of the great new features of Delphi 2005 (yes, that is the official product name). Jason Vokes covered the IDE, Unit tests integration, Error Insight, Help Insight, SyncEdit, Refactoring, ASP.NET, Borland Data Providers, and much more. Henrik Jondell demonstrated ECO II, Database reversing, Model driven development and more.

Yours truly got 15 minutes to show some slides on me, Infront AS, how we use Delphi, the most productivity enhancing new Delphi features and what's new in the Delphi language for D8, D2005 Win32, D2005 .NET and in the future.

You can download my slides and some of the demo code I didn't have time to show here: http://cc.borland.com/ccWeb.exe/listing?id=22630.

John Kaster has uploaded his BorCon Delphi 2005 slides here: http://cc.borland.com/ccWeb.exe/listing?id=22324

They contain much more detail and cover much more ground (this is were I got some of the future language stuff) - recommended!

Monday, September 27, 2004

Don’t miss the Delphi launch in Oslo!

Borland Nordic have started sending out invitations to the launch event of the next Delphi version. Based on the sneak-peak URL, the final name for Diamondback might be Delphi 2005. The event in Oslo will be Tuesday October 12th at Fabrikken (The Factory), address Nedre gate 7.

I'll be there - will you?
Register here.

Tuesday, September 14, 2004

Object-to-object casts

In object-oriented code, probably the most common type of cast is between object types.

Up-cast
This cast is not really needed at all, as the compiler will implicitly convert from a descendent class up to a base class. This is conceptually done each time you call an inherited method, pass values into TObject parameters, add objects to a TObjectList and so on. Although the compiler will implicitly handle up-casts, it does allow the programmer to explicitly apply up-casts as well, but they will be optimised away into no-ops:

type 
TParent = class
end;
TChild = class(TParent)
end;
var
Parent: TParent;
Child: TChild;
begin
Child := TChild.Create;
Parent := Child; // Implicit up-cast
Parent := TParent(Child); // Explicit hard-cast
Parent := Child as TParent; // Explicit as-cast
if Child is TParent then // is-test
Writeln('Yup');
end.

The code above demonstrates the syntax used for hard-casts, run-time checked as-casts and the is-check. The Win32 compiler assumes that you are not deliberately breaking the type-system, so it optimises the as-cast into a no-op and the is-test into an Assigned-test.


Down-casts
This means that if you play it dirty and use a hard down-cast to store a TObject in a TChild reference (effectively breaking the type system), the casts and is-check will still succeed;

var 
O: TObject;
// ...
Child := TChild(TObject.Create); // Dirty trick - don't do this
if Child is TParent then // is-test
Writeln('Fake!');
O := Child;
if not (O is TParent) then // is-test
Writeln('Nope!');

The.NET runtime makes it virtually impossible to bypass the type-system (bar unsafe or unmanaged code). While as-casts and is-tests work identically in Win32 and .NET, hard-casts that fail at runtime will return nil in .NET and break the type-system in Win32. So on .NET the code above will actually store nil in Child (because a TObject is not a TChild). A down-cast is casting from a parent-class reference to a child-class reference. This is normally what we mean with object-casts.


Safe vs. unsafe casts
In Win32, hard casts are unsafe, because the compiler will not complain if you perform obviously "illegal" or "impossible" casts. Hard-casts is a way of telling the Win32 compiler;



"Relax, I know what I'm doing. Just close your eyes and reinterpret these bits as the type I'm telling you it is".


So there are no checks and no conversions going on (there are a couple of exceptions as we shall see in a later blog post). In .NET even hard-casts are safe, in the sense that the compiler and runtime will check that the cast is valid. For object-to-object hard-casts, the CLR will check that the source is compatible with the target type - if not, nil is returned instead. Conceptually, in .NET hard-casts like this:

  Target := TTargetClass(Source); 

work like this:

if Source is TTargetClass then
Target := Source
else
Target := nil;

As-casts are safe on both platforms. If the cast does not succeed, an exception will be raised, so:

  Target := Source as TTargetClass;

is conceptually equivalent with

  if Source is TTargetClass then 
Target := Source
else
raise EInvalidCast.Create;

Performance issues
In Win32, there is a certain performance overhead with doing an is- or as-check - the compiler and RTL traverses the inheritance chain linearly to check all the class' parents to search for a match. For most code the overhead is negligible. A common need is to covert a reference to a specific type, but to avoid exceptions in the failing cases, so often you'll see code like this:

if Instance is TMyClass then 
MyObject := Instance as TMyClass;

Experienced Delphi programmers will often balk at this construct, noting that the additional test performed by the as-cast is redundant. Again, for most code the overhead is insignificant, but never-the-less you'll often find it rewritten like this:

if Instance is TMyClass then 
begin
MyObject := TMyClass (Instance);
// ... xxx


Here the safe is-cast is performed, followed by the unsafe hard-cast. But combined with the preceding is-cast, the hard-cast is actually safe! In Win32, only one check of the inheritance chain is performed. In .NET however, the hard-cast performs another is-check (returning nil if it fails, which should be never in the above code). So performance aficionados might be tempted to change the code into this:

  MyObject := TMyClass(Instance);
if Assigned(MyObject) then
begin
// ...

That will work nicely without overhead in .NET, but it will fail (typically crash) in Win32 when the cast is invalid. The examples above gracefully handle the case where the cast fails at runtime. Other times you consider it a programming or configuration error for the cast to fail. This is when you'll typically use the as-cast - it will raise an EInvalidCast exception (alias to System. InvalidCastException in .NET) if it fails. This is fairly common to do in event-handlers, for instance. The generic Sender parameter is TObject, so you might need to type cast it to the actual component type (say TDrawGrid). This is particularly useful when you share a common event handler for multiple components. One way to do this is:

procedure TMyForm.GridsDrawCell(Sender: TObject; ...);
var
Grid: TDrawGrid;
begin
Grid := Sender as TDrawGrid;
// ...
end;

This performs the as-check for each draw-cell operation for all the grids in the form. But the only way for the cast to fail is if you have somehow assigned the event handler to a non-DrawGrid component. Why burden the release version of your application with a check that is bound to succeed every time? An alternative solution is to turn the check into an Assert, like this:

procedure TMyForm.GridsDrawCell(Sender: TObject;  ...); 
var
Grid: TDrawGrid;
begin
Assert(Sender is TDrawGrid);
Grid := TDrawGrid (Sender);
// ...
end;

Now in the release builds, the runtime check will be gone (except on .NET), while silly programmer mistakes will still be caught in the internal debug builds (with assertions enabled). Be careful you're not over-applying this optimisation, however. You do want to keep as-casts or is-tests in code where the runtime type of the object may vary according to external input (user interaction, importing data files, communicating over TCP/IP etc).


The protected-access trick
I've talked about this trick earlier. It basically revolves around performing an invalid hard down-cast to a locally defined class. This is done to get access to the protected parts of the object. For instance:

type 
TControlAccess = class(TControl);
begin
TControlAccess(MyControl).Click;
end;

Click is a protected method of TControl, so normally we cannot call it on a control instance. To fool the type-system, we declare an empty local class that inherits from TControl. Because our code is in the same unit as this new type, we get access to all its protected members. Although MyControl is not really a TControlAccess instance, we cast it into one, to call the Click method. This does works in Win32, but it is really a hack, IMO.


This hack is so common, and many VCL components and Delphi applications rely on it, that Borland felt the need to somehow support this technique even in .NET. It does work in .NET as long as the accessed member resides in the same assembly as the calling code. The compiler marks all protected members as the CLR access level protected-or-assembly. This means that at the IL-level, all protected members are directly accessible to all code in the same assembly.


The Delphi compiler does not allow access to these members unless you perform the casting-trick, however. When it encounters the cast, it simply removes it - there is no trace of it in the IL code. The CLR will allow the code as long as it is intra-assembly. The Delphi compiler will also prevent the call across assemblies, of course. For instance if you try the call above when referencing (and not linking in) the Delphi.Vcl.dll assembly the compiler will emit the following compile time error:


[Error] Cross-assembly protected reference to [Borland.Vcl]TControl.Click in Upcasts.Upcasts

Friday, September 10, 2004

Blog description updated

FWIW, I've updated the blog description in the header of my blog, to better reflect what I'm actually writing about ;). The new description reads: "Confessions of a Delphi hacker - hacks, programming tips and Delphi stuff." Oh, btw, I've just returned the corrected proofs of my upcoming Delphi Magazine article to the editor - it looks good! Make sure you have that subscription updated... ;).

Win32 vs. .NET casting issues in Delphi

In a small series of blog articles I'm planning to talk about some of the different types of casts and conversions that Delphi supports, and mention any differences in casting logic and implementation between Win32 and .NET.

In general, using casts in your code indicates an impendence mismatch between your design and the problem domain. Sometimes they cannot be easily avoided, other times it might be better to refactor your design and code to remove or reduce the number of casts. Until true generic classes become available (as they probably will - at least for .NET 2.0), it is close to impossible to have no casts anywhere in the code. They can (and should) be encapsulated to a few core locations, however - typically in type-specific collection classes.

For the purposes of this discussion, I've identified 9 separate categories of casts, in terms of the types of the entities we are casting to and from.

Types of casts:

1. Object-to-object upcast

2. Object- to-object downcast


3. Object-to-interface cast

4. Interface-to-interface cast

5. Interface-to-object cast

6. Valuetype-to-valuetype cast

7. Valuetype-to-object cast

8. Object-to-valuetype cast

9. Casts to and from compiler-managed types

The upcoming blog posts will each discuss one or two of these casting categories.

Wednesday, August 11, 2004

Why the with-statement should not be extended

N. Ismail commented on my last post about the with-statement:

"I recommend the with statement be extended to allow it to also be written as such:

with a: Form1 do 
begin
a.foo := 123;
a.bar := 'xyz';
a.Caption := 'Test';
Caption := 'Foo Bar'; //The forms caption
end;"

IMO, the with statement should not be extended.


Here are my reasons:


#1. This will help nothing for existing code. Existing code uses with the old way, so the compiler would have to support that syntax for foreseeable future.


#2. Enhancing the with-statement will encourage increased use of it, which is a bad thing, IMO. My suggested warning should encourage decreased use.


#3. What you want can easiliy be obtained today and work in all versions of Delphi. Just define the identifier as a local var and get rid of the with, like this:

var 
a: TForm1;
begin
a := Form1;
a.foo := 123;
a.bar := 'xyz';
a.Caption := 'Test';
Caption := 'Foo Bar'; //The forms caption
end;

"Obviously one could have declared a variable named 'a' of same type as Form1,"
Exactly ;)
"but I think the above syntax is cleaner"

I don't. And it doesn't give you anything functionally that a local variable cannot give you. Its pure syntactic sugar (with a sour taste, IMO).

Tuesday, August 10, 2004

The with-statement considered harmful

As have been mentioned by several people in several places (newsgroups, blogs, QC etc.) Delphi's with-statements can be harmful to the readability, maintainability and robustness of your code. Not to mention it makes debugging harder ;).

Much like the problem with potential duplicate identifiers from used unit scopes, the with-statement problem could be mitigated by enhancing the compiler to produce Warnings for ambiguous with-statements.

Consider the following code:

Unit Foo; 
type
TGadget = class
bar: integer;
end;
//----
Unit Bar;
var
a: TGadget;
foo: integer
begin
with a do
foo := 123;
end;

This is version 1. Code compiles without warnings and works correctly. Now Mr. Component on the other side of the globe makes an update to the TGadget class:

Unit Foo; 
type
TGadget = class
foo: integer; // New feature!
bar: integer;
end;

Then John Doe recompiles his code with this new version:

Unit Bar; 
var
a: TGadget;
foo: integer
begin
with a do
foo := 123;
end;

Currently, with D7, there is no warning. Code compiles, but it fails mysteriously at run-time. In a future version of the compiler he could get warning such as: Warning: Potential name conflict: foo declared in unit Bar and Foo.TGadget. In this case, to get rid of the warning, the programmer could change the code to

   Bar.foo := 123; 

or (even better) he could just get rid of the with-statement.


IMO, if we could have this kind of warning for with-statements, it would be the first steps towards patching one of the very few sorespots of the otherwise elegant Object Pascal language. (and before you ask, yes, this has been QC'ed: #8770).

Friday, August 06, 2004

The uses clause considered harmful

(The headline plays with Edsger Dijkstra's famous article "Go To Statement Considered Harmful") Because the way unit references in the uses clause work in Delphi, you could occasionally get weird and even buggy effects because of changes made in one of the units you use. Note that in general, Delphi handles modularity and code imports much cleaner than most other programming languages. Still there is room for some improvement.

My suggestion is that the Delphi compiler be improved to emit a new Warning for potential name conflicts when the same identifier is imported from two or more units. The goal is to catch unintended changes in semantics after changes have been made to used units.

If I have two units, A & B, both of which export the identifier foo and then in another unit I write:

uses A, B; 
// ...
begin
foo;
end.

Here the reference to foo is without a unit prefix. Currently I would get no warning. My suggestion is to generate a warning, something like this:


"Warning: Potential name conflict: foo declared in unit A and unit B."

To resolve the warning, the programmer would prefix foo with the intended unit name:

begin   
A.foo;
end.

This would then generate no warning. Technically, the reference to 'foo' isn't really ambiguous. That's why you shouldn't get a compile error. But like most warnings, this one should be there to catch potential human error or confusion. You have two possible sceniarios:



  1. The user meant to use A.foo, but because of the unit-ordering he is actually using B.foo.

  2. Last week, foo was only defined in unit A and the program was compiling working as intended. This week a new foo has been added to B. The program still compiles, but stops working correctly at run-time. This is something that can be really hard to debug and track down - it has happened to me...

#2 is the most dangerous and where the warning would be most useful.

Thursday, August 05, 2004

Delphi 9 is Diamondback!

John Kaster of Borland just posted a new BDN article about the upcoming BorCon 2004 and the tracks devoted to the next version of Delphi, code-named DiamondBack! Borland hotshots like Danny Thorpe, Allen Bauer, John Kaster, Jim Tierney, Corbin Dunn & al will reveal the new cool features. Check it out: http://bdn.borland.com/article/0,1410,32499,00.html

Thursday, July 29, 2004

Hack #7: Interface to Object [in The Delphi Magazine]

Due to the length and style, the following hack has been turned into a The Delphi Magazine article. IMO, this is simply the best Delphi magazine for professional developers and I highly recommend a subscription. Don't forget to get the Collection CD to read all my old articles :-) (in addition lots of other great articles too, of course).

Read the magazine article (update: in the October issue) to get the complete hack, explanations, history, the clean alternative, sample code and .NET comparison. As a teaser, here is the actual hack code:

function GetImplementingObject(const I: IInterface): TObject;

const
AddByte = $04244483;
AddLong = $04244481;
type
PAdjustSelfThunk = ^TAdjustSelfThunk;
TAdjustSelfThunk = packed record
case AddInstruction: longint of
AddByte : (AdjustmentByte: shortint);
AddLong : (AdjustmentLong: longint);
end;
PInterfaceMT = ^TInterfaceMT;
TInterfaceMT = packed record
QueryInterfaceThunk: PAdjustSelfThunk;
end;
TInterfaceRef = ^PInterfaceMT;
var
QueryInterfaceThunk: PAdjustSelfThunk;
begin
Result := Pointer(I);
if Assigned(Result) then
try
QueryInterfaceThunk := TInterfaceRef(I)^.QueryInterfaceThunk;
case QueryInterfaceThunk.AddInstruction of
AddByte: Inc(PAnsiChar(Result), QueryInterfaceThunk.AdjustmentByte);
AddLong: Inc(PAnsiChar(Result), QueryInterfaceThunk.AdjustmentLong);
      else    

Result := nil;
end;
except
Result := nil;
end;
end;

Monday, June 21, 2004

Roy Nelson is blogging - and Brian Long too

If you are looking for more hacks, you should be subscribing to Roy Nelson's blog. Roy is a great asset of technical info and hacks - in this case he was helping out another great spelunker and Delphi expert, Brian Long.

Wednesday, June 16, 2004

Hack #6:Checking for a valid object instance

I just answered a question in a newsgroup, by simply copying-and-pasting an old post of mine. Reuse is nice, so I decided to put it in my blog as well. Here it is: "Just for the fun of it, I've made some routines to check if a pointer points to a valid object instance. This relies on some types and routines from Ray Lischner's "Secrets of Delphi 2" (the S_VMT unit). Note that this code is D3 specific. Small changes are needed for D1/D2. [Right :)]

uses 
S_VMT;

function ValidPtr(P: pointer; Size: Cardinal): boolean;
begin
Result := not IsBadReadPtr(P, Size);
end;

function ValidObjType(Obj: TObject; ClassType: TClass): boolean;
begin
Result := Assigned(Obj) and
ValidPtr(Pointer(Obj), SizeOf(TObject)) and
ValidPtr(Pointer(Obj), ClassType.InstanceSize);
end;

type
PClass = ^TClass;

function ValidPShortString(S: PShortString): boolean;
begin
Result := ValidPtr(S, SizeOf(Byte)) and
ValidPtr(S, Ord(S^[0])) ;
end;

function ValidClassParent(ClassParent: PClass): boolean;
begin
if ClassParent = nil then
Result := true
else
if ValidPtr(ClassParent, SizeOf(ClassParent^)) then
Result := (ClassParent^ = nil) or ValidClassType(ClassParent^)
else
Result := false;
end;

function ValidClassType(ClassType: TClass): boolean;
var
Vmt: PVmt;
begin
Vmt := GetVmt(ClassType);
Result := ValidPtr(Vmt, SizeOf(Vmt^)) and
(Vmt^.SelfPtr = ClassType) and
ValidPShortString(Vmt^.ClassName) and
ValidClassParent(PClass(Vmt^.ClassParent)) ;
end;

function ValidObj(Obj: TObject): boolean;
begin
Result := Assigned(Obj) and
ValidPtr(PClass(Obj), SizeOf(TClass)) and
ValidClassType(Obj.ClassType) and
ValidPtr(Pointer(Obj), Obj.InstanceSize);
end;

This is probably not foolproof, but it should work in most instances. It works by checking for valid pointers using the Win32 API IsBadReadPtr, checking that the VMT-pointer for the given object is valid. Using this code is not recommended as an alternative to setting instance pointers to nil after freeing them."


Update; I found another old post with a simpler (and probably safer) way of doing it:

function ValidateObj(Obj: TObject): Pointer;
type
PPVmt = ^PVmt;
PVmt = ^TVmt;
TVmt = record
SelfPtr : TClass;
Other : array[0..17] of pointer;
end;
var
Vmt: PVmt;
begin
Result := Obj;
if Assigned(Result) then
try
Vmt := PVmt(Obj.ClassType);
Dec(Vmt);
if Obj.ClassType <> Vmt.SelfPtr then
Result := nil;
except
Result := nil;
end;
end;

Notice that this version is specific to D6 and D7 (IIRC). Other versions might need to update the hardcoded magic number (17).


Update II [June 22, 2007]:
Note that Pierre's "new" FastMM (and thus the D2006/2007 MM) reuses memory blocks more aggressively than the old MM, so you may be false positives from my function if the instance pointer has been freed and just happens to be reallocated by a different object instance (of potentially a different object type).


As always, use this hack with care and a grain of salt or two.

Tuesday, June 15, 2004

Shameless plug: Buy book, get my autograph!

As part of the renumeration for tech editing Xavier Pacheco's excellent book, Delphi for .NET Developer's Guide, I got three copies of the printed version. I don't really need the extra two, so now they are for up for grabs at eBay: Book 1 Book 2 And, yes, I'll sign the books (if the buyer is interested in that). :-)

Wednesday, June 02, 2004

Hack #5: Access to private fields

As your mama taught you, you should protect your private parts, and leave other peoples private parts alone. The same goes for programming, of course. Encapsulation and information hiding is two of the main pillars of object oriented programming, so we better not be messing with those, right? Right. But once in a blue moon, we just have the urge to behave badly and reach into the private fields of another class, defined in another unit.

When possible, the previously described hack of accessing a private field through a published property's RTTI is preferred to the technique presented in this article. However, sometimes that hack is not very practical or even possible. Using RTTI is relatively slow (although the RTTI access to retrieve field offsets could be cached) and the private field you need access to may not be exposed as a property. In addition, if you need access to all or most of a class's internal private fields, the solution described here may be preferable.

As mentioned earlier, Delphi's concept of private (in contrast with D8/D2006-2007's strict private) still allows other code in the same unit full access, while excluding access to external units. So all classes and routines in the same unit are intimate friends of each other.

Have you noticed how promiscuous some of the VCL classes are with each other? For instance, the abstract base class of the component streaming system, TFiler, defines a number of private fields. The actual worker classes TReader and TWriter (both descending from TFiler) are both defined in the same unit, and blatantly access the private fields defined in TFiler (for instance the all-important FStream field).

If you should wish to write your own application-level streaming system (with the ability to stream and auto-create non-TComponent classes) that still participates in the TPersistent DefineProperties system, you would have to extend TReader and TWriter. It looks like these classes have been designed to be extended, considering the virtual protected methods they introduce. Your own TMyReader and TMyWriter classes would not have access to the TFiler private fields, nor to the private fields defined in TReader and TWriter. This could(*) severely limit what your descendant classes could do.

Ok, enough talk already - on to the meat. To get access to the private fields of TFiler, declare a shadow class that exactly matches the private declaration from Classes.pas:

type   
TFilerHack = class(TObject)
private
FStream: TStream;
// other private fields here
end;

Given a TFiler instance, you can now simply cast to TFilerHack to get access to the fields:

procedure Foo(Filer: TFiler); 
var
Stream: TStream;
begin
Stream := TFilerHack(Filer).FStream;
end;

Notice that I still use the private visibility directive for the field - this still gives access to all code in the same unit. Normally, this kind of cracker classes should be declared in the implementation section of the unit. This hack is of course extremely version sensitive to changes in the cracked class. Compared to other solutions of using magic field offsets it has the benefit of having the compiler computing the offset for you and from being protected against changes in the base class (TObject in this case). Still, if you ever use this hack, make sure you {$IFDEF} or {$IF} protect the code against changes in compiler and RTL version (or 3rd party component set version).


(*) A few years ago (using D5), I actually implemented an application-level streaming system by writing my own TReader and TWriter classes and using hacks similar to the ones described in this article. Since then TFiler, TReader and TWriter have been improved with regards to extensibility - providing protected read-only properties to get to some of the private fields, for instance (kudos to Borland CodeGear on that one). It may well be that a functional application-level streaming system could be built without using this hack in D7, but I wouldn't count on it. I still use TFiler as an illustrative example.

Saturday, May 29, 2004

What is a hack?

Before we move on to even dirtier and more dangerous hacks, lets first define what constitutes a hack. IMO, a programming hack must fulfil the following criteria:

A hack uses a non-sanctioned way of performing a task - it bypasses official and documented APIs. It uses a "clever" and non-intuitive method to perform a task that would otherwise be hard or impossible to do.

Cons:
  • Usually makes your code harder to maintain

  • Fragile code, breaks easily when the code it depends upon changes

  • Hard to port to other platforms

  • Refactoring, improving the design, is often a better solution


  • Pros:
  • No other way to perform a task - the alternative is to rewrite or re-implement OPC (Other People's Code)

  • Potential for improved performance

  • Curiosity, gaining a better understanding of the system

  • Its great fun!


  • The general advice is to avoid hacks in production code. Hacks can be very useful as short-term solutions during research and development, but is best replaced with refactored code and design before going to production. That said, most real-world software contain the odd hack or two.

    Friday, May 28, 2004

    Why blogging - a waste of time?

    There have been various discussions about the blogging phenomena and some worries that too much sensitive company information could bleed out to the public. Other people worry about the time "wasted" in writing and reading blogs. As Captain Jake put it over in delphi.non-tech:

    > Who has the time to write it all?

    Good point, actually. I'm geeky enough to (admit that) I use a laptop on the tram back-and-forth to work. That's about 1 hour per day. For some time now, I've spent that tram-time (and some evenings, I must admit) in writing TDM articles and tech editing Xavier's and Jon's books. As Jon was taking a hike in the Grand Canyon (great pictures! - well worth a look), I had some idle time between chapters.

    I wasn't up to writing a complete magazine article, so I decided to try and join this blogging craze. This means that as new chapters start dropping into my inbox, the rate of my blog articles will most certainly decline. Still, I like the persistence of a blog and the fact that I can write anything I like (within the limits of law). And the conformity of RSS and Atom feeds means that it is easy for readers to know when there is something new to read, without having to poll a web page manually.

    Newsgroups are nice, but the signal-to-noise ratio for interesting information is irritatingly low. It is much more productive to monitor a selected list of relevant and high quality blogs. I was talking to a friend, showing him the hack to gain write access to the Screen.PixelsPerInch property. He just shook his head and said;

    "Hack is your middle name".

    Thus the idea of writing about some of the hacks I've played with over the years materialized. Hopefully, I'll find the time to write about less suspicious programming techniques and more about higher level topics some time in the future... :-).

    Wednesday, May 26, 2004

    Hack #4: Access to protected methods

    As we saw earlier, it is possible to gain write access to a read-only property, or rather to its backing field (am talking about native code Delphi here, Win32 and Kylix, - the situation is quite a bit stricter and cleaner in .NET land). You can promote protected properties to public or published to gain access to them, but there is no corresponding promotion syntax for fields or methods. Often you absolutely need to call a protected method of some class - most often method in a VCL component or a 3rd party utility class. This is usually the sign of a badly design class. The author didn't think that application level code would need access to the method. A mock-up example would be:

    unit Utility;

    interface

    type
    TUtility = class
    protected
    procedure Useful;
    public
    constructor Create;
    procedure Boring;
    end;

    We want to call the protected Useful method, but all we got access to is the public Boring method. Typical, right? ;) If you are lucky, you have control over the type of the instance you are using, and can write your own descendent that promotes the method the clean way, like this:

    type
    TMyUtility = class(TUtility)
    public
    procedure Useful;
    end;

    procedure TMyUtility.Useful;
    begin
    inherited Useful;
    end;

    Now you can use TMyUtility instead of TUtility and you'll have the Useful method available as a public method. A final twist is to use the same class name, like this:

    unit MyUtility;

    interface

    uses
    Utility;

    type
    TUtility = class(Utility.TUtility)
    public
    procedure Useful;
    end;

    Now you only need to add the MyUtility reference in the uses clause after the Utility reference. However, sometimes you cannot control the type of the instance used - it is created inside a body of 3rd party code, for instance.


    As many experienced Delphi programmers will know, you can easily gain access to the protected members of a class instance, simply by casting the instance to a descendent class type declared in the same unit as the cast-expression. For instance, to call the protected Useful method of an instance of the TUtility class you can just:

    type
    TUtilityEx = class(TUtility)
    end;

    procedure Foo(Instance: TUtility);
    begin
    // Fails to compile, cannot call protected method
    Instance.Useful;
    // compiles fine, calls protected method
    TUtilityEx (Instance).Useful;
    end;

    The way the Delphi visibility directives work, all code in the same unit as the declared class will gain full public access to all members. It is a kind of implicit friend concept as known from C++.


    Thanks to the ingeniousness of Borland's compiler developers this common hack even works in .NET with Delphi 8 - as long as the caller and method is in the same assembly - it does not work across assemblies, because protected maps to the family-or-assembly method attribute in the CLR.


    One very prominent component vendor even routinely hides away truly useful (and sometimes essential) methods of their component and helper classes into the protected section, seemingly only in an attempt to hide complexity from naïve users. Then in all their own units that need access to the methods, they turn around and perform this local-descendent-cast hack and their own classes. Shudder! This hack should only be used to get access to other people's code, and not used between your own units!


    Interestingly, the compiler only uses the local hack-class to get access to the protected method. Although the TUtilityEx class is mentioned in the source code, nothing from that class (the VMT, RTTI, classname info etc.) is needed or referenced in the compiled code, so the smart linker removes it from the executable code. Otherwise this hack would be very expensive with regards to code-size.


    Thanks again, Danny & co!

    Tuesday, May 25, 2004

    Hack #3: Enabling the JIT Optimizer for D8 for .NET

    (or the hack that wasn't). You may be getting less than you bargained for in your Delphi 8 compiled assemblies and executables, due to the seemingly hard-coded setting for the Debuggable attribute in the dccil compiler.

    .NET and the CLR define a DebuggableAttribute with two boolean constructor parameters and corresponding properties, named IsJITTrackingEnabled and IsJITOptimizerDisabled. When present in an assembly the CLR's JITer will modify its machine code generation strategies to better suit debugging needs.

    IsJITTrackingEnabled = True means that the JITer spends time and memory during the just-in-time compilation process to generate tables that makes it easier for a debugger (mapping code-addresses back to source line number information, for instance).

    IsJITOptimizerDisabled = True means, well, just that - you'll get easier-to-debug machine code, but the code will perform worse and/or be larger than the JIT optimizer otherwise could produce.

    The trouble with D8 is that dccil seems to be hard-coded to always emit the DebuggableAttribute with the isJITOptimizerDisabled parameter = True. If you compile from the IDE, and turn off the integrated debugger and turn off output of the .PDB file from the Linker options, dccil can be persuaded to output DebuggableAttribute(False, True). But no matter how many different compiler options or command line tweaks I’ve tried, I’ve not been able to make the compiler output DebuggableAttribute(False, False) or no attribute at all.

    IMO, the setting of the compiler Optimization checkbox should control the isJITOptimizerDisabled parameter, but it currently doesn’t. To see the parameter values of this attribute in your assembly or EXE file, use Lutz’ Reflector instead of ILDAsm. ILDAsm doesn't show the actual value of this attribute, due to the way ILAsm works when applying the attribute (from command line parameters). Reflector gives a better picture.

    We need a little piece of test code to see how the settings of this attribute actually influence the runtime behaviour of CPU-intensive code. Thanks to pp-2ch from the newsgroups for the initial version of this mock-up code:

    program TestDebuggable;
    {$APPTYPE CONSOLE}
    function QueryPerformanceCounter(out lpPerformanceCount: Int64): LongBool; external 'kernel32.dll';
    var
    StartCount, EndCount: Int64;
    i, j, k: Integer;
    begin
    QueryPerformanceCounter(StartCount);
    i := 0;
    while i < 1000000 do
    begin
    k := ((i*2)+2);
    i := k div 2;
    dec(i);
    j := i;
    inc(j);
    i := j;
    end;
    QueryPerformanceCounter(EndCount);
    Writeln(Convert.ToString(EndCount-StartCount));
    Readln;
    end.

    Compile this from the IDE or command line with default settings, and load the assembly in Reflector. Enable the display of Custom Attributes. Look at the TestDebuggable node and notice the value of the Debuggable attribute in the lower pane. It says Debuggable(true, true). Now run the test project and notice the runtime ticks.


    On my machine it clocks in at around 42000 ticks. The initial (but less than ideal) hack-around would be to explicitly include the Debuggable attribute in the project file. DebuggableAttribute is defined in the System.Diagnostics namespace, se we need to add that to the uses clause. And it can only be applied to assemblies and modules, so we must prefix it with an attribute target specifier, like this:

    uses
    System.Diagnostics;

    [assembly: Debuggable(false, false)]

    If you compile and run this from the IDE, make sure you turn off the integrated debugging (Tools | Options | Debugger Options | [ ] Integrated Debugging). Refresh the list of assemblies in Reflector, just press F5 (I love this new feature). Now you will see two Debuggable attributes, and depending on how (un)lucky you are, you will see them in this order: Debuggable(false, false), Debuggable(true, true).


    Run the test code again. On my machine, it now clocks in at an impressing 13000, shaving off 70% of the original run time, so I was lucky . The problem is that I have had other instances where the attribute order was reversed, like this: Debuggable(true, true), Debuggable(false, false). DebuggableAttribute does not support multiple instances of itself (AttributeUsage.AllowMultiple = False), but we explicitly add our own in addition to the one always added by the compiler, so we break this rule. Another rule is that the order of multiple attributes is arbitrary, and cannot be relied upon. So sometimes this hack works, sometimes it doesn’t. Back to square one.


    If we look closely at the documentation for DebuggableAttribute and specifically for the IsJITOptimizerDisabled parameter, we’ll see that they say:


    “The runtime … might disable certain optimizations based on the values contained within this attribute.”
    Notice the word “might”. A little more experimentation has shown that (in the current V1.1 CLR) you will in fact get the JIT optimized code we’re after, if you:

    • Don’t run from a debugger (disable the IDE debugger or run from the command line)

    • Use Debuggable(false, true) (turn off the generation of the .PDB file)

    So just make sure you compile without enabling the PDB file (Project | Options | Linker | [ ] Generate .PDB debug info file), or from the command-line use the –V- option. Now the compiler will emit Debuggable (False, True). Now Reflector only shows one instance of the Debuggable attribute (assuming you reverted the failed hack above). And the code runs fast from the IDE (with debugging disabled) and from the command line.


    Even if you compile with the IDE debugger enabled, the code will still run fast when running standalone. So the hack wasn’t really needed – just use the right options before compiling the release version (turning off the external debug file), and you will get optimized JITed code. Notice that deleting the .PDB file after compiling has no effect. It is the value of the first parameter to DebuggableAttribute that matters, and the same compiler option controls both.

    Monday, May 24, 2004

    Hack #2: Fixing System.pas line numbers after D7.1 update

    There has been some discussion in the newsgroups about the Delphi 7.1 update and the apparent mismatch between the patched System.dcu and the original System.pas source unit. Unfortunately, Borland did not include it in the original D7.1 patch, but I understand they are planning to include the correct version soon

    . The problem occurs when you try to step into RTL code that resides in the System unit. For instance, fire up a new Console project (File | New | Other | Console Application - yeah, we D7'ers do envy you D8'ers the File | New | Customize... command) and add the following high-tech code:

    program Project1; 
    {$APPTYPE CONSOLE}

    begin
    Writeln('Look - no hands, ma!');
    end.

    Now enable the RTL/VCL units with debug info enabled by selecting Project | Options | Compiler | [X] Use Debug DCUs. Put the cursor on the Writeln line and press F4 to run to cursor (assuming you use the 'correct' keyboard bindings). Then press F7 to step into the code. With the D7.1 update applied you'll now find yourself at the end of the _LStrOfChar procedure inside System.pas. In other words, you're lost.


    You'll also see that the blue dots signifying breakpointable code-lines are not aligned correctly with the actual source code. This is a typical sign of a mismatch between the source code and the actually compiled code. So until Borland gets the correct version of System.pas published in an updated update, we are stuck.


    Or are we? Hackers like us don't like twiddling our thumbs, declaring defeat. If you look a little more closely at the mis-aligned blue dots and the wrongly positioned cursor in System.pas (it should have been down in Write0LString a few lines below), we can quickly determine how many lines we're off.


    To get it right, we have to delete some lines - 8 lines, actually. But what should we delete?. If we scroll back to the top of the System.pas unit, we find some interesting, but less-than-useful comments about the GNU GPL license. As luck should have it, there are 8 lines in there. So, I delete them. And re-run the test application. Lo and behold, we now end up were we expect - in the Write0LString function! Great! A quick testing shows that other System routines now map to the correct line numbers too. Neat!


    The conclusions are:



    • The original D7.0 System.pas included that extra 8 lines with GNU GPL info

    • The D7.1 compiled System.dcu was apparently compiled with a version of System.pas without that extra comment header

    • The D7.1 update did *not* include the updated System.pas

    • Apparently, no other line-number changing modifications were made to System.pas

    And finally: This is only a short-term stop-gap measure until the official update with the correct System.pas is available from Borland. That file probably will contain other changes as well (otherwise System.dcu wouldn't have been updated in the first place).


    Update: The System unit was updated due to a 1K buffer-size issue in LoadResString, see BDN article.

    Sunday, May 23, 2004

    Hack #1: Write access to a read-only property

    The other day I was faced with the task of making our main application behave better systems with different screen resolutions (or rather pixel density, as in pixels per inch). This is the classic Large Font/Small Font problem, and getting forms and dialogs to scale properly to show readable fonts and text on all displays. There are several things to keep in mind, some of them are covered here - there are more complications due to MDI windows and Form inheritance.

    To make my testing easier (and possibly to let the end-user override the default scaling behavior) I decided to let the current screen density (as determined by Screen.PixelsPerInch) be controlled from a setting in the Registry. The built-in Delphi form scaling works reasonably well, and relies on the fact that the form's design-time PixelsPerInch value is different form the run-time Screen.PixelsPerInch value. Now, PixelsPerInch is a read-only, public property of the singleton TScreen class. It is initialized in the TScreen constructor to the number of vertical pixels per inch as returned by the current graphics driver:

      DC := GetDC(0); 
    FPixelsPerInch := GetDeviceCaps(DC, LOGPIXELSY);

    For my testing purposes, I wanted to set the value of the PixelsPerInch property without going to the hassle of actually changing my system setup, but to do that I would somehow have to modify the value of the read-only property. Impossible, right? Well, in software, nothing is really impossible. Software is soft, so we can change it :-). Changing the declaration of TScreen to make the property writeable would work, but as Allen has pointed out, making changes in the interface section of RTL and VCL units can have cascading effects, that are often undesirable. Besides, that would not really qualify as a bona-fide hack - it would have been too easy. Nah, lets do something a little more fun ;P. PixelsPerInch is only a public property, so there is no RTTI for it. Lets declare a descendent class, that promotes the property to published:

    type 
    TScreenEx = class(TScreen)
    published
    property PixelsPerInch;
    end;

    Now, since TScreen indirectly inherits from TPersistent, and TPersistent was compiled in the $M+ mode, published properties in our TScreenEx class will have RTTI generated for them. But PixelsPerInch is still a read-only property - and there is no way our TScreenEx can make it writeable, because the backing field FPixelsPerInch is private, not protected, and so is off-limits for us.


    The cunning thing about the RTTI generated for the TScreenEx.PixelsPerInch property, is that it includes enough information about where to find the backing field in the object instance. Open TypInfo.pas and locate the TPropInfo record that describes the RTTI for each property. Included is the GetProc pointer. For backing fields, this contains the offset off the field in the object instance (sans some flag bits). After decoding this offset and adding it to the base address of the object instance, we now can get a pointer to the backing field and thus modify it - voila write-access! Here is the short version:

    procedure SetPixelsPerInch(Value: integer); 
    begin
    PInteger(Integer(Screen) + (Integer(GetPropInfo(TScreenEx, 'PixelsPerInch').GetProc) and $00FFFFFF))^ := Value;
    end;

    Decoding that is left as an exercisecise for the reader.

    Thursday, May 13, 2004

    First blog post

    The blogmania never seems to stop, so now even I have been enticed into trying this new fad... :)



    Copyright © 2004-2007 by Hallvard Vassbotn